2024-09-16T14:37:37.048 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T14:37:37.066 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T14:37:37.163 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764 branch: reef description: rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906764' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 2500 ms tcp read timeout: 5 osd_object_clean_region_max_num_intervals: 1000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.1 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 6 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - slow request - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: ae7f37330c1a72493684eb1fcaed6f2a01195bb6 owner: scheduled_teuthology@teuthology priority: 931 repo: https://git.ceph.com/ceph.git roles: - - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - mon.a - - osd.4 - osd.5 - osd.6 - osd.7 - mon.b - mgr.x - - osd.8 - osd.9 - osd.10 - osd.11 - mon.c seed: 1390 sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 sleep_before_teardown: 0 subset: 5319/100000 suite: rados suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ae7f37330c1a72493684eb1fcaed6f2a01195bb6 targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFh27YyxVqFLxa26xlc0MXHu8TQcmlM5mzKb+6IJL+5SX5FmxZpYD5tJ0Zg2QmbXS6qsnc52ZeyjJyWcg3GijD8= smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA7VsOOTMTAokhrlWnkWq1YXVXoL8Ajq6J8LAAXEnE78TtiIfL4rQzlyobhKhV6jQAmKBnzd0whMZIGJZMv2Gpc= smithi114.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDAXtHXk2eiVWvBYb5Ctqzzm6lcQQ/gJDA46C9tOu43pC/bkUgPPNSQcWYZPmDg9ig0jaRBff65WMtESNXuJajs= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: bluestore_new_sharding: random chance_bluestore_reshard: 1 chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 min_in: 8 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: crush-failure-domain: osd k: 4 l: 3 m: 2 name: lrcprofile plugin: lrc objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 400 write_append_excl: false teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_22:00:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T14:37:37.163 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa; will attempt to use it 2024-09-16T14:37:37.164 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa/tasks 2024-09-16T14:37:37.164 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T14:37:37.165 INFO:teuthology.task.internal:Checking packages... 2024-09-16T14:37:37.187 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8' 2024-09-16T14:37:37.187 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T14:37:37.187 INFO:teuthology.packaging:ref: None 2024-09-16T14:37:37.187 INFO:teuthology.packaging:tag: None 2024-09-16T14:37:37.187 INFO:teuthology.packaging:branch: reef 2024-09-16T14:37:37.187 INFO:teuthology.packaging:sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:37:37.187 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-16T14:37:37.441 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-853.gae7f3733 2024-09-16T14:37:37.442 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T14:37:37.448 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T14:37:37.448 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T14:37:37.502 INFO:teuthology.task.internal:Saving configuration 2024-09-16T14:37:37.517 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T14:37:37.544 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T14:37:37.570 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 14:32:03.611123', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFh27YyxVqFLxa26xlc0MXHu8TQcmlM5mzKb+6IJL+5SX5FmxZpYD5tJ0Zg2QmbXS6qsnc52ZeyjJyWcg3GijD8='} 2024-09-16T14:37:37.591 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 14:32:03.613549', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA7VsOOTMTAokhrlWnkWq1YXVXoL8Ajq6J8LAAXEnE78TtiIfL4rQzlyobhKhV6jQAmKBnzd0whMZIGJZMv2Gpc='} 2024-09-16T14:37:37.611 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 14:32:03.612456', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDAXtHXk2eiVWvBYb5Ctqzzm6lcQQ/gJDA46C9tOu43pC/bkUgPPNSQcWYZPmDg9ig0jaRBff65WMtESNXuJajs='} 2024-09-16T14:37:37.611 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T14:37:37.619 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a'] 2024-09-16T14:37:37.619 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x'] 2024-09-16T14:37:37.620 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c'] 2024-09-16T14:37:37.620 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T14:37:37.700 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1c3f6cfac0>, signals=[15]) 2024-09-16T14:37:37.700 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T14:37:37.707 INFO:teuthology.task.internal:Opening connections... 2024-09-16T14:37:37.708 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2024-09-16T14:37:37.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:37:37.787 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2024-09-16T14:37:37.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:37:37.864 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2024-09-16T14:37:37.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:37:37.940 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T14:37:37.947 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2024-09-16T14:37:37.965 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2024-09-16T14:37:37.965 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2024-09-16T14:37:38.023 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2024-09-16T14:37:38.023 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2024-09-16T14:37:38.024 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2024-09-16T14:37:38.024 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2024-09-16T14:37:38.024 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2024-09-16T14:37:38.024 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2024-09-16T14:37:38.024 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T14:37:38.024 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2024-09-16T14:37:38.024 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2024-09-16T14:37:38.024 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T14:37:38.024 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2024-09-16T14:37:38.024 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T14:37:38.024 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T14:37:38.024 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T14:37:38.025 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2024-09-16T14:37:38.043 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2024-09-16T14:37:38.061 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2024-09-16T14:37:38.062 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2024-09-16T14:37:38.120 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2024-09-16T14:37:38.120 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="9" 2024-09-16T14:37:38.120 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2024-09-16T14:37:38.120 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2024-09-16T14:37:38.120 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="9" 2024-09-16T14:37:38.120 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el9" 2024-09-16T14:37:38.120 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T14:37:38.121 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2024-09-16T14:37:38.121 INFO:teuthology.orchestra.run.smithi111.stdout:LOGO="fedora-logo-icon" 2024-09-16T14:37:38.121 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T14:37:38.121 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2024-09-16T14:37:38.121 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T14:37:38.121 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T14:37:38.121 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T14:37:38.121 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2024-09-16T14:37:38.143 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2024-09-16T14:37:38.162 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2024-09-16T14:37:38.162 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2024-09-16T14:37:38.221 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream" 2024-09-16T14:37:38.221 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9" 2024-09-16T14:37:38.221 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2024-09-16T14:37:38.221 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2024-09-16T14:37:38.221 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9" 2024-09-16T14:37:38.221 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9" 2024-09-16T14:37:38.221 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T14:37:38.221 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2024-09-16T14:37:38.221 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon" 2024-09-16T14:37:38.221 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T14:37:38.222 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/" 2024-09-16T14:37:38.222 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T14:37:38.222 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T14:37:38.222 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T14:37:38.222 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2024-09-16T14:37:38.243 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T14:37:38.252 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T14:37:38.259 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T14:37:38.259 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2024-09-16T14:37:38.262 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2024-09-16T14:37:38.264 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2024-09-16T14:37:38.281 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T14:37:38.287 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T14:37:38.288 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2024-09-16T14:37:38.319 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2024-09-16T14:37:38.321 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2024-09-16T14:37:38.354 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T14:37:38.432 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T14:37:38.448 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T14:37:38.448 INFO:teuthology.task.kernel:config {'osd.0': {'kdb': 1, 'sha1': 'distro'}, 'osd.4': {'kdb': 1, 'sha1': 'distro'}, 'osd.8': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T14:37:38.448 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T14:37:38.449 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T14:37:38.449 DEBUG:teuthology.orchestra.run.smithi114:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T14:37:38.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:37:38.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:37:38.467 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-09-16T14:37:38.467 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-09-16T14:37:38.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:37:38.468 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-09-16T14:37:38.525 INFO:teuthology.orchestra.run.smithi111.stdout:5.14.0-503.el9.x86_64 2024-09-16T14:37:38.525 INFO:teuthology.task.kernel:Running kernel on smithi111: 5.14.0-503.el9.x86_64 2024-09-16T14:37:38.525 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum install -y kernel 2024-09-16T14:37:38.527 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-503.el9.x86_64 2024-09-16T14:37:38.527 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-503.el9.x86_64 2024-09-16T14:37:38.528 INFO:teuthology.task.kernel:Running kernel on smithi114: 5.14.0-503.el9.x86_64 2024-09-16T14:37:38.528 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum install -y kernel 2024-09-16T14:37:38.528 INFO:teuthology.task.kernel:Running kernel on smithi040: 5.14.0-503.el9.x86_64 2024-09-16T14:37:38.528 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum install -y kernel 2024-09-16T14:37:40.996 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-16T14:37:41.047 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.2 MB 00:00 2024-09-16T14:37:42.010 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-16T14:37:42.680 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-09-16T14:37:43.986 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-16T14:37:46.947 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-16T14:37:46.964 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 4.2 MB/s | 20 MB 00:04 2024-09-16T14:37:48.573 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-16T14:37:49.012 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.5 MB 00:01 2024-09-16T14:37:49.986 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-16T14:37:50.462 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 93 kB/s | 19 kB 00:00 2024-09-16T14:37:51.841 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-16T14:37:52.523 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-16T14:37:54.407 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-16T14:37:55.419 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-16T14:37:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T14:37:57.431 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T14:37:57.431 INFO:teuthology.orchestra.run.smithi114.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T14:37:57.447 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-16T14:37:57.517 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:37:57.523 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:37:57.523 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-09-16T14:37:57.523 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:37:57.523 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:37:57.523 INFO:teuthology.orchestra.run.smithi114.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T14:37:57.524 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-16T14:37:57.524 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T14:37:57.524 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T14:37:57.524 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T14:37:57.524 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:37:57.524 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:37:57.524 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:37:57.524 INFO:teuthology.orchestra.run.smithi114.stdout:Install 4 Packages 2024-09-16T14:37:57.524 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:37:57.524 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 79 M 2024-09-16T14:37:57.524 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 126 M 2024-09-16T14:37:57.524 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:37:57.751 INFO:teuthology.orchestra.run.smithi114.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 235 kB/s | 23 kB 00:00 2024-09-16T14:37:58.143 INFO:teuthology.orchestra.run.smithi114.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-16T14:37:58.568 INFO:teuthology.orchestra.run.smithi114.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-16T14:37:58.827 INFO:teuthology.orchestra.run.smithi114.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 27 MB/s | 29 MB 00:01 2024-09-16T14:37:58.827 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:37:58.827 INFO:teuthology.orchestra.run.smithi114.stdout:Total 60 MB/s | 79 MB 00:01 2024-09-16T14:37:59.487 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T14:37:59.487 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T14:37:59.510 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:37:59.580 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:37:59.589 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:37:59.589 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-09-16T14:37:59.589 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:37:59.589 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:37:59.589 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T14:37:59.590 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-16T14:37:59.590 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T14:37:59.590 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T14:37:59.590 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T14:37:59.590 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:37:59.590 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:37:59.590 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:37:59.590 INFO:teuthology.orchestra.run.smithi040.stdout:Install 4 Packages 2024-09-16T14:37:59.590 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:37:59.591 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 79 M 2024-09-16T14:37:59.591 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 126 M 2024-09-16T14:37:59.591 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:37:59.715 INFO:teuthology.orchestra.run.smithi040.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 288 kB/s | 23 kB 00:00 2024-09-16T14:37:59.860 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:37:59.860 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:38:00.666 INFO:teuthology.orchestra.run.smithi040.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 30 MB/s | 29 MB 00:00 2024-09-16T14:38:00.821 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:38:00.821 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:38:00.859 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-16T14:38:01.349 INFO:teuthology.orchestra.run.smithi040.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 9.1 MB/s | 16 MB 00:01 2024-09-16T14:38:02.341 INFO:teuthology.orchestra.run.smithi040.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 13 MB/s | 35 MB 00:02 2024-09-16T14:38:02.341 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:38:02.341 INFO:teuthology.orchestra.run.smithi040.stdout:Total 29 MB/s | 79 MB 00:02 2024-09-16T14:38:02.890 INFO:teuthology.orchestra.run.smithi111.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T14:38:02.890 INFO:teuthology.orchestra.run.smithi111.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T14:38:02.991 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:38:03.002 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:38:03.002 INFO:teuthology.orchestra.run.smithi111.stdout: Package Architecture Version Repository Size 2024-09-16T14:38:03.002 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:38:03.002 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:38:03.002 INFO:teuthology.orchestra.run.smithi111.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T14:38:03.002 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-16T14:38:03.002 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T14:38:03.002 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T14:38:03.002 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T14:38:03.002 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:38:03.002 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:38:03.003 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:38:03.003 INFO:teuthology.orchestra.run.smithi111.stdout:Install 4 Packages 2024-09-16T14:38:03.003 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:38:03.003 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 79 M 2024-09-16T14:38:03.003 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 126 M 2024-09-16T14:38:03.003 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:38:03.054 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:38:03.135 INFO:teuthology.orchestra.run.smithi111.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 263 kB/s | 23 kB 00:00 2024-09-16T14:38:03.266 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:38:03.425 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:38:03.425 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:38:03.925 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T14:38:04.011 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:38:04.395 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:38:04.395 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:38:06.518 INFO:teuthology.orchestra.run.smithi111.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 4.5 MB/s | 16 MB 00:03 2024-09-16T14:38:06.781 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:38:06.904 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:38:06.961 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:38:07.535 INFO:teuthology.orchestra.run.smithi111.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.5 MB/s | 29 MB 00:04 2024-09-16T14:38:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T14:38:07.783 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:38:09.085 INFO:teuthology.orchestra.run.smithi111.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 5.7 MB/s | 35 MB 00:06 2024-09-16T14:38:09.086 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:38:09.086 INFO:teuthology.orchestra.run.smithi111.stdout:Total 13 MB/s | 79 MB 00:06 2024-09-16T14:38:09.767 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:38:10.141 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:38:10.142 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:38:10.553 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:38:10.709 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:38:11.085 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:38:11.086 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:38:11.752 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:38:11.871 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:38:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:38:14.178 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T14:38:14.294 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:38:15.837 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:38:15.880 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:38:16.726 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:38:17.085 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:38:17.213 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:38:21.015 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:38:22.022 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:38:22.063 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:38:26.915 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:38:55.065 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:38:55.110 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:38:55.441 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:38:55.441 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T14:38:55.441 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:38:55.441 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:38:56.368 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:38:56.368 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:38:56.369 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:38:56.369 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T14:38:56.369 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T14:38:56.369 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T14:38:56.369 INFO:teuthology.orchestra.run.smithi114.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T14:38:56.369 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:38:56.369 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:38:56.706 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T14:38:56.790 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T14:38:56.790 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T14:38:56.790 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T14:38:56.791 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T14:38:56.791 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T14:38:56.791 INFO:teuthology.task.kernel:Installing distro kernel on osd.8... 2024-09-16T14:38:56.791 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi114.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T14:38:56.791 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T14:38:56.850 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T14:38:56.850 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T14:38:56.850 DEBUG:teuthology.orchestra.run.smithi114:> sudo rpm -qi grub2-tools 2024-09-16T14:38:56.885 INFO:teuthology.orchestra.run.smithi114.stdout:Name : grub2-tools 2024-09-16T14:38:56.886 INFO:teuthology.orchestra.run.smithi114.stdout:Epoch : 1 2024-09-16T14:38:56.886 INFO:teuthology.orchestra.run.smithi114.stdout:Version : 2.06 2024-09-16T14:38:56.886 INFO:teuthology.orchestra.run.smithi114.stdout:Release : 82.el9 2024-09-16T14:38:56.886 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: x86_64 2024-09-16T14:38:56.886 INFO:teuthology.orchestra.run.smithi114.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T14:38:56.886 INFO:teuthology.orchestra.run.smithi114.stdout:Group : Unspecified 2024-09-16T14:38:56.886 INFO:teuthology.orchestra.run.smithi114.stdout:Size : 8274670 2024-09-16T14:38:56.886 INFO:teuthology.orchestra.run.smithi114.stdout:License : GPLv3+ 2024-09-16T14:38:56.886 INFO:teuthology.orchestra.run.smithi114.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T14:38:56.886 INFO:teuthology.orchestra.run.smithi114.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T14:38:56.886 INFO:teuthology.orchestra.run.smithi114.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T14:38:56.886 INFO:teuthology.orchestra.run.smithi114.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T14:38:56.886 INFO:teuthology.orchestra.run.smithi114.stdout:Packager : builder@centos.org 2024-09-16T14:38:56.887 INFO:teuthology.orchestra.run.smithi114.stdout:Vendor : CentOS 2024-09-16T14:38:56.887 INFO:teuthology.orchestra.run.smithi114.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T14:38:56.887 INFO:teuthology.orchestra.run.smithi114.stdout:Summary : Support tools for GRUB. 2024-09-16T14:38:56.887 INFO:teuthology.orchestra.run.smithi114.stdout:Description : 2024-09-16T14:38:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:38:56.887 INFO:teuthology.orchestra.run.smithi114.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T14:38:56.887 INFO:teuthology.orchestra.run.smithi114.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T14:38:56.887 INFO:teuthology.orchestra.run.smithi114.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T14:38:56.887 INFO:teuthology.orchestra.run.smithi114.stdout:hardware devices. 2024-09-16T14:38:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:38:56.887 INFO:teuthology.orchestra.run.smithi114.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T14:38:56.888 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T14:38:56.888 INFO:teuthology.task.kernel:Updating grub on smithi114 to boot 5.14.0-505.el9.x86_64 2024-09-16T14:38:56.889 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T14:38:57.517 INFO:teuthology.orchestra.run.smithi114.stderr:Generating grub configuration file ... 2024-09-16T14:38:58.865 INFO:teuthology.orchestra.run.smithi114.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T14:38:58.904 INFO:teuthology.orchestra.run.smithi114.stderr:done 2024-09-16T14:38:58.907 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-09-16T14:38:58.922 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.3kJh4jibCa 2024-09-16T14:38:58.922 DEBUG:teuthology.orchestra.run.smithi114:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3kJh4jibCa 2024-09-16T14:38:58.988 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.3kJh4jibCa 2024-09-16T14:38:59.118 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.3kJh4jibCa is 6KB 2024-09-16T14:38:59.139 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.3kJh4jibCa 2024-09-16T14:38:59.153 DEBUG:teuthology.orchestra.run.smithi114:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T14:38:59.216 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T14:38:59.217 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T14:38:59.217 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T14:38:59.217 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T14:38:59.218 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T14:38:59.374 DEBUG:teuthology.orchestra.run.smithi114:> sudo shutdown -r now 2024-09-16T14:39:00.395 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:39:00.436 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:39:00.746 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:39:00.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T14:39:00.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:39:00.746 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:39:01.599 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:39:01.600 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:39:01.600 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:39:01.600 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T14:39:01.600 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T14:39:01.600 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T14:39:01.600 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T14:39:01.600 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:39:01.600 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:39:01.938 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T14:39:02.033 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T14:39:02.034 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T14:39:02.034 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T14:39:02.034 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T14:39:02.034 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T14:39:02.034 INFO:teuthology.task.kernel:Installing distro kernel on osd.0... 2024-09-16T14:39:02.034 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi040.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T14:39:02.034 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T14:39:02.093 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T14:39:02.094 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T14:39:02.094 DEBUG:teuthology.orchestra.run.smithi040:> sudo rpm -qi grub2-tools 2024-09-16T14:39:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:Name : grub2-tools 2024-09-16T14:39:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:Epoch : 1 2024-09-16T14:39:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:Version : 2.06 2024-09-16T14:39:02.125 INFO:teuthology.orchestra.run.smithi040.stdout:Release : 82.el9 2024-09-16T14:39:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: x86_64 2024-09-16T14:39:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T14:39:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:Group : Unspecified 2024-09-16T14:39:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:Size : 8274670 2024-09-16T14:39:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:License : GPLv3+ 2024-09-16T14:39:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T14:39:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T14:39:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T14:39:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T14:39:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:Packager : builder@centos.org 2024-09-16T14:39:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:Vendor : CentOS 2024-09-16T14:39:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T14:39:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:Summary : Support tools for GRUB. 2024-09-16T14:39:02.126 INFO:teuthology.orchestra.run.smithi040.stdout:Description : 2024-09-16T14:39:02.127 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:39:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T14:39:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T14:39:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T14:39:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:hardware devices. 2024-09-16T14:39:02.127 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:39:02.127 INFO:teuthology.orchestra.run.smithi040.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T14:39:02.128 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T14:39:02.128 INFO:teuthology.task.kernel:Updating grub on smithi040 to boot 5.14.0-505.el9.x86_64 2024-09-16T14:39:02.128 DEBUG:teuthology.orchestra.run.smithi040:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T14:39:02.833 INFO:teuthology.orchestra.run.smithi040.stderr:Generating grub configuration file ... 2024-09-16T14:39:04.152 INFO:teuthology.orchestra.run.smithi040.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T14:39:04.182 INFO:teuthology.orchestra.run.smithi040.stderr:done 2024-09-16T14:39:04.185 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-09-16T14:39:04.200 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.nhsQnWWuDr 2024-09-16T14:39:04.200 DEBUG:teuthology.orchestra.run.smithi040:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nhsQnWWuDr 2024-09-16T14:39:04.266 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.nhsQnWWuDr 2024-09-16T14:39:04.405 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.nhsQnWWuDr is 6KB 2024-09-16T14:39:04.454 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.nhsQnWWuDr 2024-09-16T14:39:04.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T14:39:04.539 INFO:teuthology.orchestra.run.smithi040.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T14:39:04.539 INFO:teuthology.orchestra.run.smithi040.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T14:39:04.539 INFO:teuthology.orchestra.run.smithi040.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T14:39:04.539 INFO:teuthology.orchestra.run.smithi040.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T14:39:04.541 DEBUG:teuthology.orchestra.run.smithi040:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T14:39:04.707 DEBUG:teuthology.orchestra.run.smithi040:> sudo shutdown -r now 2024-09-16T14:39:05.388 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:39:05.429 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:39:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:39:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T14:39:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T14:39:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T14:39:06.621 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T14:39:06.621 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:39:06.621 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:39:06.621 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T14:39:06.621 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T14:39:06.621 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T14:39:06.621 INFO:teuthology.orchestra.run.smithi111.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T14:39:06.622 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:39:06.622 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:39:06.983 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T14:39:07.070 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T14:39:07.070 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T14:39:07.070 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T14:39:07.071 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T14:39:07.071 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T14:39:07.071 INFO:teuthology.task.kernel:Installing distro kernel on osd.4... 2024-09-16T14:39:07.071 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi111.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T14:39:07.071 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T14:39:07.130 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T14:39:07.130 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T14:39:07.130 DEBUG:teuthology.orchestra.run.smithi111:> sudo rpm -qi grub2-tools 2024-09-16T14:39:07.171 INFO:teuthology.orchestra.run.smithi111.stdout:Name : grub2-tools 2024-09-16T14:39:07.171 INFO:teuthology.orchestra.run.smithi111.stdout:Epoch : 1 2024-09-16T14:39:07.171 INFO:teuthology.orchestra.run.smithi111.stdout:Version : 2.06 2024-09-16T14:39:07.171 INFO:teuthology.orchestra.run.smithi111.stdout:Release : 82.el9 2024-09-16T14:39:07.171 INFO:teuthology.orchestra.run.smithi111.stdout:Architecture: x86_64 2024-09-16T14:39:07.171 INFO:teuthology.orchestra.run.smithi111.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T14:39:07.171 INFO:teuthology.orchestra.run.smithi111.stdout:Group : Unspecified 2024-09-16T14:39:07.171 INFO:teuthology.orchestra.run.smithi111.stdout:Size : 8274670 2024-09-16T14:39:07.171 INFO:teuthology.orchestra.run.smithi111.stdout:License : GPLv3+ 2024-09-16T14:39:07.172 INFO:teuthology.orchestra.run.smithi111.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T14:39:07.172 INFO:teuthology.orchestra.run.smithi111.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T14:39:07.172 INFO:teuthology.orchestra.run.smithi111.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T14:39:07.172 INFO:teuthology.orchestra.run.smithi111.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T14:39:07.172 INFO:teuthology.orchestra.run.smithi111.stdout:Packager : builder@centos.org 2024-09-16T14:39:07.172 INFO:teuthology.orchestra.run.smithi111.stdout:Vendor : CentOS 2024-09-16T14:39:07.172 INFO:teuthology.orchestra.run.smithi111.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T14:39:07.172 INFO:teuthology.orchestra.run.smithi111.stdout:Summary : Support tools for GRUB. 2024-09-16T14:39:07.172 INFO:teuthology.orchestra.run.smithi111.stdout:Description : 2024-09-16T14:39:07.172 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:39:07.172 INFO:teuthology.orchestra.run.smithi111.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T14:39:07.172 INFO:teuthology.orchestra.run.smithi111.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T14:39:07.172 INFO:teuthology.orchestra.run.smithi111.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T14:39:07.173 INFO:teuthology.orchestra.run.smithi111.stdout:hardware devices. 2024-09-16T14:39:07.173 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:39:07.173 INFO:teuthology.orchestra.run.smithi111.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T14:39:07.173 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T14:39:07.174 INFO:teuthology.task.kernel:Updating grub on smithi111 to boot 5.14.0-505.el9.x86_64 2024-09-16T14:39:07.174 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T14:39:07.873 INFO:teuthology.orchestra.run.smithi111.stderr:Generating grub configuration file ... 2024-09-16T14:39:09.163 INFO:teuthology.orchestra.run.smithi111.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T14:39:09.191 INFO:teuthology.orchestra.run.smithi111.stderr:done 2024-09-16T14:39:09.194 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2024-09-16T14:39:09.209 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.VvpmJqmFd5 2024-09-16T14:39:09.210 DEBUG:teuthology.orchestra.run.smithi111:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VvpmJqmFd5 2024-09-16T14:39:09.281 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.VvpmJqmFd5 2024-09-16T14:39:09.415 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.VvpmJqmFd5 is 6KB 2024-09-16T14:39:09.463 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.VvpmJqmFd5 2024-09-16T14:39:09.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T14:39:09.541 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T14:39:09.541 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T14:39:09.541 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T14:39:09.541 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T14:39:09.543 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T14:39:09.708 DEBUG:teuthology.orchestra.run.smithi111:> sudo shutdown -r now 2024-09-16T14:39:29.392 INFO:teuthology.task.kernel:Checking client osd.8 for new kernel version... 2024-09-16T14:39:29.393 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-16T14:39:29.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:39:34.715 INFO:teuthology.task.kernel:Checking client osd.0 for new kernel version... 2024-09-16T14:39:34.715 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-16T14:39:34.716 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:39:39.717 INFO:teuthology.task.kernel:Checking client osd.4 for new kernel version... 2024-09-16T14:39:39.717 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-16T14:39:39.718 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:40:14.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-09-16T14:40:23.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-16T14:40:23.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:40:26.461 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-09-16T14:40:29.398 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T14:40:34.723 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T14:40:38.403 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-16T14:40:38.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:40:38.508 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-16T14:40:38.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:40:39.080 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-09-16T14:40:39.507 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-16T14:40:39.507 INFO:teuthology.task.kernel:Checking kernel version of osd.8, want "5.14.0-505.el9.x86_64"... 2024-09-16T14:40:39.507 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-09-16T14:40:39.522 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-505.el9.x86_64 2024-09-16T14:40:39.522 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T14:40:39.522 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T14:40:39.522 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T14:40:40.523 INFO:teuthology.task.kernel:Enabling kdb on osd.8... 2024-09-16T14:40:40.524 DEBUG:teuthology.orchestra.run.smithi114:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T14:40:40.615 INFO:teuthology.orchestra.run.smithi114.stdout:ttyS1 2024-09-16T14:40:40.644 DEBUG:teuthology.parallel:result is None 2024-09-16T14:40:43.724 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-16T14:40:43.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:40:44.136 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-09-16T14:40:44.518 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-16T14:40:44.519 INFO:teuthology.task.kernel:Checking kernel version of osd.0, want "5.14.0-505.el9.x86_64"... 2024-09-16T14:40:44.519 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-09-16T14:40:44.534 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-505.el9.x86_64 2024-09-16T14:40:44.534 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T14:40:44.534 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T14:40:44.535 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T14:40:45.535 INFO:teuthology.task.kernel:Enabling kdb on osd.0... 2024-09-16T14:40:45.536 DEBUG:teuthology.orchestra.run.smithi040:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T14:40:45.606 INFO:teuthology.orchestra.run.smithi040.stdout:ttyS1 2024-09-16T14:40:45.631 DEBUG:teuthology.parallel:result is None 2024-09-16T14:40:56.893 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-09-16T14:41:11.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-16T14:41:11.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:41:12.297 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-09-16T14:41:12.697 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-16T14:41:12.697 INFO:teuthology.task.kernel:Checking kernel version of osd.4, want "5.14.0-505.el9.x86_64"... 2024-09-16T14:41:12.697 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-09-16T14:41:12.711 INFO:teuthology.orchestra.run.smithi111.stdout:5.14.0-505.el9.x86_64 2024-09-16T14:41:12.712 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T14:41:12.712 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T14:41:12.712 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T14:41:13.712 INFO:teuthology.task.kernel:Enabling kdb on osd.4... 2024-09-16T14:41:13.713 DEBUG:teuthology.orchestra.run.smithi111:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T14:41:13.815 INFO:teuthology.orchestra.run.smithi111.stdout:ttyS1 2024-09-16T14:41:13.841 DEBUG:teuthology.parallel:result is None 2024-09-16T14:41:13.842 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T14:41:13.849 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T14:41:13.849 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T14:41:13.853 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T14:41:13.855 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T14:41:13.875 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T14:41:13.884 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T14:41:13.893 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T14:41:13.894 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T14:41:13.913 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T14:41:13.915 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T14:41:13.959 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T14:41:13.967 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T14:41:13.967 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T14:41:13.978 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T14:41:13.980 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T14:41:14.002 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:41:14.012 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:41:14.016 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:41:14.027 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:41:14.034 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:41:14.045 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T14:41:14.047 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T14:41:14.054 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T14:41:14.054 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T14:41:14.071 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T14:41:14.074 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T14:41:14.114 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T14:41:14.130 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T14:41:14.130 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T14:41:14.140 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T14:41:14.146 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T14:41:14.172 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:41:14.243 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:41:14.292 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:41:14.377 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:41:14.418 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:41:14.418 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T14:41:14.485 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:41:14.527 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:41:14.592 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:41:14.644 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:41:14.731 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:41:14.731 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T14:41:14.803 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:41:14.845 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T14:41:14.906 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:41:14.992 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T14:41:15.036 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:41:15.036 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T14:41:15.103 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2024-09-16T14:41:15.106 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2024-09-16T14:41:15.111 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2024-09-16T14:41:15.150 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T14:41:15.196 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T14:41:15.210 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T14:41:15.716 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T14:41:15.724 INFO:teuthology.task.internal:Starting timer... 2024-09-16T14:41:15.724 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T14:41:15.737 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T14:41:15.748 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-16T14:41:15.748 DEBUG:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2024-09-16T14:41:15.824 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: 2024-09-16T14:41:15.826 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2024-09-16T14:41:16.115 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: 2024-09-16T14:41:16.118 DEBUG:teuthology.orchestra.run.smithi114:> sudo service auditd rotate 2024-09-16T14:41:16.182 INFO:teuthology.orchestra.run.smithi114.stdout:Rotating logs: 2024-09-16T14:41:16.184 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T14:41:16.184 DEBUG:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2024-09-16T14:41:16.213 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2024-09-16T14:41:16.213 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2024-09-16T14:41:16.272 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2024-09-16T14:41:16.272 DEBUG:teuthology.orchestra.run.smithi114:> /usr/sbin/getenforce 2024-09-16T14:41:16.298 INFO:teuthology.orchestra.run.smithi114.stdout:Permissive 2024-09-16T14:41:16.298 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi040.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi111.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi114.front.sepia.ceph.com': 'permissive'} 2024-09-16T14:41:16.298 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T14:41:16.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:41:16.326 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T14:41:16.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:41:16.357 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T14:41:16.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:41:16.383 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T14:41:16.383 DEBUG:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2024-09-16T14:41:16.429 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2024-09-16T14:41:16.510 DEBUG:teuthology.orchestra.run.smithi114:> sudo /usr/sbin/setenforce permissive 2024-09-16T14:41:16.539 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T14:41:16.548 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T14:41:16.559 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T14:41:16.560 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi111.front.sepia.ceph.com,smithi114.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T14:45:03.892 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi111.front.sepia.ceph.com'), Remote(name='ubuntu@smithi114.front.sepia.ceph.com')] 2024-09-16T14:45:03.894 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-16T14:45:03.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:45:03.977 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-09-16T14:45:04.053 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-16T14:45:04.054 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-16T14:45:04.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:45:04.132 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-09-16T14:45:04.207 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-16T14:45:04.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-16T14:45:04.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T14:45:04.284 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-09-16T14:45:04.365 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-16T14:45:04.366 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T14:45:04.376 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T14:45:04.377 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T14:45:04.378 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:45:04.380 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T14:45:04.380 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:45:04.382 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T14:45:04.382 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:45:04.409 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T14:45:04.412 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T14:45:04.423 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T14:45:04.429 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T14:45:04.436 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T14:45:04.447 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2024-09-16T14:45:04.451 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T14:45:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2024-09-16T14:45:04.458 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2024-09-16T14:45:04.468 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2024-09-16T14:45:04.472 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T14:45:04.476 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: ntpd: command not found 2024-09-16T14:45:04.483 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T14:45:04.486 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T14:45:04.487 INFO:teuthology.orchestra.run.smithi114.stdout:506 Cannot talk to daemon 2024-09-16T14:45:04.498 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T14:45:04.502 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T14:45:04.516 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T14:45:04.544 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-09-16T14:45:04.547 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T14:45:04.547 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-09-16T14:45:04.547 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:45:04.547 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:45:04.547 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:45:04.548 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:45:04.554 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-09-16T14:45:04.557 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T14:45:04.557 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-09-16T14:45:04.557 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:45:04.557 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:45:04.558 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:45:04.558 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:45:04.569 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-09-16T14:45:04.572 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T14:45:04.573 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-09-16T14:45:04.573 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:45:04.573 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:45:04.573 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:45:04.573 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:45:04.574 INFO:teuthology.run_tasks:Running task install... 2024-09-16T14:45:04.582 DEBUG:teuthology.task.install:project ceph 2024-09-16T14:45:04.582 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}} 2024-09-16T14:45:04.582 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'} 2024-09-16T14:45:04.582 INFO:teuthology.task.install:Using flavor: default 2024-09-16T14:45:04.588 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T14:45:04.588 INFO:teuthology.task.install:extra packages: [] 2024-09-16T14:45:04.588 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'tag': None, 'wait_for_package': False} 2024-09-16T14:45:04.589 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:45:04.591 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'tag': None, 'wait_for_package': False} 2024-09-16T14:45:04.591 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:45:04.592 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'tag': None, 'wait_for_package': False} 2024-09-16T14:45:04.592 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:45:04.741 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/ 2024-09-16T14:45:04.741 INFO:teuthology.task.install.rpm:Package version is 18.2.4-845.ge4e227b3 2024-09-16T14:45:04.745 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/ 2024-09-16T14:45:04.745 INFO:teuthology.task.install.rpm:Package version is 18.2.4-845.ge4e227b3 2024-09-16T14:45:04.750 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/ 2024-09-16T14:45:04.750 INFO:teuthology.task.install.rpm:Package version is 18.2.4-845.ge4e227b3 2024-09-16T14:45:04.856 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T14:45:04.856 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:45:04.856 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T14:45:04.861 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T14:45:04.862 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:45:04.862 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T14:45:04.866 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T14:45:04.866 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:45:04.866 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T14:45:04.882 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T14:45:04.882 DEBUG:teuthology.orchestra.run.smithi114:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T14:45:04.888 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T14:45:04.888 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T14:45:04.893 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T14:45:04.893 DEBUG:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T14:45:04.947 DEBUG:teuthology.orchestra.run.smithi114:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T14:45:04.953 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T14:45:04.960 DEBUG:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T14:45:05.022 DEBUG:teuthology.orchestra.run.smithi114:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T14:45:05.029 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T14:45:05.036 DEBUG:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T14:45:05.086 INFO:teuthology.orchestra.run.smithi114.stdout:check_obsoletes = 1 2024-09-16T14:45:05.087 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum clean all 2024-09-16T14:45:05.093 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2024-09-16T14:45:05.094 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2024-09-16T14:45:05.100 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2024-09-16T14:45:05.102 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum clean all 2024-09-16T14:45:05.319 INFO:teuthology.orchestra.run.smithi111.stdout:45 files removed 2024-09-16T14:45:05.323 INFO:teuthology.orchestra.run.smithi114.stdout:45 files removed 2024-09-16T14:45:05.339 INFO:teuthology.orchestra.run.smithi040.stdout:45 files removed 2024-09-16T14:45:05.345 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw 2024-09-16T14:45:05.350 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-radosgw 2024-09-16T14:45:05.367 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw 2024-09-16T14:45:05.896 INFO:teuthology.orchestra.run.smithi114.stdout:ceph packages for x86_64 296 kB/s | 77 kB 00:00 2024-09-16T14:45:05.900 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 288 kB/s | 77 kB 00:00 2024-09-16T14:45:05.925 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 298 kB/s | 77 kB 00:00 2024-09-16T14:45:06.176 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 49 kB/s | 11 kB 00:00 2024-09-16T14:45:06.179 INFO:teuthology.orchestra.run.smithi114.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-16T14:45:06.209 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-16T14:45:06.451 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-16T14:45:06.463 INFO:teuthology.orchestra.run.smithi114.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-09-16T14:45:06.485 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-16T14:45:06.938 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.2 MB 00:00 2024-09-16T14:45:06.968 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-16T14:45:07.085 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-16T14:45:08.897 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-16T14:45:08.957 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-16T14:45:09.119 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-16T14:45:13.395 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-16T14:45:13.406 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-16T14:45:14.908 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 85 kB/s | 19 kB 00:00 2024-09-16T14:45:14.913 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 19 kB 00:00 2024-09-16T14:45:16.386 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-16T14:45:16.408 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-16T14:45:17.132 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.5 MB 00:04 2024-09-16T14:45:18.800 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-16T14:45:20.212 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-16T14:45:21.773 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-16T14:45:21.975 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T14:45:24.098 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:45:24.117 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:45:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-16T14:45:24.117 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:45:24.117 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:45:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 7.7 M 2024-09-16T14:45:24.120 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2024-09-16T14:45:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.3 M 2024-09-16T14:45:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.0 M 2024-09-16T14:45:24.120 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-16T14:45:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T14:45:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 5.1 M 2024-09-16T14:45:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 18 M 2024-09-16T14:45:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 25 k 2024-09-16T14:45:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-16T14:45:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T14:45:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T14:45:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T14:45:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 712 k 2024-09-16T14:45:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T14:45:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T14:45:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T14:45:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 478 k 2024-09-16T14:45:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T14:45:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 4.5 M 2024-09-16T14:45:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 45 k 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 129 k 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 162 k 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 321 k 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 300 k 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 99 k 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:45:24.122 INFO:teuthology.orchestra.run.smithi111.stdout:Install 28 Packages 2024-09-16T14:45:24.123 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 2 Packages 2024-09-16T14:45:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:45:24.123 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 53 M 2024-09-16T14:45:24.123 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:45:24.436 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:45:24.453 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:45:24.453 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-16T14:45:24.453 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:45:24.453 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:45:24.453 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 7.7 M 2024-09-16T14:45:24.453 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2024-09-16T14:45:24.453 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.3 M 2024-09-16T14:45:24.453 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.0 M 2024-09-16T14:45:24.453 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-16T14:45:24.454 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T14:45:24.454 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 5.1 M 2024-09-16T14:45:24.454 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 18 M 2024-09-16T14:45:24.454 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 25 k 2024-09-16T14:45:24.454 INFO:teuthology.orchestra.run.smithi040.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-16T14:45:24.454 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T14:45:24.454 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T14:45:24.454 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T14:45:24.454 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 712 k 2024-09-16T14:45:24.454 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T14:45:24.454 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T14:45:24.454 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T14:45:24.455 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 478 k 2024-09-16T14:45:24.455 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T14:45:24.455 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 4.5 M 2024-09-16T14:45:24.455 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T14:45:24.455 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T14:45:24.455 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T14:45:24.455 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 45 k 2024-09-16T14:45:24.455 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 129 k 2024-09-16T14:45:24.455 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 162 k 2024-09-16T14:45:24.455 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T14:45:24.455 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 321 k 2024-09-16T14:45:24.456 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 300 k 2024-09-16T14:45:24.456 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 99 k 2024-09-16T14:45:24.456 INFO:teuthology.orchestra.run.smithi040.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T14:45:24.456 INFO:teuthology.orchestra.run.smithi040.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T14:45:24.456 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:45:24.456 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:45:24.456 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:45:24.456 INFO:teuthology.orchestra.run.smithi040.stdout:Install 28 Packages 2024-09-16T14:45:24.456 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 2 Packages 2024-09-16T14:45:24.456 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:45:24.457 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 53 M 2024-09-16T14:45:24.457 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:45:24.991 INFO:teuthology.orchestra.run.smithi111.stdout:(1/30): ceph-base-18.2.4-845.ge4e227b3.el9.x86_ 12 MB/s | 5.1 MB 00:00 2024-09-16T14:45:25.041 INFO:teuthology.orchestra.run.smithi111.stdout:(2/30): ceph-selinux-18.2.4-845.ge4e227b3.el9.x 506 kB/s | 25 kB 00:00 2024-09-16T14:45:25.116 INFO:teuthology.orchestra.run.smithi111.stdout:(3/30): libcephfs2-18.2.4-845.ge4e227b3.el9.x86 9.3 MB/s | 712 kB 00:00 2024-09-16T14:45:25.183 INFO:teuthology.orchestra.run.smithi111.stdout:(4/30): libradosstriper1-18.2.4-845.ge4e227b3.e 7.0 MB/s | 478 kB 00:00 2024-09-16T14:45:25.563 INFO:teuthology.orchestra.run.smithi111.stdout:(5/30): ceph-common-18.2.4-845.ge4e227b3.el9.x8 19 MB/s | 18 MB 00:00 2024-09-16T14:45:25.634 INFO:teuthology.orchestra.run.smithi111.stdout:(6/30): librgw2-18.2.4-845.ge4e227b3.el9.x86_64 9.9 MB/s | 4.5 MB 00:00 2024-09-16T14:45:25.659 INFO:teuthology.orchestra.run.smithi111.stdout:(7/30): python3-ceph-argparse-18.2.4-845.ge4e22 450 kB/s | 45 kB 00:00 2024-09-16T14:45:25.684 INFO:teuthology.orchestra.run.smithi111.stdout:(8/30): python3-ceph-common-18.2.4-845.ge4e227b 2.5 MB/s | 129 kB 00:00 2024-09-16T14:45:25.710 INFO:teuthology.orchestra.run.smithi111.stdout:(9/30): python3-cephfs-18.2.4-845.ge4e227b3.el9 3.1 MB/s | 162 kB 00:00 2024-09-16T14:45:25.735 INFO:teuthology.orchestra.run.smithi111.stdout:(10/30): python3-rados-18.2.4-845.ge4e227b3.el9 6.2 MB/s | 321 kB 00:00 2024-09-16T14:45:25.760 INFO:teuthology.orchestra.run.smithi111.stdout:(11/30): python3-rbd-18.2.4-845.ge4e227b3.el9.x 5.8 MB/s | 300 kB 00:00 2024-09-16T14:45:25.785 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 7.3 kB/s | 1.7 kB 00:00 2024-09-16T14:45:25.786 INFO:teuthology.orchestra.run.smithi111.stdout:(12/30): python3-rgw-18.2.4-845.ge4e227b3.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-16T14:45:26.084 INFO:teuthology.orchestra.run.smithi040.stdout:(1/30): ceph-base-18.2.4-845.ge4e227b3.el9.x86_ 3.7 MB/s | 5.1 MB 00:01 2024-09-16T14:45:26.134 INFO:teuthology.orchestra.run.smithi040.stdout:(2/30): ceph-selinux-18.2.4-845.ge4e227b3.el9.x 504 kB/s | 25 kB 00:00 2024-09-16T14:45:26.261 INFO:teuthology.orchestra.run.smithi111.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 214 kB/s | 107 kB 00:00 2024-09-16T14:45:26.336 INFO:teuthology.orchestra.run.smithi111.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 291 kB/s | 160 kB 00:00 2024-09-16T14:45:26.370 INFO:teuthology.orchestra.run.smithi111.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-09-16T14:45:26.410 INFO:teuthology.orchestra.run.smithi040.stdout:(3/30): ceph-common-18.2.4-845.ge4e227b3.el9.x8 11 MB/s | 18 MB 00:01 2024-09-16T14:45:26.443 INFO:teuthology.orchestra.run.smithi040.stdout:(4/30): libcephfs2-18.2.4-845.ge4e227b3.el9.x86 2.3 MB/s | 712 kB 00:00 2024-09-16T14:45:26.477 INFO:teuthology.orchestra.run.smithi040.stdout:(5/30): libradosstriper1-18.2.4-845.ge4e227b3.e 7.0 MB/s | 478 kB 00:00 2024-09-16T14:45:26.519 INFO:teuthology.orchestra.run.smithi040.stdout:(6/30): python3-ceph-argparse-18.2.4-845.ge4e22 1.1 MB/s | 45 kB 00:00 2024-09-16T14:45:26.561 INFO:teuthology.orchestra.run.smithi040.stdout:(7/30): python3-ceph-common-18.2.4-845.ge4e227b 3.0 MB/s | 129 kB 00:00 2024-09-16T14:45:26.603 INFO:teuthology.orchestra.run.smithi040.stdout:(8/30): python3-cephfs-18.2.4-845.ge4e227b3.el9 3.8 MB/s | 162 kB 00:00 2024-09-16T14:45:26.653 INFO:teuthology.orchestra.run.smithi040.stdout:(9/30): python3-rados-18.2.4-845.ge4e227b3.el9. 6.3 MB/s | 321 kB 00:00 2024-09-16T14:45:26.704 INFO:teuthology.orchestra.run.smithi040.stdout:(10/30): python3-rbd-18.2.4-845.ge4e227b3.el9.x 5.9 MB/s | 300 kB 00:00 2024-09-16T14:45:26.720 INFO:teuthology.orchestra.run.smithi111.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 1.7 MB/s | 662 kB 00:00 2024-09-16T14:45:26.746 INFO:teuthology.orchestra.run.smithi111.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 779 kB/s | 292 kB 00:00 2024-09-16T14:45:26.754 INFO:teuthology.orchestra.run.smithi040.stdout:(11/30): python3-rgw-18.2.4-845.ge4e227b3.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-16T14:45:26.796 INFO:teuthology.orchestra.run.smithi111.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 2.2 MB/s | 111 kB 00:00 2024-09-16T14:45:26.829 INFO:teuthology.orchestra.run.smithi111.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 387 kB/s | 42 kB 00:00 2024-09-16T14:45:26.854 INFO:teuthology.orchestra.run.smithi040.stdout:(12/30): boost-program-options-1.75.0-8.el9.x86 1.0 MB/s | 107 kB 00:00 2024-09-16T14:45:26.863 INFO:teuthology.orchestra.run.smithi111.stdout:(20/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.5 MB/s | 308 kB 00:00 2024-09-16T14:45:26.897 INFO:teuthology.orchestra.run.smithi111.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T14:45:26.913 INFO:teuthology.orchestra.run.smithi040.stdout:(13/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-16T14:45:26.922 INFO:teuthology.orchestra.run.smithi111.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-16T14:45:26.947 INFO:teuthology.orchestra.run.smithi111.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T14:45:26.955 INFO:teuthology.orchestra.run.smithi040.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-16T14:45:27.022 INFO:teuthology.orchestra.run.smithi040.stdout:(15/30): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-16T14:45:27.031 INFO:teuthology.orchestra.run.smithi111.stdout:(24/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-16T14:45:27.063 INFO:teuthology.orchestra.run.smithi040.stdout:(16/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-09-16T14:45:27.073 INFO:teuthology.orchestra.run.smithi111.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-16T14:45:27.097 INFO:teuthology.orchestra.run.smithi040.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 1.2 MB/s | 42 kB 00:00 2024-09-16T14:45:27.164 INFO:teuthology.orchestra.run.smithi040.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 1.6 MB/s | 111 kB 00:00 2024-09-16T14:45:27.206 INFO:teuthology.orchestra.run.smithi111.stdout:(26/30): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-16T14:45:27.214 INFO:teuthology.orchestra.run.smithi040.stdout:(19/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 6.0 MB/s | 308 kB 00:00 2024-09-16T14:45:27.323 INFO:teuthology.orchestra.run.smithi111.stdout:(27/30): libarrow-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 4.4 MB 00:00 2024-09-16T14:45:27.398 INFO:teuthology.orchestra.run.smithi040.stdout:(20/30): librgw2-18.2.4-845.ge4e227b3.el9.x86_6 4.7 MB/s | 4.5 MB 00:00 2024-09-16T14:45:27.499 INFO:teuthology.orchestra.run.smithi111.stdout:(28/30): librados2-18.2.4-845.ge4e227b3.el9.x86 11 MB/s | 3.3 MB 00:00 2024-09-16T14:45:27.515 INFO:teuthology.orchestra.run.smithi040.stdout:(21/30): ceph-radosgw-18.2.4-845.ge4e227b3.el9. 2.7 MB/s | 7.7 MB 00:02 2024-09-16T14:45:27.540 INFO:teuthology.orchestra.run.smithi040.stdout:(22/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 183 kB/s | 26 kB 00:00 2024-09-16T14:45:27.565 INFO:teuthology.orchestra.run.smithi111.stdout:(29/30): librbd1-18.2.4-845.ge4e227b3.el9.x86_6 12 MB/s | 3.0 MB 00:00 2024-09-16T14:45:27.574 INFO:teuthology.orchestra.run.smithi040.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-16T14:45:27.608 INFO:teuthology.orchestra.run.smithi040.stdout:(24/30): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-09-16T14:45:27.674 INFO:teuthology.orchestra.run.smithi040.stdout:(25/30): libarrow-9.0.0-11.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-09-16T14:45:27.700 INFO:teuthology.orchestra.run.smithi040.stdout:(26/30): re2-20211101-20.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-16T14:45:27.742 INFO:teuthology.orchestra.run.smithi040.stdout:(27/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.0 MB/s | 839 kB 00:00 2024-09-16T14:45:27.783 INFO:teuthology.orchestra.run.smithi040.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-16T14:45:27.959 INFO:teuthology.orchestra.run.smithi040.stdout:(29/30): librbd1-18.2.4-845.ge4e227b3.el9.x86_6 14 MB/s | 3.0 MB 00:00 2024-09-16T14:45:28.132 INFO:teuthology.orchestra.run.smithi111.stdout:(30/30): ceph-radosgw-18.2.4-845.ge4e227b3.el9. 2.2 MB/s | 7.7 MB 00:03 2024-09-16T14:45:28.138 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:45:28.138 INFO:teuthology.orchestra.run.smithi111.stdout:Total 13 MB/s | 53 MB 00:04 2024-09-16T14:45:28.161 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:45:28.169 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:45:28.169 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-16T14:45:28.169 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 7.7 M 2024-09-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrading: 2024-09-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi114.stdout: librados2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.3 M 2024-09-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.0 M 2024-09-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 5.1 M 2024-09-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 18 M 2024-09-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 25 k 2024-09-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi114.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 712 k 2024-09-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi114.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 478 k 2024-09-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2 x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 4.5 M 2024-09-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 45 k 2024-09-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 129 k 2024-09-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 162 k 2024-09-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 321 k 2024-09-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 300 k 2024-09-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 99 k 2024-09-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi114.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi114.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi114.stdout:Install 28 Packages 2024-09-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrade 2 Packages 2024-09-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:45:28.173 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 53 M 2024-09-16T14:45:28.173 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:45:28.214 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:45:28.249 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:45:28.249 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:45:28.267 INFO:teuthology.orchestra.run.smithi040.stdout:(30/30): librados2-18.2.4-845.ge4e227b3.el9.x86 5.8 MB/s | 3.3 MB 00:00 2024-09-16T14:45:28.274 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:45:28.274 INFO:teuthology.orchestra.run.smithi040.stdout:Total 14 MB/s | 53 MB 00:03 2024-09-16T14:45:28.346 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:45:28.385 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:45:28.385 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:45:28.877 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:45:28.878 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:45:29.039 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:45:29.039 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:45:29.244 INFO:teuthology.orchestra.run.smithi114.stdout:(1/30): ceph-base-18.2.4-845.ge4e227b3.el9.x86_ 5.1 MB/s | 5.1 MB 00:00 2024-09-16T14:45:29.286 INFO:teuthology.orchestra.run.smithi114.stdout:(2/30): ceph-selinux-18.2.4-845.ge4e227b3.el9.x 609 kB/s | 25 kB 00:00 2024-09-16T14:45:29.445 INFO:teuthology.orchestra.run.smithi114.stdout:(3/30): ceph-radosgw-18.2.4-845.ge4e227b3.el9.x 6.5 MB/s | 7.7 MB 00:01 2024-09-16T14:45:29.479 INFO:teuthology.orchestra.run.smithi114.stdout:(4/30): libcephfs2-18.2.4-845.ge4e227b3.el9.x86 3.6 MB/s | 712 kB 00:00 2024-09-16T14:45:29.529 INFO:teuthology.orchestra.run.smithi114.stdout:(5/30): libradosstriper1-18.2.4-845.ge4e227b3.e 5.6 MB/s | 478 kB 00:00 2024-09-16T14:45:29.587 INFO:teuthology.orchestra.run.smithi114.stdout:(6/30): python3-ceph-argparse-18.2.4-845.ge4e22 772 kB/s | 45 kB 00:00 2024-09-16T14:45:29.646 INFO:teuthology.orchestra.run.smithi114.stdout:(7/30): python3-ceph-common-18.2.4-845.ge4e227b 2.2 MB/s | 129 kB 00:00 2024-09-16T14:45:29.705 INFO:teuthology.orchestra.run.smithi114.stdout:(8/30): python3-cephfs-18.2.4-845.ge4e227b3.el9 2.7 MB/s | 162 kB 00:00 2024-09-16T14:45:29.772 INFO:teuthology.orchestra.run.smithi114.stdout:(9/30): python3-rados-18.2.4-845.ge4e227b3.el9. 4.7 MB/s | 321 kB 00:00 2024-09-16T14:45:29.839 INFO:teuthology.orchestra.run.smithi114.stdout:(10/30): python3-rbd-18.2.4-845.ge4e227b3.el9.x 4.4 MB/s | 300 kB 00:00 2024-09-16T14:45:29.897 INFO:teuthology.orchestra.run.smithi114.stdout:(11/30): librgw2-18.2.4-845.ge4e227b3.el9.x86_6 11 MB/s | 4.5 MB 00:00 2024-09-16T14:45:29.923 INFO:teuthology.orchestra.run.smithi114.stdout:(12/30): python3-rgw-18.2.4-845.ge4e227b3.el9.x 1.2 MB/s | 99 kB 00:00 2024-09-16T14:45:30.106 INFO:teuthology.orchestra.run.smithi114.stdout:(13/30): ceph-common-18.2.4-845.ge4e227b3.el9.x 9.9 MB/s | 18 MB 00:01 2024-09-16T14:45:30.198 INFO:teuthology.orchestra.run.smithi114.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 355 kB/s | 107 kB 00:00 2024-09-16T14:45:30.223 INFO:teuthology.orchestra.run.smithi114.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-09-16T14:45:30.249 INFO:teuthology.orchestra.run.smithi114.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 492 kB/s | 160 kB 00:00 2024-09-16T14:45:30.269 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:45:30.291 INFO:teuthology.orchestra.run.smithi114.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-16T14:45:30.316 INFO:teuthology.orchestra.run.smithi114.stdout:(18/30): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-16T14:45:30.341 INFO:teuthology.orchestra.run.smithi114.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-16T14:45:30.350 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-16T14:45:30.379 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-16T14:45:30.400 INFO:teuthology.orchestra.run.smithi114.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 1.0 MB/s | 111 kB 00:00 2024-09-16T14:45:30.429 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-16T14:45:30.433 INFO:teuthology.orchestra.run.smithi114.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 824 kB/s | 26 kB 00:00 2024-09-16T14:45:30.459 INFO:teuthology.orchestra.run.smithi114.stdout:(22/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.1 MB/s | 308 kB 00:00 2024-09-16T14:45:30.474 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:45:30.484 INFO:teuthology.orchestra.run.smithi114.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-09-16T14:45:30.509 INFO:teuthology.orchestra.run.smithi114.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-16T14:45:30.551 INFO:teuthology.orchestra.run.smithi114.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-16T14:45:30.552 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-16T14:45:30.589 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-16T14:45:30.602 INFO:teuthology.orchestra.run.smithi114.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-16T14:45:30.640 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-16T14:45:30.741 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-16T14:45:30.768 INFO:teuthology.orchestra.run.smithi114.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-09-16T14:45:30.853 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:45:30.869 INFO:teuthology.orchestra.run.smithi114.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 8.6 MB/s | 4.4 MB 00:00 2024-09-16T14:45:30.936 INFO:teuthology.orchestra.run.smithi114.stdout:(29/30): librados2-18.2.4-845.ge4e227b3.el9.x86 9.9 MB/s | 3.3 MB 00:00 2024-09-16T14:45:30.945 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:45:31.037 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 6/32 2024-09-16T14:45:31.039 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-16T14:45:31.052 INFO:teuthology.orchestra.run.smithi114.stdout:(30/30): librbd1-18.2.4-845.ge4e227b3.el9.x86_6 11 MB/s | 3.0 MB 00:00 2024-09-16T14:45:31.058 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:45:31.058 INFO:teuthology.orchestra.run.smithi114.stdout:Total 18 MB/s | 53 MB 00:02 2024-09-16T14:45:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:45:31.096 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:45:31.109 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:45:31.129 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:45:31.137 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-16T14:45:31.164 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:45:31.164 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:45:31.181 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:45:31.181 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x 9/32 2024-09-16T14:45:31.247 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 10/32 2024-09-16T14:45:31.267 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 6/32 2024-09-16T14:45:31.274 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:45:31.306 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:45:31.342 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:45:31.352 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:45:31.381 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-16T14:45:31.425 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x 9/32 2024-09-16T14:45:31.447 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-16T14:45:31.483 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-16T14:45:31.493 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 10/32 2024-09-16T14:45:31.516 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:45:31.530 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-16T14:45:31.565 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-16T14:45:31.583 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:45:31.592 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-16T14:45:31.628 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-16T14:45:31.661 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-16T14:45:31.692 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-16T14:45:31.749 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-16T14:45:31.792 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:45:31.792 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:45:31.801 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-16T14:45:31.831 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-16T14:45:31.858 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-16T14:45:31.894 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-16T14:45:31.927 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-16T14:45:31.961 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-16T14:45:31.998 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 20/32 2024-09-16T14:45:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 20/32 2024-09-16T14:45:32.120 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 21/32 2024-09-16T14:45:32.155 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-16T14:45:32.242 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-16T14:45:32.281 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 20/32 2024-09-16T14:45:32.364 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 20/32 2024-09-16T14:45:32.394 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 21/32 2024-09-16T14:45:32.431 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-16T14:45:32.609 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-16T14:45:32.634 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 24/32 2024-09-16T14:45:32.697 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 24/32 2024-09-16T14:45:32.765 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 25/32 2024-09-16T14:45:32.794 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86 26/32 2024-09-16T14:45:32.858 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-16T14:45:32.883 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 24/32 2024-09-16T14:45:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 24/32 2024-09-16T14:45:33.034 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 25/32 2024-09-16T14:45:33.061 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86 26/32 2024-09-16T14:45:33.194 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:45:33.262 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-16T14:45:33.299 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-16T14:45:33.357 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-16T14:45:33.722 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-16T14:45:33.782 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:45:33.873 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:45:33.958 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 6/32 2024-09-16T14:45:33.991 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:45:34.055 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:45:34.091 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-16T14:45:34.136 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x 9/32 2024-09-16T14:45:34.226 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 10/32 2024-09-16T14:45:34.261 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:45:34.326 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:45:34.434 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-16T14:45:34.462 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-16T14:45:34.521 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-16T14:45:34.552 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-16T14:45:34.578 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-16T14:45:34.623 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-16T14:45:34.665 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-16T14:45:34.930 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:45:34.964 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-16T14:45:34.974 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:45:35.010 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 20/32 2024-09-16T14:45:35.101 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 20/32 2024-09-16T14:45:35.131 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 21/32 2024-09-16T14:45:35.167 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-16T14:45:35.503 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:45:35.516 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:45:35.544 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 28/32 2024-09-16T14:45:35.557 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:45:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-16T14:45:35.604 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 24/32 2024-09-16T14:45:35.643 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 28/32 2024-09-16T14:45:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T14:45:35.643 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T14:45:35.643 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:45:35.666 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 24/32 2024-09-16T14:45:35.679 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:45:35.735 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 25/32 2024-09-16T14:45:36.040 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86 26/32 2024-09-16T14:45:36.137 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:45:36.177 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 28/32 2024-09-16T14:45:36.281 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 28/32 2024-09-16T14:45:36.281 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T14:45:36.281 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T14:45:36.281 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:45:36.312 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:45:37.896 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:45:37.928 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:45:38.480 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 27/32 2024-09-16T14:45:38.515 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 28/32 2024-09-16T14:45:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 28/32 2024-09-16T14:45:38.613 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T14:45:38.613 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T14:45:38.613 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:45:38.667 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:45:45.339 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:45:45.339 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2024-09-16T14:45:45.339 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2024-09-16T14:45:45.339 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2024-09-16T14:45:45.340 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2024-09-16T14:45:45.340 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2024-09-16T14:45:45.340 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /root 2024-09-16T14:45:45.340 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2024-09-16T14:45:45.340 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:45:46.094 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 30/32 2024-09-16T14:45:46.120 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 30/32 2024-09-16T14:45:46.120 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:45:46.120 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T14:45:46.120 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T14:45:46.120 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T14:45:46.120 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:45:46.122 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T14:45:46.205 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T14:45:46.205 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:45:46.694 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:45:46.694 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2024-09-16T14:45:46.694 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2024-09-16T14:45:46.694 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2024-09-16T14:45:46.694 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2024-09-16T14:45:46.694 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2024-09-16T14:45:46.694 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /root 2024-09-16T14:45:46.694 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2024-09-16T14:45:46.695 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:45:47.427 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:45:47.427 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 1/32 2024-09-16T14:45:47.427 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2/32 2024-09-16T14:45:47.427 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 3/32 2024-09-16T14:45:47.428 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 4/32 2024-09-16T14:45:47.428 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:45:47.428 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 6/32 2024-09-16T14:45:47.428 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:45:47.428 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x 8/32 2024-09-16T14:45:47.428 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86 9/32 2024-09-16T14:45:47.428 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 10/32 2024-09-16T14:45:47.428 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:45:47.428 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 12/32 2024-09-16T14:45:47.428 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 13/32 2024-09-16T14:45:47.428 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-16T14:45:47.428 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-16T14:45:47.429 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-16T14:45:47.429 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-16T14:45:47.429 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-16T14:45:47.429 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-16T14:45:47.429 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-16T14:45:47.429 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-16T14:45:47.429 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-16T14:45:47.429 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-16T14:45:47.429 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-16T14:45:47.429 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-16T14:45:47.429 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-16T14:45:47.429 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-16T14:45:47.429 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-16T14:45:47.430 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:45:47.430 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-16T14:45:47.430 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 31/32 2024-09-16T14:45:47.480 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 30/32 2024-09-16T14:45:47.506 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 30/32 2024-09-16T14:45:47.506 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:45:47.507 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T14:45:47.507 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T14:45:47.507 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T14:45:47.507 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:45:47.509 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T14:45:47.591 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T14:45:47.591 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:45:48.545 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:45:48.546 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /sys 2024-09-16T14:45:48.546 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /proc 2024-09-16T14:45:48.546 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /mnt 2024-09-16T14:45:48.546 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /var/tmp 2024-09-16T14:45:48.546 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /home 2024-09-16T14:45:48.546 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /root 2024-09-16T14:45:48.546 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /tmp 2024-09-16T14:45:48.546 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:45:48.615 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:45:48.615 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:45:48.615 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2024-09-16T14:45:48.615 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.615 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.615 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:45:48.615 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T14:45:48.615 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.615 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.615 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.615 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.615 INFO:teuthology.orchestra.run.smithi111.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-16T14:45:48.616 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T14:45:48.616 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T14:45:48.616 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T14:45:48.616 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.616 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T14:45:48.616 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T14:45:48.616 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T14:45:48.616 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.616 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T14:45:48.616 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.616 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T14:45:48.616 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T14:45:48.617 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T14:45:48.617 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.617 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.617 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.617 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T14:45:48.617 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.617 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.617 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:48.617 INFO:teuthology.orchestra.run.smithi111.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T14:45:48.617 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T14:45:48.617 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:45:48.617 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:45:48.723 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:45:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 1/32 2024-09-16T14:45:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2/32 2024-09-16T14:45:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 3/32 2024-09-16T14:45:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 4/32 2024-09-16T14:45:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:45:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 6/32 2024-09-16T14:45:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:45:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x 8/32 2024-09-16T14:45:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86 9/32 2024-09-16T14:45:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 10/32 2024-09-16T14:45:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:45:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 12/32 2024-09-16T14:45:48.724 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 13/32 2024-09-16T14:45:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-16T14:45:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-16T14:45:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-16T14:45:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-16T14:45:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-16T14:45:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-16T14:45:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-16T14:45:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-16T14:45:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-16T14:45:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-16T14:45:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-16T14:45:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-16T14:45:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-16T14:45:48.725 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-16T14:45:48.726 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-16T14:45:48.726 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:45:48.726 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-16T14:45:48.726 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 31/32 2024-09-16T14:45:48.972 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-test 2024-09-16T14:45:49.308 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 30/32 2024-09-16T14:45:49.333 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 30/32 2024-09-16T14:45:49.333 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:45:49.333 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T14:45:49.333 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T14:45:49.333 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T14:45:49.334 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:45:49.335 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T14:45:49.402 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T14:45:49.402 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:45:49.543 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:00:28 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:45:49.675 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:45:49.676 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:45:49.676 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-16T14:45:49.676 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:45:49.676 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:45:49.676 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 40 M 2024-09-16T14:45:49.676 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-16T14:45:49.676 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 166 k 2024-09-16T14:45:49.676 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T14:45:49.676 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T14:45:49.677 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:45:49.677 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:45:49.677 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:45:49.677 INFO:teuthology.orchestra.run.smithi111.stdout:Install 4 Packages 2024-09-16T14:45:49.677 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:45:49.678 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 40 M 2024-09-16T14:45:49.678 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 164 M 2024-09-16T14:45:49.678 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:45:49.790 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:45:49.790 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:45:49.790 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2024-09-16T14:45:49.790 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.790 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.790 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:45:49.791 INFO:teuthology.orchestra.run.smithi040.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T14:45:49.791 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.791 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.791 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.791 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.791 INFO:teuthology.orchestra.run.smithi040.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-16T14:45:49.791 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T14:45:49.791 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T14:45:49.791 INFO:teuthology.orchestra.run.smithi040.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T14:45:49.791 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.791 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T14:45:49.791 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T14:45:49.791 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T14:45:49.792 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.792 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T14:45:49.792 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.792 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T14:45:49.792 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T14:45:49.792 INFO:teuthology.orchestra.run.smithi040.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T14:45:49.792 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.792 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.792 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.792 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T14:45:49.792 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.792 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.793 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:49.793 INFO:teuthology.orchestra.run.smithi040.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T14:45:49.793 INFO:teuthology.orchestra.run.smithi040.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T14:45:49.793 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:45:49.793 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:45:50.012 INFO:teuthology.orchestra.run.smithi111.stdout:(1/4): libcephsqlite-18.2.4-845.ge4e227b3.el9.x 777 kB/s | 166 kB 00:00 2024-09-16T14:45:50.164 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-test 2024-09-16T14:45:50.378 INFO:teuthology.orchestra.run.smithi111.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 174 kB/s | 64 kB 00:00 2024-09-16T14:45:50.429 INFO:teuthology.orchestra.run.smithi111.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 482 kB/s | 304 kB 00:00 2024-09-16T14:45:50.573 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:45:50.573 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 1/32 2024-09-16T14:45:50.574 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2/32 2024-09-16T14:45:50.574 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 3/32 2024-09-16T14:45:50.574 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 4/32 2024-09-16T14:45:50.574 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 5/32 2024-09-16T14:45:50.574 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 6/32 2024-09-16T14:45:50.574 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 7/32 2024-09-16T14:45:50.574 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x 8/32 2024-09-16T14:45:50.574 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86 9/32 2024-09-16T14:45:50.574 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 10/32 2024-09-16T14:45:50.574 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 11/32 2024-09-16T14:45:50.574 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 12/32 2024-09-16T14:45:50.574 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 13/32 2024-09-16T14:45:50.574 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-16T14:45:50.575 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-16T14:45:50.575 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-16T14:45:50.575 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-16T14:45:50.575 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-16T14:45:50.575 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-16T14:45:50.575 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-16T14:45:50.575 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-16T14:45:50.575 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-16T14:45:50.575 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-16T14:45:50.575 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-16T14:45:50.575 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-16T14:45:50.575 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-16T14:45:50.575 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-16T14:45:50.575 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-16T14:45:50.576 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 29/32 2024-09-16T14:45:50.576 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-16T14:45:50.576 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 31/32 2024-09-16T14:45:50.756 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:29 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:45:50.900 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:45:50.901 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:45:50.901 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-16T14:45:50.901 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:45:50.901 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:45:50.901 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 40 M 2024-09-16T14:45:50.901 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-16T14:45:50.901 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 166 k 2024-09-16T14:45:50.902 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T14:45:50.902 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T14:45:50.902 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:45:50.902 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:45:50.902 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:45:50.902 INFO:teuthology.orchestra.run.smithi040.stdout:Install 4 Packages 2024-09-16T14:45:50.902 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:45:50.903 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 40 M 2024-09-16T14:45:50.903 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 164 M 2024-09-16T14:45:50.903 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:45:51.112 INFO:teuthology.orchestra.run.smithi111.stdout:(4/4): ceph-test-18.2.4-845.ge4e227b3.el9.x86_6 30 MB/s | 40 MB 00:01 2024-09-16T14:45:51.114 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:45:51.115 INFO:teuthology.orchestra.run.smithi111.stdout:Total 28 MB/s | 40 MB 00:01 2024-09-16T14:45:51.142 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:45:51.161 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:45:51.162 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:45:51.346 INFO:teuthology.orchestra.run.smithi040.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.8 MB/s | 304 kB 00:00 2024-09-16T14:45:51.413 INFO:teuthology.orchestra.run.smithi040.stdout:(2/4): libcephsqlite-18.2.4-845.ge4e227b3.el9.x 719 kB/s | 166 kB 00:00 2024-09-16T14:45:51.438 INFO:teuthology.orchestra.run.smithi040.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 694 kB/s | 64 kB 00:00 2024-09-16T14:45:51.485 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:45:51.485 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:45:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T14:45:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:45:51.613 INFO:teuthology.orchestra.run.smithi114.stdout:Upgraded: 2024-09-16T14:45:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.613 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:45:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T14:45:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-16T14:45:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T14:45:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T14:45:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T14:45:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T14:45:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T14:45:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T14:45:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T14:45:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T14:45:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T14:45:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T14:45:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T14:45:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T14:45:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T14:45:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:45:51.615 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:45:51.963 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:45:52.017 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-test 2024-09-16T14:45:52.045 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T14:45:52.095 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T14:45:52.122 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 3/4 2024-09-16T14:45:52.472 INFO:teuthology.orchestra.run.smithi040.stdout:(4/4): ceph-test-18.2.4-845.ge4e227b3.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-16T14:45:52.474 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:45:52.474 INFO:teuthology.orchestra.run.smithi040.stdout:Total 26 MB/s | 40 MB 00:01 2024-09-16T14:45:52.502 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:45:52.524 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:45:52.524 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:45:52.589 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:27 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:45:52.723 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:45:52.723 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:45:52.724 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-16T14:45:52.724 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:45:52.724 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:45:52.724 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 40 M 2024-09-16T14:45:52.724 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-16T14:45:52.724 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 166 k 2024-09-16T14:45:52.724 INFO:teuthology.orchestra.run.smithi114.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T14:45:52.724 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T14:45:52.724 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:45:52.724 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:45:52.724 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:45:52.724 INFO:teuthology.orchestra.run.smithi114.stdout:Install 4 Packages 2024-09-16T14:45:52.724 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:45:52.725 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 40 M 2024-09-16T14:45:52.725 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 164 M 2024-09-16T14:45:52.725 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:45:52.869 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:45:52.869 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:45:52.984 INFO:teuthology.orchestra.run.smithi114.stdout:(1/4): libcephsqlite-18.2.4-845.ge4e227b3.el9.x 787 kB/s | 166 kB 00:00 2024-09-16T14:45:53.026 INFO:teuthology.orchestra.run.smithi114.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-09-16T14:45:53.134 INFO:teuthology.orchestra.run.smithi114.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 424 kB/s | 64 kB 00:00 2024-09-16T14:45:53.358 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:45:53.451 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T14:45:53.581 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T14:45:53.607 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 3/4 2024-09-16T14:45:54.043 INFO:teuthology.orchestra.run.smithi114.stdout:(4/4): ceph-test-18.2.4-845.ge4e227b3.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-16T14:45:54.045 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:45:54.045 INFO:teuthology.orchestra.run.smithi114.stdout:Total 30 MB/s | 40 MB 00:01 2024-09-16T14:45:54.072 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:45:54.092 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:45:54.092 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:45:54.416 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:45:54.416 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:45:54.896 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:45:54.979 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T14:45:55.025 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T14:45:55.053 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 3/4 2024-09-16T14:45:55.805 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 3/4 2024-09-16T14:45:55.877 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 4/4 2024-09-16T14:45:57.540 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 3/4 2024-09-16T14:45:57.625 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 4/4 2024-09-16T14:45:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 4/4 2024-09-16T14:45:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 1/4 2024-09-16T14:45:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 2/4 2024-09-16T14:45:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T14:45:58.178 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T14:45:58.179 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:45:58.179 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:45:58.179 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:58.179 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:58.179 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T14:45:58.179 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T14:45:58.179 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:45:58.179 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:45:58.507 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph 2024-09-16T14:45:58.739 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 3/4 2024-09-16T14:45:58.817 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 4/4 2024-09-16T14:45:59.073 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:00:38 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:45:59.208 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:45:59.210 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:45:59.211 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-16T14:45:59.211 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:45:59.211 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:45:59.211 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 6.5 k 2024-09-16T14:45:59.211 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-16T14:45:59.211 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 2.1 M 2024-09-16T14:45:59.211 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 1.5 M 2024-09-16T14:45:59.211 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 246 k 2024-09-16T14:45:59.211 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 4.7 M 2024-09-16T14:45:59.211 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 17 M 2024-09-16T14:45:59.211 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T14:45:59.211 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T14:45:59.211 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T14:45:59.212 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T14:45:59.212 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T14:45:59.212 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T14:45:59.212 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T14:45:59.212 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T14:45:59.212 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T14:45:59.212 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T14:45:59.212 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T14:45:59.212 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T14:45:59.212 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T14:45:59.212 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T14:45:59.212 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T14:45:59.212 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T14:45:59.213 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T14:45:59.213 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T14:45:59.213 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T14:45:59.213 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T14:45:59.213 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T14:45:59.213 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T14:45:59.213 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T14:45:59.213 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T14:45:59.213 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T14:45:59.213 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T14:45:59.213 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T14:45:59.213 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:45:59.213 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:45:59.213 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:45:59.214 INFO:teuthology.orchestra.run.smithi111.stdout:Install 33 Packages 2024-09-16T14:45:59.214 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:45:59.214 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 29 M 2024-09-16T14:45:59.214 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 104 M 2024-09-16T14:45:59.214 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:45:59.538 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 4/4 2024-09-16T14:45:59.538 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 1/4 2024-09-16T14:45:59.538 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 2/4 2024-09-16T14:45:59.538 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T14:45:59.887 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T14:45:59.887 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:45:59.887 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:45:59.887 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:59.887 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:45:59.887 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T14:45:59.888 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T14:45:59.888 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:45:59.888 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:45:59.979 INFO:teuthology.orchestra.run.smithi111.stdout:(1/33): ceph-18.2.4-845.ge4e227b3.el9.x86_64.rp 48 kB/s | 6.5 kB 00:00 2024-09-16T14:46:00.155 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph 2024-09-16T14:46:00.188 INFO:teuthology.orchestra.run.smithi111.stdout:(2/33): ceph-mds-18.2.4-845.ge4e227b3.el9.x86_6 6.1 MB/s | 2.1 MB 00:00 2024-09-16T14:46:00.222 INFO:teuthology.orchestra.run.smithi111.stdout:(3/33): ceph-mgr-18.2.4-845.ge4e227b3.el9.x86_6 3.9 MB/s | 1.5 MB 00:00 2024-09-16T14:46:00.297 INFO:teuthology.orchestra.run.smithi111.stdout:(4/33): ceph-mgr-modules-core-18.2.4-845.ge4e22 3.2 MB/s | 246 kB 00:00 2024-09-16T14:46:00.372 INFO:teuthology.orchestra.run.smithi111.stdout:(5/33): ceph-mon-18.2.4-845.ge4e227b3.el9.x86_6 12 MB/s | 4.7 MB 00:00 2024-09-16T14:46:00.547 INFO:teuthology.orchestra.run.smithi111.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-09-16T14:46:00.739 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:39 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:00.748 INFO:teuthology.orchestra.run.smithi111.stdout:(7/33): ceph-osd-18.2.4-845.ge4e227b3.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-16T14:46:00.773 INFO:teuthology.orchestra.run.smithi111.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 473 kB/s | 106 kB 00:00 2024-09-16T14:46:00.823 INFO:teuthology.orchestra.run.smithi111.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-16T14:46:00.857 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 4/4 2024-09-16T14:46:00.858 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 1/4 2024-09-16T14:46:00.858 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 2/4 2024-09-16T14:46:00.858 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T14:46:00.865 INFO:teuthology.orchestra.run.smithi111.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 2.5 MB/s | 1.2 MB 00:00 2024-09-16T14:46:00.880 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:46:00.882 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:00.882 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-16T14:46:00.882 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:00.883 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:46:00.883 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 6.5 k 2024-09-16T14:46:00.883 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-16T14:46:00.883 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 2.1 M 2024-09-16T14:46:00.883 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 1.5 M 2024-09-16T14:46:00.884 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 246 k 2024-09-16T14:46:00.884 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 4.7 M 2024-09-16T14:46:00.884 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 17 M 2024-09-16T14:46:00.884 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T14:46:00.884 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T14:46:00.884 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T14:46:00.885 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T14:46:00.885 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T14:46:00.885 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T14:46:00.885 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T14:46:00.885 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T14:46:00.885 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T14:46:00.886 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T14:46:00.886 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T14:46:00.886 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T14:46:00.886 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T14:46:00.886 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T14:46:00.887 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T14:46:00.887 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T14:46:00.887 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T14:46:00.887 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T14:46:00.888 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T14:46:00.888 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T14:46:00.888 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T14:46:00.888 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T14:46:00.888 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T14:46:00.888 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T14:46:00.889 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T14:46:00.889 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T14:46:00.889 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T14:46:00.889 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:00.889 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:46:00.889 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:00.890 INFO:teuthology.orchestra.run.smithi040.stdout:Install 33 Packages 2024-09-16T14:46:00.890 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:00.891 INFO:teuthology.orchestra.run.smithi111.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-09-16T14:46:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 29 M 2024-09-16T14:46:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 104 M 2024-09-16T14:46:00.892 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:46:00.949 INFO:teuthology.orchestra.run.smithi111.stdout:(12/33): python3-pycparser-2.20-6.el9.noarch.rp 673 kB/s | 135 kB 00:00 2024-09-16T14:46:00.974 INFO:teuthology.orchestra.run.smithi111.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 416 kB/s | 35 kB 00:00 2024-09-16T14:46:01.008 INFO:teuthology.orchestra.run.smithi111.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-16T14:46:01.033 INFO:teuthology.orchestra.run.smithi111.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-09-16T14:46:01.066 INFO:teuthology.orchestra.run.smithi111.stdout:(16/33): python3-pytz-2021.1-5.el9.noarch.rpm 434 kB/s | 51 kB 00:00 2024-09-16T14:46:01.100 INFO:teuthology.orchestra.run.smithi111.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 472 kB/s | 43 kB 00:00 2024-09-16T14:46:01.134 INFO:teuthology.orchestra.run.smithi111.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 431 kB/s | 11 kB 00:00 2024-09-16T14:46:01.147 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T14:46:01.147 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:01.147 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:46:01.148 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:01.148 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:01.148 INFO:teuthology.orchestra.run.smithi114.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T14:46:01.148 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T14:46:01.148 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:01.148 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:01.159 INFO:teuthology.orchestra.run.smithi111.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-16T14:46:01.176 INFO:teuthology.orchestra.run.smithi111.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 439 kB/s | 18 kB 00:00 2024-09-16T14:46:01.201 INFO:teuthology.orchestra.run.smithi111.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 555 kB/s | 23 kB 00:00 2024-09-16T14:46:01.226 INFO:teuthology.orchestra.run.smithi111.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-16T14:46:01.244 INFO:teuthology.orchestra.run.smithi111.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 469 kB/s | 20 kB 00:00 2024-09-16T14:46:01.260 INFO:teuthology.orchestra.run.smithi111.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 1.3 MB/s | 46 kB 00:00 2024-09-16T14:46:01.286 INFO:teuthology.orchestra.run.smithi111.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-16T14:46:01.311 INFO:teuthology.orchestra.run.smithi111.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2024-09-16T14:46:01.328 INFO:teuthology.orchestra.run.smithi111.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 392 kB/s | 16 kB 00:00 2024-09-16T14:46:01.353 INFO:teuthology.orchestra.run.smithi111.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-09-16T14:46:01.378 INFO:teuthology.orchestra.run.smithi111.stdout:(29/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-09-16T14:46:01.404 INFO:teuthology.orchestra.run.smithi111.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 474 kB/s | 36 kB 00:00 2024-09-16T14:46:01.422 INFO:teuthology.orchestra.run.smithi040.stdout:(1/33): ceph-18.2.4-845.ge4e227b3.el9.x86_64.rp 57 kB/s | 6.5 kB 00:00 2024-09-16T14:46:01.429 INFO:teuthology.orchestra.run.smithi111.stdout:(31/33): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-16T14:46:01.454 INFO:teuthology.orchestra.run.smithi111.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 400 kB/s | 20 kB 00:00 2024-09-16T14:46:01.470 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph 2024-09-16T14:46:01.479 INFO:teuthology.orchestra.run.smithi111.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-16T14:46:01.483 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:01.484 INFO:teuthology.orchestra.run.smithi111.stdout:Total 13 MB/s | 29 MB 00:02 2024-09-16T14:46:01.597 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:46:01.632 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:46:01.633 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:46:01.648 INFO:teuthology.orchestra.run.smithi040.stdout:(2/33): ceph-mds-18.2.4-845.ge4e227b3.el9.x86_6 6.2 MB/s | 2.1 MB 00:00 2024-09-16T14:46:01.690 INFO:teuthology.orchestra.run.smithi040.stdout:(3/33): ceph-mgr-18.2.4-845.ge4e227b3.el9.x86_6 3.8 MB/s | 1.5 MB 00:00 2024-09-16T14:46:01.773 INFO:teuthology.orchestra.run.smithi040.stdout:(4/33): ceph-mgr-modules-core-18.2.4-845.ge4e22 2.9 MB/s | 246 kB 00:00 2024-09-16T14:46:01.857 INFO:teuthology.orchestra.run.smithi040.stdout:(5/33): ceph-mon-18.2.4-845.ge4e227b3.el9.x86_6 11 MB/s | 4.7 MB 00:00 2024-09-16T14:46:01.932 INFO:teuthology.orchestra.run.smithi040.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-16T14:46:01.974 INFO:teuthology.orchestra.run.smithi040.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-16T14:46:02.016 INFO:teuthology.orchestra.run.smithi040.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-09-16T14:46:02.024 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:46:02.025 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:46:02.045 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:37 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:02.187 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:02.189 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:02.189 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-16T14:46:02.189 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:02.190 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:46:02.190 INFO:teuthology.orchestra.run.smithi114.stdout: ceph x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 6.5 k 2024-09-16T14:46:02.190 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-16T14:46:02.190 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 2.1 M 2024-09-16T14:46:02.190 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 1.5 M 2024-09-16T14:46:02.191 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 246 k 2024-09-16T14:46:02.191 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 4.7 M 2024-09-16T14:46:02.191 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 17 M 2024-09-16T14:46:02.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T14:46:02.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T14:46:02.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T14:46:02.192 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T14:46:02.192 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T14:46:02.192 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T14:46:02.192 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T14:46:02.192 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T14:46:02.193 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T14:46:02.193 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T14:46:02.193 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T14:46:02.193 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T14:46:02.193 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T14:46:02.193 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T14:46:02.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T14:46:02.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T14:46:02.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T14:46:02.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T14:46:02.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T14:46:02.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T14:46:02.195 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T14:46:02.195 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T14:46:02.195 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T14:46:02.195 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T14:46:02.195 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T14:46:02.195 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T14:46:02.196 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T14:46:02.196 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:02.196 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:46:02.196 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:02.196 INFO:teuthology.orchestra.run.smithi114.stdout:Install 33 Packages 2024-09-16T14:46:02.196 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:02.198 INFO:teuthology.orchestra.run.smithi040.stdout:(9/33): ceph-osd-18.2.4-845.ge4e227b3.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-16T14:46:02.198 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 29 M 2024-09-16T14:46:02.199 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 104 M 2024-09-16T14:46:02.199 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:46:02.217 INFO:teuthology.orchestra.run.smithi040.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 623 kB/s | 125 kB 00:00 2024-09-16T14:46:02.259 INFO:teuthology.orchestra.run.smithi040.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 3.1 MB/s | 1.2 MB 00:00 2024-09-16T14:46:02.334 INFO:teuthology.orchestra.run.smithi040.stdout:(12/33): python3-urllib3-1.26.5-6.el9.noarch.rp 1.5 MB/s | 215 kB 00:00 2024-09-16T14:46:02.826 INFO:teuthology.orchestra.run.smithi040.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 61 kB/s | 35 kB 00:00 2024-09-16T14:46:02.869 INFO:teuthology.orchestra.run.smithi114.stdout:(1/33): ceph-18.2.4-845.ge4e227b3.el9.x86_64.rp 50 kB/s | 6.5 kB 00:00 2024-09-16T14:46:02.952 INFO:teuthology.orchestra.run.smithi040.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 83 kB/s | 51 kB 00:00 2024-09-16T14:46:02.993 INFO:teuthology.orchestra.run.smithi040.stdout:(15/33): python3-mako-1.1.4-6.el9.noarch.rpm 222 kB/s | 172 kB 00:00 2024-09-16T14:46:03.027 INFO:teuthology.orchestra.run.smithi040.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 581 kB/s | 43 kB 00:00 2024-09-16T14:46:03.061 INFO:teuthology.orchestra.run.smithi114.stdout:(2/33): ceph-mds-18.2.4-845.ge4e227b3.el9.x86_6 6.5 MB/s | 2.1 MB 00:00 2024-09-16T14:46:03.069 INFO:teuthology.orchestra.run.smithi040.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-09-16T14:46:03.102 INFO:teuthology.orchestra.run.smithi040.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 420 kB/s | 11 kB 00:00 2024-09-16T14:46:03.103 INFO:teuthology.orchestra.run.smithi114.stdout:(3/33): ceph-mgr-18.2.4-845.ge4e227b3.el9.x86_6 4.0 MB/s | 1.5 MB 00:00 2024-09-16T14:46:03.128 INFO:teuthology.orchestra.run.smithi040.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 708 kB/s | 18 kB 00:00 2024-09-16T14:46:03.153 INFO:teuthology.orchestra.run.smithi040.stdout:(20/33): python3-cherrypy-18.6.1-2.el9.noarch.r 2.8 MB/s | 358 kB 00:00 2024-09-16T14:46:03.178 INFO:teuthology.orchestra.run.smithi040.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-09-16T14:46:03.204 INFO:teuthology.orchestra.run.smithi040.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 393 kB/s | 19 kB 00:00 2024-09-16T14:46:03.211 INFO:teuthology.orchestra.run.smithi114.stdout:(4/33): ceph-mgr-modules-core-18.2.4-845.ge4e22 2.2 MB/s | 246 kB 00:00 2024-09-16T14:46:03.229 INFO:teuthology.orchestra.run.smithi040.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-16T14:46:03.254 INFO:teuthology.orchestra.run.smithi040.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-16T14:46:03.279 INFO:teuthology.orchestra.run.smithi040.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-16T14:46:03.287 INFO:teuthology.orchestra.run.smithi114.stdout:(5/33): ceph-mon-18.2.4-845.ge4e227b3.el9.x86_6 11 MB/s | 4.7 MB 00:00 2024-09-16T14:46:03.288 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:46:03.313 INFO:teuthology.orchestra.run.smithi040.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 495 kB/s | 16 kB 00:00 2024-09-16T14:46:03.338 INFO:teuthology.orchestra.run.smithi040.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-16T14:46:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-16T14:46:03.363 INFO:teuthology.orchestra.run.smithi040.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-16T14:46:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-16T14:46:03.397 INFO:teuthology.orchestra.run.smithi040.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 612 kB/s | 36 kB 00:00 2024-09-16T14:46:03.415 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-16T14:46:03.431 INFO:teuthology.orchestra.run.smithi040.stdout:(30/33): python3-webob-1.8.8-2.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-09-16T14:46:03.456 INFO:teuthology.orchestra.run.smithi040.stdout:(31/33): python3-toml-0.10.2-6.el9.noarch.rpm 66 kB/s | 42 kB 00:00 2024-09-16T14:46:03.481 INFO:teuthology.orchestra.run.smithi040.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 407 kB/s | 20 kB 00:00 2024-09-16T14:46:03.506 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-16T14:46:03.523 INFO:teuthology.orchestra.run.smithi040.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-09-16T14:46:03.528 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:03.529 INFO:teuthology.orchestra.run.smithi040.stdout:Total 11 MB/s | 29 MB 00:02 2024-09-16T14:46:03.593 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-16T14:46:03.612 INFO:teuthology.orchestra.run.smithi114.stdout:(6/33): ceph-osd-18.2.4-845.ge4e227b3.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-16T14:46:03.645 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-16T14:46:03.655 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:46:03.689 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-16T14:46:03.699 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:46:03.699 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:46:03.718 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-16T14:46:03.753 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-16T14:46:03.820 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-16T14:46:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-16T14:46:03.895 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-16T14:46:03.930 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-16T14:46:03.968 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-16T14:46:04.041 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-16T14:46:04.125 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:46:04.126 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:46:04.181 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-16T14:46:04.258 INFO:teuthology.orchestra.run.smithi114.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 242 kB/s | 253 kB 00:01 2024-09-16T14:46:04.274 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-16T14:46:04.299 INFO:teuthology.orchestra.run.smithi114.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 158 kB/s | 106 kB 00:00 2024-09-16T14:46:04.328 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-16T14:46:04.358 INFO:teuthology.orchestra.run.smithi114.stdout:(9/33): python3-cryptography-36.0.1-4.el9.x86_6 1.1 MB/s | 1.2 MB 00:01 2024-09-16T14:46:04.368 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-16T14:46:04.424 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-16T14:46:04.480 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-16T14:46:04.665 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-16T14:46:04.713 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-16T14:46:04.783 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-16T14:46:04.874 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-16T14:46:04.913 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-16T14:46:04.983 INFO:teuthology.orchestra.run.smithi114.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 183 kB/s | 125 kB 00:00 2024-09-16T14:46:05.008 INFO:teuthology.orchestra.run.smithi114.stdout:(11/33): python3-pycparser-2.20-6.el9.noarch.rp 180 kB/s | 135 kB 00:00 2024-09-16T14:46:05.032 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-16T14:46:05.118 INFO:teuthology.orchestra.run.smithi114.stdout:(12/33): python3-urllib3-1.26.5-6.el9.noarch.rp 283 kB/s | 215 kB 00:00 2024-09-16T14:46:05.142 INFO:teuthology.orchestra.run.smithi114.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 260 kB/s | 35 kB 00:00 2024-09-16T14:46:05.167 INFO:teuthology.orchestra.run.smithi114.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 938 kB/s | 172 kB 00:00 2024-09-16T14:46:05.195 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.n 28/33 2024-09-16T14:46:05.201 INFO:teuthology.orchestra.run.smithi114.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 611 kB/s | 51 kB 00:00 2024-09-16T14:46:05.224 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 29/33 2024-09-16T14:46:05.235 INFO:teuthology.orchestra.run.smithi114.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 649 kB/s | 43 kB 00:00 2024-09-16T14:46:05.249 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 29/33 2024-09-16T14:46:05.250 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:05.250 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T14:46:05.250 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T14:46:05.250 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T14:46:05.250 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:05.260 INFO:teuthology.orchestra.run.smithi114.stdout:(17/33): python3-toml-0.10.2-6.el9.noarch.rpm 354 kB/s | 42 kB 00:00 2024-09-16T14:46:05.293 INFO:teuthology.orchestra.run.smithi114.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.9 MB/s | 172 kB 00:00 2024-09-16T14:46:05.319 INFO:teuthology.orchestra.run.smithi114.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 4.3 MB/s | 358 kB 00:00 2024-09-16T14:46:05.336 INFO:teuthology.orchestra.run.smithi114.stdout:(20/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 142 kB/s | 11 kB 00:00 2024-09-16T14:46:05.352 INFO:teuthology.orchestra.run.smithi114.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 309 kB/s | 18 kB 00:00 2024-09-16T14:46:05.369 INFO:teuthology.orchestra.run.smithi114.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-16T14:46:05.386 INFO:teuthology.orchestra.run.smithi114.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-16T14:46:05.412 INFO:teuthology.orchestra.run.smithi114.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 335 kB/s | 20 kB 00:00 2024-09-16T14:46:05.429 INFO:teuthology.orchestra.run.smithi114.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2024-09-16T14:46:05.454 INFO:teuthology.orchestra.run.smithi114.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.1 MB/s | 79 kB 00:00 2024-09-16T14:46:05.467 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:46:05.479 INFO:teuthology.orchestra.run.smithi114.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-16T14:46:05.496 INFO:teuthology.orchestra.run.smithi114.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-09-16T14:46:05.521 INFO:teuthology.orchestra.run.smithi114.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 854 kB/s | 36 kB 00:00 2024-09-16T14:46:05.534 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-16T14:46:05.555 INFO:teuthology.orchestra.run.smithi114.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-16T14:46:05.567 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-16T14:46:05.580 INFO:teuthology.orchestra.run.smithi114.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 811 kB/s | 20 kB 00:00 2024-09-16T14:46:05.600 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-16T14:46:05.605 INFO:teuthology.orchestra.run.smithi114.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-16T14:46:05.631 INFO:teuthology.orchestra.run.smithi114.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-16T14:46:05.636 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:05.636 INFO:teuthology.orchestra.run.smithi114.stdout:Total 8.5 MB/s | 29 MB 00:03 2024-09-16T14:46:05.698 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-16T14:46:05.757 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:46:05.785 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-16T14:46:05.794 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:46:05.794 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:46:05.837 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-16T14:46:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-16T14:46:05.904 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-16T14:46:05.939 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-16T14:46:05.972 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-16T14:46:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-16T14:46:06.056 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-16T14:46:06.090 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-16T14:46:06.129 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-16T14:46:06.206 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-16T14:46:06.221 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:46:06.221 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:46:06.356 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-16T14:46:06.439 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-16T14:46:06.491 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-16T14:46:06.528 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-16T14:46:06.587 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-16T14:46:06.651 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-16T14:46:06.717 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 30/33 2024-09-16T14:46:06.744 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 30/33 2024-09-16T14:46:06.744 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:06.744 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T14:46:06.745 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T14:46:06.745 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T14:46:06.745 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:06.846 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-16T14:46:06.899 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-16T14:46:07.005 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-16T14:46:07.094 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-16T14:46:07.131 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-16T14:46:07.192 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 31/33 2024-09-16T14:46:07.215 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 31/33 2024-09-16T14:46:07.216 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:07.216 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T14:46:07.216 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T14:46:07.216 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T14:46:07.216 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:07.263 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-16T14:46:07.418 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 32/33 2024-09-16T14:46:07.429 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.n 28/33 2024-09-16T14:46:07.441 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 32/33 2024-09-16T14:46:07.441 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:07.441 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T14:46:07.441 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T14:46:07.442 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T14:46:07.442 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:07.458 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 29/33 2024-09-16T14:46:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:46:07.486 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 29/33 2024-09-16T14:46:07.486 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:07.486 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T14:46:07.486 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T14:46:07.486 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T14:46:07.486 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:07.507 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 33/33 2024-09-16T14:46:07.550 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-16T14:46:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-16T14:46:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-16T14:46:07.708 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-16T14:46:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-16T14:46:07.826 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-16T14:46:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-16T14:46:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-16T14:46:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-16T14:46:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-16T14:46:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-16T14:46:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-16T14:46:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-16T14:46:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-16T14:46:08.188 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-16T14:46:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-16T14:46:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-16T14:46:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-16T14:46:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-16T14:46:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-16T14:46:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-16T14:46:08.814 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-16T14:46:08.870 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-16T14:46:08.939 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-16T14:46:08.983 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 30/33 2024-09-16T14:46:09.012 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 30/33 2024-09-16T14:46:09.013 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:09.013 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T14:46:09.013 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T14:46:09.013 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T14:46:09.013 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:09.026 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-16T14:46:09.060 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-16T14:46:09.184 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-16T14:46:09.330 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 33/33 2024-09-16T14:46:09.330 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 1/33 2024-09-16T14:46:09.330 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 2/33 2024-09-16T14:46:09.330 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 3/33 2024-09-16T14:46:09.330 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 4/33 2024-09-16T14:46:09.330 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 5/33 2024-09-16T14:46:09.330 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.n 6/33 2024-09-16T14:46:09.330 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-16T14:46:09.330 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-16T14:46:09.330 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-16T14:46:09.331 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-16T14:46:09.331 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-16T14:46:09.331 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-16T14:46:09.331 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-16T14:46:09.331 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-16T14:46:09.331 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-16T14:46:09.331 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-16T14:46:09.331 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-16T14:46:09.331 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-16T14:46:09.331 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-16T14:46:09.331 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-16T14:46:09.331 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-16T14:46:09.331 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-16T14:46:09.332 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-16T14:46:09.332 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-16T14:46:09.332 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-16T14:46:09.333 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-16T14:46:09.333 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-16T14:46:09.333 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-16T14:46:09.333 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-16T14:46:09.333 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-16T14:46:09.333 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-16T14:46:09.333 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-16T14:46:09.335 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.n 28/33 2024-09-16T14:46:09.362 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 29/33 2024-09-16T14:46:09.388 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 29/33 2024-09-16T14:46:09.388 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:09.388 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T14:46:09.388 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T14:46:09.388 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T14:46:09.388 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:09.486 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 31/33 2024-09-16T14:46:09.511 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 31/33 2024-09-16T14:46:09.511 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:09.511 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T14:46:09.511 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T14:46:09.512 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T14:46:09.512 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:09.720 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 32/33 2024-09-16T14:46:09.745 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 32/33 2024-09-16T14:46:09.745 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:09.745 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T14:46:09.745 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T14:46:09.745 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T14:46:09.745 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:09.811 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 33/33 2024-09-16T14:46:10.251 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-16T14:46:10.251 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:10.251 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:46:10.251 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:10.251 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:10.252 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:10.252 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:10.252 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:10.252 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:10.252 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T14:46:10.252 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T14:46:10.252 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T14:46:10.252 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T14:46:10.252 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T14:46:10.252 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T14:46:10.252 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T14:46:10.252 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T14:46:10.252 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T14:46:10.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T14:46:10.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T14:46:10.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T14:46:10.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T14:46:10.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T14:46:10.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T14:46:10.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T14:46:10.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T14:46:10.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T14:46:10.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T14:46:10.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T14:46:10.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T14:46:10.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T14:46:10.254 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T14:46:10.254 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T14:46:10.254 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T14:46:10.254 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T14:46:10.254 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T14:46:10.254 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:10.254 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:10.645 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-base 2024-09-16T14:46:10.805 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 30/33 2024-09-16T14:46:10.834 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 30/33 2024-09-16T14:46:10.834 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:10.834 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T14:46:10.834 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T14:46:10.834 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T14:46:10.834 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:11.217 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:00:50 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:11.298 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 31/33 2024-09-16T14:46:11.307 INFO:teuthology.orchestra.run.smithi111.stdout:Package ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:11.324 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 31/33 2024-09-16T14:46:11.324 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:11.325 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T14:46:11.325 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T14:46:11.325 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T14:46:11.325 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:11.348 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:11.349 INFO:teuthology.orchestra.run.smithi111.stdout:Nothing to do. 2024-09-16T14:46:11.349 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:11.414 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install cephadm 2024-09-16T14:46:11.524 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 32/33 2024-09-16T14:46:11.548 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 32/33 2024-09-16T14:46:11.549 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:11.549 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T14:46:11.549 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T14:46:11.549 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T14:46:11.549 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:11.574 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 33/33 2024-09-16T14:46:11.574 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 1/33 2024-09-16T14:46:11.574 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 2/33 2024-09-16T14:46:11.574 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 3/33 2024-09-16T14:46:11.574 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 4/33 2024-09-16T14:46:11.574 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 5/33 2024-09-16T14:46:11.574 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.n 6/33 2024-09-16T14:46:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-16T14:46:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-16T14:46:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-16T14:46:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-16T14:46:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-16T14:46:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-16T14:46:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-16T14:46:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-16T14:46:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-16T14:46:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-16T14:46:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-16T14:46:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-16T14:46:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-16T14:46:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-16T14:46:11.576 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-16T14:46:11.576 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-16T14:46:11.576 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-16T14:46:11.576 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-16T14:46:11.576 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-16T14:46:11.576 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-16T14:46:11.576 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-16T14:46:11.576 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-16T14:46:11.576 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-16T14:46:11.576 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-16T14:46:11.576 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-16T14:46:11.576 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-16T14:46:11.613 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 33/33 2024-09-16T14:46:11.970 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:00:50 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:12.101 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:12.102 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:12.102 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-16T14:46:12.102 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:12.102 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:46:12.103 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 226 k 2024-09-16T14:46:12.103 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:12.103 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:46:12.103 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:12.103 INFO:teuthology.orchestra.run.smithi111.stdout:Install 1 Package 2024-09-16T14:46:12.103 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:12.103 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 226 k 2024-09-16T14:46:12.103 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 217 k 2024-09-16T14:46:12.103 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:46:12.324 INFO:teuthology.orchestra.run.smithi111.stdout:cephadm-18.2.4-845.ge4e227b3.el9.noarch.rpm 1.0 MB/s | 226 kB 00:00 2024-09-16T14:46:12.325 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:12.325 INFO:teuthology.orchestra.run.smithi111.stdout:Total 1.0 MB/s | 226 kB 00:00 2024-09-16T14:46:12.325 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:46:12.330 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:46:12.330 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:46:12.333 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:46:12.333 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:46:12.400 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:46:12.481 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-16T14:46:12.482 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:12.482 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:46:12.482 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:12.482 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:12.482 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:12.483 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:12.483 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:12.483 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:12.483 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T14:46:12.483 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T14:46:12.483 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T14:46:12.484 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T14:46:12.484 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T14:46:12.484 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T14:46:12.484 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T14:46:12.484 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T14:46:12.484 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T14:46:12.485 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T14:46:12.485 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T14:46:12.485 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T14:46:12.485 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T14:46:12.485 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T14:46:12.486 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T14:46:12.486 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T14:46:12.486 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T14:46:12.486 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T14:46:12.487 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T14:46:12.487 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T14:46:12.487 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T14:46:12.487 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T14:46:12.487 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T14:46:12.487 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T14:46:12.487 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T14:46:12.488 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T14:46:12.488 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T14:46:12.488 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:12.488 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:46:12.820 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:46:12.852 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-base 2024-09-16T14:46:12.918 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:46:13.282 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 33/33 2024-09-16T14:46:13.282 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 1/33 2024-09-16T14:46:13.283 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 2/33 2024-09-16T14:46:13.283 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 3/33 2024-09-16T14:46:13.283 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 4/33 2024-09-16T14:46:13.283 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 5/33 2024-09-16T14:46:13.283 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.n 6/33 2024-09-16T14:46:13.283 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-16T14:46:13.283 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-16T14:46:13.283 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-16T14:46:13.283 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-16T14:46:13.283 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-16T14:46:13.283 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-16T14:46:13.284 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-16T14:46:13.284 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-16T14:46:13.284 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-16T14:46:13.284 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-16T14:46:13.284 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-16T14:46:13.284 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-16T14:46:13.284 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-16T14:46:13.284 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-16T14:46:13.284 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-16T14:46:13.284 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-16T14:46:13.284 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-16T14:46:13.284 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-16T14:46:13.285 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-16T14:46:13.285 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-16T14:46:13.285 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-16T14:46:13.285 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-16T14:46:13.285 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-16T14:46:13.285 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-16T14:46:13.285 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-16T14:46:13.285 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-16T14:46:13.303 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:46:13.444 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:52 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:13.536 INFO:teuthology.orchestra.run.smithi040.stdout:Package ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:46:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:13.537 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:46:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:13.537 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:13.581 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:46:13.582 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-16T14:46:13.582 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:46:13.641 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install cephadm 2024-09-16T14:46:13.751 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T14:46:14.199 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-16T14:46:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:14.200 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:46:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T14:46:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T14:46:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T14:46:14.200 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T14:46:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T14:46:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T14:46:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T14:46:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T14:46:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T14:46:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T14:46:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T14:46:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T14:46:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T14:46:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T14:46:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T14:46:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T14:46:14.201 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T14:46:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T14:46:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T14:46:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T14:46:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T14:46:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T14:46:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T14:46:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T14:46:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T14:46:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T14:46:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T14:46:14.202 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:14.202 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:14.228 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:53 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:14.320 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:00:53 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:14.366 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:46:14.366 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:14.367 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-16T14:46:14.367 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:14.367 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:46:14.367 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 226 k 2024-09-16T14:46:14.367 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:14.367 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:46:14.367 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:14.367 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-16T14:46:14.367 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:14.367 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 226 k 2024-09-16T14:46:14.367 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 217 k 2024-09-16T14:46:14.367 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:46:14.452 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:14.453 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:14.453 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repo Size 2024-09-16T14:46:14.453 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:14.453 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:46:14.453 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 143 k 2024-09-16T14:46:14.453 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:14.453 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:46:14.453 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:14.453 INFO:teuthology.orchestra.run.smithi111.stdout:Install 1 Package 2024-09-16T14:46:14.454 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:14.454 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 143 k 2024-09-16T14:46:14.454 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 395 k 2024-09-16T14:46:14.454 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:46:14.586 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-base 2024-09-16T14:46:14.587 INFO:teuthology.orchestra.run.smithi040.stdout:cephadm-18.2.4-845.ge4e227b3.el9.noarch.rpm 1.0 MB/s | 226 kB 00:00 2024-09-16T14:46:14.587 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:14.587 INFO:teuthology.orchestra.run.smithi040.stdout:Total 1.0 MB/s | 226 kB 00:00 2024-09-16T14:46:14.587 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:46:14.593 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:46:14.593 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:46:14.595 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:46:14.596 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:46:14.655 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-immutable-object-cache-18.2.4-845.ge4e227b 711 kB/s | 143 kB 00:00 2024-09-16T14:46:14.656 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:14.656 INFO:teuthology.orchestra.run.smithi111.stdout:Total 703 kB/s | 143 kB 00:00 2024-09-16T14:46:14.656 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:46:14.664 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:46:14.664 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:46:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:46:14.721 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:46:14.721 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:46:14.873 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:46:14.956 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:46:14.980 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:46:14.980 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:14.980 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T14:46:14.981 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:15.083 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:46:15.159 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:50 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:15.184 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:46:15.249 INFO:teuthology.orchestra.run.smithi114.stdout:Package ceph-base-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:15.292 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:15.293 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-16T14:46:15.293 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:15.362 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install cephadm 2024-09-16T14:46:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:46:15.830 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:46:15.830 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:15.830 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:46:15.830 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:15.830 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:15.830 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:46:15.924 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:50 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:15.966 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:46:15.966 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:15.966 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:46:15.966 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:15.966 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:15.967 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:16.021 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T14:46:16.054 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:16.055 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:16.055 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-16T14:46:16.055 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:16.055 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:46:16.055 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 226 k 2024-09-16T14:46:16.055 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:16.055 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:46:16.055 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:16.055 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-16T14:46:16.056 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:16.056 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 226 k 2024-09-16T14:46:16.056 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 217 k 2024-09-16T14:46:16.056 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:46:16.180 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-mgr 2024-09-16T14:46:16.264 INFO:teuthology.orchestra.run.smithi114.stdout:cephadm-18.2.4-845.ge4e227b3.el9.noarch.rpm 1.1 MB/s | 226 kB 00:00 2024-09-16T14:46:16.264 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:16.265 INFO:teuthology.orchestra.run.smithi114.stdout:Total 1.1 MB/s | 226 kB 00:00 2024-09-16T14:46:16.265 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:46:16.270 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:46:16.270 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:46:16.276 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:46:16.276 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:46:16.343 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:46:16.610 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:55 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:16.749 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:46:16.749 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-09-16T14:46:16.750 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:16.750 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:46:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 143 k 2024-09-16T14:46:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:16.750 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:46:16.750 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:16.750 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-16T14:46:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:16.750 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 143 k 2024-09-16T14:46:16.750 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 395 k 2024-09-16T14:46:16.750 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:46:16.757 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:00:55 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:16.801 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:46:16.847 INFO:teuthology.orchestra.run.smithi111.stdout:Package ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:16.889 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:16.890 INFO:teuthology.orchestra.run.smithi111.stdout:Nothing to do. 2024-09-16T14:46:16.890 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:16.891 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:46:16.954 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T14:46:16.975 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-immutable-object-cache-18.2.4-845.ge4e227b 636 kB/s | 143 kB 00:00 2024-09-16T14:46:16.975 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:16.976 INFO:teuthology.orchestra.run.smithi040.stdout:Total 632 kB/s | 143 kB 00:00 2024-09-16T14:46:16.976 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:46:16.984 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:46:16.984 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:46:17.045 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:46:17.045 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:46:17.214 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:46:17.283 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:46:17.284 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:46:17.311 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:46:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:17.311 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T14:46:17.311 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:17.510 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:00:56 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:17.525 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/1 2024-09-16T14:46:17.525 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:17.525 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:46:17.525 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:17.525 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:17.525 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:17.641 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:17.642 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:17.642 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-16T14:46:17.642 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:17.642 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:46:17.642 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 3.5 M 2024-09-16T14:46:17.642 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-16T14:46:17.642 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 24 k 2024-09-16T14:46:17.642 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 16 k 2024-09-16T14:46:17.643 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T14:46:17.643 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T14:46:17.643 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:17.643 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:46:17.643 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:17.643 INFO:teuthology.orchestra.run.smithi111.stdout:Install 5 Packages 2024-09-16T14:46:17.643 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:17.644 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 3.8 M 2024-09-16T14:46:17.644 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 87 M 2024-09-16T14:46:17.644 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:46:17.776 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T14:46:17.992 INFO:teuthology.orchestra.run.smithi111.stdout:(1/5): ceph-prometheus-alerts-18.2.4-845.ge4e22 106 kB/s | 16 kB 00:00 2024-09-16T14:46:18.017 INFO:teuthology.orchestra.run.smithi111.stdout:(2/5): ceph-grafana-dashboards-18.2.4-845.ge4e2 136 kB/s | 24 kB 00:00 2024-09-16T14:46:18.059 INFO:teuthology.orchestra.run.smithi111.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 461 kB/s | 31 kB 00:00 2024-09-16T14:46:18.092 INFO:teuthology.orchestra.run.smithi111.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.5 MB/s | 188 kB 00:00 2024-09-16T14:46:18.193 INFO:teuthology.orchestra.run.smithi111.stdout:(5/5): ceph-mgr-dashboard-18.2.4-845.ge4e227b3. 9.9 MB/s | 3.5 MB 00:00 2024-09-16T14:46:18.196 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:18.196 INFO:teuthology.orchestra.run.smithi111.stdout:Total 6.8 MB/s | 3.8 MB 00:00 2024-09-16T14:46:18.196 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:46:18.203 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:46:18.203 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:46:18.210 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:46:18.211 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:18.211 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:46:18.211 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:18.211 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:18.211 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:46:18.287 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:46:18.288 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:46:18.352 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:53 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:18.420 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr 2024-09-16T14:46:18.484 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:18.485 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:18.485 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repo Size 2024-09-16T14:46:18.486 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:18.486 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:46:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 143 k 2024-09-16T14:46:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:18.486 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:46:18.486 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:18.486 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-16T14:46:18.486 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:18.486 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 143 k 2024-09-16T14:46:18.486 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 395 k 2024-09-16T14:46:18.486 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:46:18.531 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:46:18.617 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-16T14:46:18.647 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-16T14:46:18.676 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.no 3/5 2024-09-16T14:46:18.704 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-immutable-object-cache-18.2.4-845.ge4e227b 655 kB/s | 143 kB 00:00 2024-09-16T14:46:18.705 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:18.705 INFO:teuthology.orchestra.run.smithi114.stdout:Total 651 kB/s | 143 kB 00:00 2024-09-16T14:46:18.705 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:46:18.712 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:46:18.712 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:46:18.769 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:46:18.770 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:46:18.925 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:46:18.997 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:46:19.011 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:58 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:46:19.022 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:19.022 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T14:46:19.022 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:19.104 INFO:teuthology.orchestra.run.smithi040.stdout:Package ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:19.148 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:46:19.149 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-16T14:46:19.149 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:46:19.220 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T14:46:19.721 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.n 4/5 2024-09-16T14:46:19.771 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 5/5 2024-09-16T14:46:19.805 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:58 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:19.943 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:46:19.944 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:19.944 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-16T14:46:19.944 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:19.944 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:46:19.944 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 3.5 M 2024-09-16T14:46:19.944 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-16T14:46:19.944 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 24 k 2024-09-16T14:46:19.944 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 16 k 2024-09-16T14:46:19.944 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T14:46:19.944 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T14:46:19.944 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:19.945 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:46:19.945 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:19.945 INFO:teuthology.orchestra.run.smithi040.stdout:Install 5 Packages 2024-09-16T14:46:19.945 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:19.945 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 3.8 M 2024-09-16T14:46:19.945 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 87 M 2024-09-16T14:46:19.946 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:46:19.992 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.e 1/1 2024-09-16T14:46:19.992 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:19.992 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:46:19.992 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:19.993 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:19.993 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:20.156 INFO:teuthology.orchestra.run.smithi040.stdout:(1/5): ceph-grafana-dashboards-18.2.4-845.ge4e2 163 kB/s | 24 kB 00:00 2024-09-16T14:46:20.180 INFO:teuthology.orchestra.run.smithi040.stdout:(2/5): ceph-prometheus-alerts-18.2.4-845.ge4e22 94 kB/s | 16 kB 00:00 2024-09-16T14:46:20.198 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr 2024-09-16T14:46:20.231 INFO:teuthology.orchestra.run.smithi040.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 408 kB/s | 31 kB 00:00 2024-09-16T14:46:20.273 INFO:teuthology.orchestra.run.smithi040.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.0 MB/s | 188 kB 00:00 2024-09-16T14:46:20.381 INFO:teuthology.orchestra.run.smithi040.stdout:(5/5): ceph-mgr-dashboard-18.2.4-845.ge4e227b3. 9.4 MB/s | 3.5 MB 00:00 2024-09-16T14:46:20.384 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:20.385 INFO:teuthology.orchestra.run.smithi040.stdout:Total 8.6 MB/s | 3.8 MB 00:00 2024-09-16T14:46:20.385 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:46:20.393 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:46:20.393 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:46:20.493 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:46:20.493 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:46:20.747 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:46:20.762 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:55 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:20.839 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-16T14:46:20.853 INFO:teuthology.orchestra.run.smithi114.stdout:Package ceph-mgr-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:20.866 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 5/5 2024-09-16T14:46:20.867 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.n 1/5 2024-09-16T14:46:20.867 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 2/5 2024-09-16T14:46:20.867 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.no 3/5 2024-09-16T14:46:20.867 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-16T14:46:20.869 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-16T14:46:20.890 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.no 3/5 2024-09-16T14:46:20.898 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:20.899 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-16T14:46:20.899 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:20.981 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T14:46:21.148 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-16T14:46:21.149 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:21.149 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:46:21.149 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:21.149 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:21.149 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:21.150 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T14:46:21.150 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T14:46:21.150 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:21.150 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:21.363 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T14:46:21.549 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:00:56 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:21.680 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:21.681 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:21.681 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-16T14:46:21.681 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:21.681 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:46:21.681 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 3.5 M 2024-09-16T14:46:21.681 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-16T14:46:21.681 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 24 k 2024-09-16T14:46:21.681 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 16 k 2024-09-16T14:46:21.681 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T14:46:21.681 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T14:46:21.681 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:21.681 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:46:21.681 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:21.682 INFO:teuthology.orchestra.run.smithi114.stdout:Install 5 Packages 2024-09-16T14:46:21.682 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:21.682 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 3.8 M 2024-09-16T14:46:21.682 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 87 M 2024-09-16T14:46:21.682 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:46:21.900 INFO:teuthology.orchestra.run.smithi114.stdout:(1/5): ceph-grafana-dashboards-18.2.4-845.ge4e2 156 kB/s | 24 kB 00:00 2024-09-16T14:46:22.151 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.n 4/5 2024-09-16T14:46:22.151 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 5/5 2024-09-16T14:46:22.151 INFO:teuthology.orchestra.run.smithi114.stdout:(2/5): ceph-prometheus-alerts-18.2.4-845.ge4e22 91 kB/s | 16 kB 00:00 2024-09-16T14:46:22.151 INFO:teuthology.orchestra.run.smithi114.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 460 kB/s | 31 kB 00:00 2024-09-16T14:46:22.151 INFO:teuthology.orchestra.run.smithi114.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.4 MB/s | 188 kB 00:00 2024-09-16T14:46:22.152 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:00 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:22.152 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:22.152 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-16T14:46:22.152 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-16T14:46:22.152 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-16T14:46:22.152 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:46:22.152 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 7.4 M 2024-09-16T14:46:22.152 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-16T14:46:22.152 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T14:46:22.152 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T14:46:22.153 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T14:46:22.153 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T14:46:22.153 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T14:46:22.153 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T14:46:22.153 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T14:46:22.153 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T14:46:22.153 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T14:46:22.153 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T14:46:22.153 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T14:46:22.153 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:22.153 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:46:22.153 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-16T14:46:22.153 INFO:teuthology.orchestra.run.smithi111.stdout:Install 12 Packages 2024-09-16T14:46:22.153 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:22.154 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 38 M 2024-09-16T14:46:22.154 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 212 M 2024-09-16T14:46:22.154 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:46:22.168 INFO:teuthology.orchestra.run.smithi114.stdout:(5/5): ceph-mgr-dashboard-18.2.4-845.ge4e227b3. 8.3 MB/s | 3.5 MB 00:00 2024-09-16T14:46:22.171 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:22.171 INFO:teuthology.orchestra.run.smithi114.stdout:Total 7.7 MB/s | 3.8 MB 00:00 2024-09-16T14:46:22.172 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:46:22.179 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:46:22.179 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:46:22.278 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:46:22.279 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:46:22.291 INFO:teuthology.orchestra.run.smithi111.stdout:(1/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.9 MB/s | 801 kB 00:00 2024-09-16T14:46:22.316 INFO:teuthology.orchestra.run.smithi111.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-16T14:46:22.358 INFO:teuthology.orchestra.run.smithi111.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-09-16T14:46:22.425 INFO:teuthology.orchestra.run.smithi111.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 273 kB/s | 18 kB 00:00 2024-09-16T14:46:22.501 INFO:teuthology.orchestra.run.smithi111.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 16 MB/s | 3.0 MB 00:00 2024-09-16T14:46:22.526 INFO:teuthology.orchestra.run.smithi111.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 383 kB/s | 38 kB 00:00 2024-09-16T14:46:22.542 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:46:22.624 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-16T14:46:22.635 INFO:teuthology.orchestra.run.smithi111.stdout:(7/12): ceph-mgr-diskprediction-local-18.2.4-84 15 MB/s | 7.4 MB 00:00 2024-09-16T14:46:22.660 INFO:teuthology.orchestra.run.smithi111.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-16T14:46:22.664 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-16T14:46:22.693 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.no 3/5 2024-09-16T14:46:22.744 INFO:teuthology.orchestra.run.smithi111.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 20 MB/s | 4.9 MB 00:00 2024-09-16T14:46:22.769 INFO:teuthology.orchestra.run.smithi111.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.3 MB/s | 250 kB 00:00 2024-09-16T14:46:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 5/5 2024-09-16T14:46:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.n 1/5 2024-09-16T14:46:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 2/5 2024-09-16T14:46:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.no 3/5 2024-09-16T14:46:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-16T14:46:23.127 INFO:teuthology.orchestra.run.smithi111.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-09-16T14:46:23.328 INFO:teuthology.orchestra.run.smithi111.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.4 MB/s | 5.1 MB 00:00 2024-09-16T14:46:23.328 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:23.329 INFO:teuthology.orchestra.run.smithi111.stdout:Total 30 MB/s | 38 MB 00:01 2024-09-16T14:46:23.385 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-16T14:46:23.386 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:23.386 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:46:23.386 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:23.386 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:23.386 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:23.386 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T14:46:23.386 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T14:46:23.386 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:23.386 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:46:23.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T14:46:23.680 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:46:23.705 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:46:23.705 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:46:23.739 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.n 4/5 2024-09-16T14:46:23.780 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 5/5 2024-09-16T14:46:24.190 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:03 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:24.198 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:46:24.198 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:46:24.331 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:46:24.332 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2024-09-16T14:46:24.332 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-16T14:46:24.332 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2024-09-16T14:46:24.332 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:46:24.332 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 7.4 M 2024-09-16T14:46:24.333 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-16T14:46:24.333 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T14:46:24.333 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T14:46:24.333 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T14:46:24.333 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T14:46:24.333 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T14:46:24.333 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T14:46:24.333 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T14:46:24.333 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T14:46:24.333 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T14:46:24.333 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T14:46:24.333 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T14:46:24.333 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:24.334 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:46:24.334 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2024-09-16T14:46:24.334 INFO:teuthology.orchestra.run.smithi040.stdout:Install 12 Packages 2024-09-16T14:46:24.334 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:24.334 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 38 M 2024-09-16T14:46:24.334 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 212 M 2024-09-16T14:46:24.334 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:46:24.730 INFO:teuthology.orchestra.run.smithi040.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 773 kB/s | 191 kB 00:00 2024-09-16T14:46:24.822 INFO:teuthology.orchestra.run.smithi040.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-16T14:46:24.930 INFO:teuthology.orchestra.run.smithi040.stdout:(3/12): ceph-mgr-diskprediction-local-18.2.4-84 16 MB/s | 7.4 MB 00:00 2024-09-16T14:46:24.936 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 5/5 2024-09-16T14:46:24.936 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.n 1/5 2024-09-16T14:46:24.936 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 2/5 2024-09-16T14:46:24.936 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.no 3/5 2024-09-16T14:46:24.936 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-16T14:46:24.952 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:46:25.014 INFO:teuthology.orchestra.run.smithi040.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 115 kB/s | 33 kB 00:00 2024-09-16T14:46:25.021 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T14:46:25.067 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T14:46:25.166 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T14:46:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-16T14:46:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:25.173 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:46:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T14:46:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T14:46:25.173 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:25.173 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:25.206 INFO:teuthology.orchestra.run.smithi040.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 200 kB/s | 38 kB 00:00 2024-09-16T14:46:25.210 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T14:46:25.223 INFO:teuthology.orchestra.run.smithi040.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 62 kB/s | 18 kB 00:00 2024-09-16T14:46:25.385 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T14:46:25.393 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T14:46:25.426 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T14:46:25.503 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T14:46:25.681 INFO:teuthology.orchestra.run.smithi040.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 534 kB/s | 245 kB 00:00 2024-09-16T14:46:25.882 INFO:teuthology.orchestra.run.smithi040.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.8 MB/s | 3.0 MB 00:01 2024-09-16T14:46:25.942 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:00 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:25.999 INFO:teuthology.orchestra.run.smithi040.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-09-16T14:46:26.075 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:26.076 INFO:teuthology.orchestra.run.smithi114.stdout:====================================================================================== 2024-09-16T14:46:26.076 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-16T14:46:26.076 INFO:teuthology.orchestra.run.smithi114.stdout:====================================================================================== 2024-09-16T14:46:26.077 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:46:26.077 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 7.4 M 2024-09-16T14:46:26.077 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-16T14:46:26.077 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T14:46:26.077 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T14:46:26.077 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T14:46:26.077 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T14:46:26.077 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T14:46:26.077 INFO:teuthology.orchestra.run.smithi114.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T14:46:26.077 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T14:46:26.077 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T14:46:26.078 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T14:46:26.078 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T14:46:26.078 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T14:46:26.078 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:26.078 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:46:26.078 INFO:teuthology.orchestra.run.smithi114.stdout:====================================================================================== 2024-09-16T14:46:26.078 INFO:teuthology.orchestra.run.smithi114.stdout:Install 12 Packages 2024-09-16T14:46:26.078 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:26.079 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 38 M 2024-09-16T14:46:26.079 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 212 M 2024-09-16T14:46:26.079 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:46:26.183 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T14:46:26.299 INFO:teuthology.orchestra.run.smithi040.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.5 MB/s | 4.9 MB 00:01 2024-09-16T14:46:26.300 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T14:46:26.409 INFO:teuthology.orchestra.run.smithi114.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-16T14:46:26.451 INFO:teuthology.orchestra.run.smithi114.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-16T14:46:26.501 INFO:teuthology.orchestra.run.smithi114.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 355 kB/s | 33 kB 00:00 2024-09-16T14:46:26.543 INFO:teuthology.orchestra.run.smithi114.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 436 kB/s | 18 kB 00:00 2024-09-16T14:46:26.585 INFO:teuthology.orchestra.run.smithi114.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 920 kB/s | 38 kB 00:00 2024-09-16T14:46:26.735 INFO:teuthology.orchestra.run.smithi114.stdout:(6/12): ceph-mgr-diskprediction-local-18.2.4-84 14 MB/s | 7.4 MB 00:00 2024-09-16T14:46:26.794 INFO:teuthology.orchestra.run.smithi114.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.7 MB/s | 3.0 MB 00:00 2024-09-16T14:46:26.952 INFO:teuthology.orchestra.run.smithi114.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-09-16T14:46:27.020 INFO:teuthology.orchestra.run.smithi114.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 11 MB/s | 4.9 MB 00:00 2024-09-16T14:46:27.045 INFO:teuthology.orchestra.run.smithi114.stdout:(10/12): python3-devel-3.9.19-8.el9.x86_64.rpm 792 kB/s | 245 kB 00:00 2024-09-16T14:46:27.071 INFO:teuthology.orchestra.run.smithi114.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.1 MB/s | 250 kB 00:00 2024-09-16T14:46:27.370 INFO:teuthology.orchestra.run.smithi114.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-09-16T14:46:27.371 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:27.371 INFO:teuthology.orchestra.run.smithi114.stdout:Total 30 MB/s | 38 MB 00:01 2024-09-16T14:46:27.573 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T14:46:27.722 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:46:27.748 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:46:27.748 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:46:27.999 INFO:teuthology.orchestra.run.smithi040.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.2 MB/s | 5.1 MB 00:02 2024-09-16T14:46:28.239 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:46:28.239 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:46:28.606 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T14:46:28.635 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 12/12 2024-09-16T14:46:29.000 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:46:29.075 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T14:46:29.109 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T14:46:29.210 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T14:46:29.245 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T14:46:29.429 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T14:46:29.485 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T14:46:29.570 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T14:46:30.237 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T14:46:30.339 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T14:46:30.649 INFO:teuthology.orchestra.run.smithi040.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.5 MB/s | 16 MB 00:04 2024-09-16T14:46:30.650 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:30.650 INFO:teuthology.orchestra.run.smithi040.stdout:Total 6.0 MB/s | 38 MB 00:06 2024-09-16T14:46:31.022 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:46:31.050 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:46:31.050 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:46:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 12/12 2024-09-16T14:46:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 1/12 2024-09-16T14:46:31.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T14:46:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T14:46:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T14:46:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T14:46:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T14:46:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T14:46:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T14:46:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T14:46:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T14:46:31.159 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T14:46:31.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T14:46:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:31.543 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:46:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T14:46:31.543 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T14:46:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T14:46:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T14:46:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T14:46:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T14:46:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T14:46:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T14:46:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T14:46:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T14:46:31.544 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T14:46:31.545 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:31.545 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:46:31.561 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:46:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T14:46:31.923 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-mgr-rook 2024-09-16T14:46:32.346 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:46:32.424 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T14:46:32.464 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T14:46:32.495 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:11 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T14:46:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T14:46:32.617 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T14:46:32.628 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:32.629 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:32.629 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-16T14:46:32.629 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:32.629 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:46:32.629 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 49 k 2024-09-16T14:46:32.630 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-16T14:46:32.630 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T14:46:32.630 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T14:46:32.630 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T14:46:32.630 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T14:46:32.630 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T14:46:32.630 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T14:46:32.630 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T14:46:32.630 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T14:46:32.630 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T14:46:32.630 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T14:46:32.630 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T14:46:32.630 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T14:46:32.631 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:32.631 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:46:32.631 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:32.631 INFO:teuthology.orchestra.run.smithi111.stdout:Install 13 Packages 2024-09-16T14:46:32.631 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:32.631 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 2.2 M 2024-09-16T14:46:32.631 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 26 M 2024-09-16T14:46:32.631 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:46:32.644 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 12/12 2024-09-16T14:46:32.797 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T14:46:32.830 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T14:46:32.909 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T14:46:33.201 INFO:teuthology.orchestra.run.smithi111.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 213 kB/s | 26 kB 00:00 2024-09-16T14:46:33.226 INFO:teuthology.orchestra.run.smithi111.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 132 kB/s | 19 kB 00:00 2024-09-16T14:46:33.251 INFO:teuthology.orchestra.run.smithi111.stdout:(3/13): ceph-mgr-rook-18.2.4-845.ge4e227b3.el9. 289 kB/s | 49 kB 00:00 2024-09-16T14:46:33.368 INFO:teuthology.orchestra.run.smithi111.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-16T14:46:33.394 INFO:teuthology.orchestra.run.smithi111.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.1 MB/s | 222 kB 00:00 2024-09-16T14:46:33.428 INFO:teuthology.orchestra.run.smithi111.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 909 kB/s | 54 kB 00:00 2024-09-16T14:46:33.461 INFO:teuthology.orchestra.run.smithi111.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 479 kB/s | 32 kB 00:00 2024-09-16T14:46:33.495 INFO:teuthology.orchestra.run.smithi111.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-09-16T14:46:33.520 INFO:teuthology.orchestra.run.smithi111.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 156 kB/s | 14 kB 00:00 2024-09-16T14:46:33.554 INFO:teuthology.orchestra.run.smithi111.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.4 MB/s | 220 kB 00:00 2024-09-16T14:46:33.579 INFO:teuthology.orchestra.run.smithi111.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-16T14:46:33.604 INFO:teuthology.orchestra.run.smithi111.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-16T14:46:33.721 INFO:teuthology.orchestra.run.smithi111.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.5 MB/s | 1.0 MB 00:00 2024-09-16T14:46:33.723 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:33.724 INFO:teuthology.orchestra.run.smithi111.stdout:Total 2.0 MB/s | 2.2 MB 00:01 2024-09-16T14:46:33.786 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:46:33.801 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:46:33.801 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:46:33.840 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T14:46:33.887 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:46:33.888 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:46:33.949 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T14:46:34.394 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:46:34.568 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T14:46:34.621 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T14:46:34.677 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T14:46:34.723 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T14:46:34.764 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T14:46:34.866 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T14:46:34.966 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T14:46:35.024 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T14:46:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 12/12 2024-09-16T14:46:35.238 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 1/12 2024-09-16T14:46:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T14:46:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T14:46:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T14:46:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T14:46:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T14:46:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T14:46:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T14:46:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T14:46:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T14:46:35.239 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T14:46:35.295 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T14:46:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T14:46:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:35.682 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:46:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T14:46:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T14:46:35.682 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T14:46:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T14:46:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T14:46:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T14:46:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T14:46:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T14:46:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T14:46:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T14:46:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T14:46:35.683 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:35.683 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:35.735 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T14:46:35.788 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T14:46:35.821 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T14:46:35.869 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T14:46:35.891 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 13/13 2024-09-16T14:46:36.034 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-rook 2024-09-16T14:46:36.392 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T14:46:36.421 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 12/12 2024-09-16T14:46:36.605 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:11 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:36.738 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:36.739 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 13/13 2024-09-16T14:46:36.739 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 1/13 2024-09-16T14:46:36.740 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T14:46:36.740 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T14:46:36.740 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T14:46:36.740 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T14:46:36.740 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T14:46:36.740 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T14:46:36.740 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T14:46:36.740 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T14:46:36.740 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T14:46:36.740 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T14:46:36.740 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T14:46:36.741 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:36.741 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-16T14:46:36.741 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:36.742 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:46:36.742 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 49 k 2024-09-16T14:46:36.742 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-16T14:46:36.742 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T14:46:36.742 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T14:46:36.742 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T14:46:36.742 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout:Install 13 Packages 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 2.2 M 2024-09-16T14:46:36.743 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 26 M 2024-09-16T14:46:36.744 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:46:36.986 INFO:teuthology.orchestra.run.smithi114.stdout:(1/13): ceph-mgr-rook-18.2.4-845.ge4e227b3.el9. 299 kB/s | 49 kB 00:00 2024-09-16T14:46:37.069 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T14:46:37.069 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:37.069 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:46:37.069 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:37.069 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T14:46:37.069 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T14:46:37.069 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T14:46:37.069 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T14:46:37.069 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T14:46:37.069 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T14:46:37.070 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T14:46:37.070 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T14:46:37.070 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T14:46:37.070 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T14:46:37.070 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T14:46:37.070 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T14:46:37.070 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:37.070 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:37.087 INFO:teuthology.orchestra.run.smithi114.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 72 kB/s | 19 kB 00:00 2024-09-16T14:46:37.112 INFO:teuthology.orchestra.run.smithi114.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 88 kB/s | 26 kB 00:00 2024-09-16T14:46:37.328 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T14:46:37.504 INFO:teuthology.orchestra.run.smithi114.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 429 kB/s | 222 kB 00:00 2024-09-16T14:46:37.579 INFO:teuthology.orchestra.run.smithi114.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 322 kB/s | 159 kB 00:00 2024-09-16T14:46:37.613 INFO:teuthology.orchestra.run.smithi114.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 557 kB/s | 279 kB 00:00 2024-09-16T14:46:37.663 INFO:teuthology.orchestra.run.smithi114.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 385 kB/s | 32 kB 00:00 2024-09-16T14:46:37.688 INFO:teuthology.orchestra.run.smithi114.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 192 kB/s | 14 kB 00:00 2024-09-16T14:46:37.722 INFO:teuthology.orchestra.run.smithi114.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 3.8 MB/s | 220 kB 00:00 2024-09-16T14:46:37.755 INFO:teuthology.orchestra.run.smithi114.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-09-16T14:46:37.789 INFO:teuthology.orchestra.run.smithi114.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.6 MB/s | 90 kB 00:00 2024-09-16T14:46:37.839 INFO:teuthology.orchestra.run.smithi114.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-16T14:46:37.864 INFO:teuthology.orchestra.run.smithi114.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 149 kB/s | 54 kB 00:00 2024-09-16T14:46:37.867 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:37.868 INFO:teuthology.orchestra.run.smithi114.stdout:Total 2.0 MB/s | 2.2 MB 00:01 2024-09-16T14:46:37.896 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:16 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:37.938 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:46:37.953 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:46:37.954 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:46:38.027 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:38.028 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:38.028 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-16T14:46:38.028 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:38.029 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:46:38.029 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 139 k 2024-09-16T14:46:38.029 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-16T14:46:38.029 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T14:46:38.029 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T14:46:38.029 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T14:46:38.029 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T14:46:38.029 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T14:46:38.029 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:38.029 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:46:38.029 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:38.029 INFO:teuthology.orchestra.run.smithi111.stdout:Install 6 Packages 2024-09-16T14:46:38.029 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:38.030 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 7.0 M 2024-09-16T14:46:38.030 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 33 M 2024-09-16T14:46:38.030 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:46:38.041 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:46:38.041 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:46:38.481 INFO:teuthology.orchestra.run.smithi111.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.8 MB/s | 248 kB 00:00 2024-09-16T14:46:38.538 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:46:38.540 INFO:teuthology.orchestra.run.smithi111.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-16T14:46:38.565 INFO:teuthology.orchestra.run.smithi111.stdout:(3/6): ceph-mgr-cephadm-18.2.4-845.ge4e227b3.el 625 kB/s | 139 kB 00:00 2024-09-16T14:46:38.591 INFO:teuthology.orchestra.run.smithi111.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T14:46:38.632 INFO:teuthology.orchestra.run.smithi111.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-16T14:46:38.710 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T14:46:38.716 INFO:teuthology.orchestra.run.smithi111.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-16T14:46:38.720 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:38.720 INFO:teuthology.orchestra.run.smithi111.stdout:Total 10 MB/s | 7.0 MB 00:00 2024-09-16T14:46:38.775 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T14:46:38.790 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:46:38.800 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:46:38.801 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:46:38.847 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T14:46:38.884 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T14:46:38.898 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:46:38.899 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:46:38.925 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T14:46:38.979 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 12/12 2024-09-16T14:46:38.979 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227 1/12 2024-09-16T14:46:38.979 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T14:46:38.979 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T14:46:38.979 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T14:46:38.979 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T14:46:38.979 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T14:46:38.979 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T14:46:38.979 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T14:46:38.979 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T14:46:38.979 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T14:46:38.979 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T14:46:39.026 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T14:46:39.137 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T14:46:39.185 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T14:46:39.207 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:46:39.383 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T14:46:39.410 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T14:46:39.410 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:39.410 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:46:39.410 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:39.410 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T14:46:39.411 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T14:46:39.411 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T14:46:39.411 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T14:46:39.411 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T14:46:39.411 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T14:46:39.411 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T14:46:39.411 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T14:46:39.411 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T14:46:39.411 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T14:46:39.411 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T14:46:39.411 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:39.411 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:46:39.426 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T14:46:39.822 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-rook 2024-09-16T14:46:39.899 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T14:46:39.949 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T14:46:39.949 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T14:46:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T14:46:40.022 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T14:46:40.038 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T14:46:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 13/13 2024-09-16T14:46:40.107 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T14:46:40.130 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 6/6 2024-09-16T14:46:40.418 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:19 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:40.558 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:46:40.559 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:40.559 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-16T14:46:40.559 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:40.559 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:46:40.559 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 49 k 2024-09-16T14:46:40.559 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-16T14:46:40.559 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T14:46:40.559 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T14:46:40.559 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T14:46:40.560 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T14:46:40.560 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T14:46:40.560 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T14:46:40.560 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T14:46:40.560 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T14:46:40.560 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T14:46:40.560 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T14:46:40.560 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T14:46:40.560 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T14:46:40.560 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:40.561 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:46:40.561 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:40.561 INFO:teuthology.orchestra.run.smithi040.stdout:Install 13 Packages 2024-09-16T14:46:40.561 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:40.561 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 2.2 M 2024-09-16T14:46:40.562 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 26 M 2024-09-16T14:46:40.562 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:46:40.865 INFO:teuthology.orchestra.run.smithi040.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 223 kB/s | 19 kB 00:00 2024-09-16T14:46:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 13/13 2024-09-16T14:46:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 1/13 2024-09-16T14:46:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T14:46:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T14:46:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T14:46:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T14:46:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T14:46:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T14:46:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T14:46:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T14:46:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T14:46:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T14:46:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T14:46:40.890 INFO:teuthology.orchestra.run.smithi040.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 230 kB/s | 26 kB 00:00 2024-09-16T14:46:40.929 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 6/6 2024-09-16T14:46:40.930 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/6 2024-09-16T14:46:40.930 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T14:46:40.930 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T14:46:40.930 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T14:46:40.930 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T14:46:40.949 INFO:teuthology.orchestra.run.smithi040.stdout:(3/13): ceph-mgr-rook-18.2.4-845.ge4e227b3.el9. 290 kB/s | 49 kB 00:00 2024-09-16T14:46:40.982 INFO:teuthology.orchestra.run.smithi040.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-09-16T14:46:41.007 INFO:teuthology.orchestra.run.smithi040.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-16T14:46:41.033 INFO:teuthology.orchestra.run.smithi040.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-16T14:46:41.075 INFO:teuthology.orchestra.run.smithi040.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-16T14:46:41.109 INFO:teuthology.orchestra.run.smithi040.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 188 kB/s | 14 kB 00:00 2024-09-16T14:46:41.134 INFO:teuthology.orchestra.run.smithi040.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-09-16T14:46:41.159 INFO:teuthology.orchestra.run.smithi040.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-16T14:46:41.192 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T14:46:41.192 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:41.192 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:46:41.192 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:41.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T14:46:41.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T14:46:41.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T14:46:41.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T14:46:41.192 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T14:46:41.193 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:41.193 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:41.201 INFO:teuthology.orchestra.run.smithi040.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.6 MB/s | 90 kB 00:00 2024-09-16T14:46:41.212 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T14:46:41.212 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:41.212 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:46:41.212 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:41.212 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T14:46:41.212 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T14:46:41.212 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T14:46:41.212 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T14:46:41.212 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T14:46:41.212 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T14:46:41.213 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T14:46:41.213 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T14:46:41.213 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T14:46:41.213 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T14:46:41.213 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T14:46:41.213 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T14:46:41.213 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:41.213 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:41.226 INFO:teuthology.orchestra.run.smithi040.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-09-16T14:46:41.260 INFO:teuthology.orchestra.run.smithi040.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-16T14:46:41.263 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:41.263 INFO:teuthology.orchestra.run.smithi040.stdout:Total 3.2 MB/s | 2.2 MB 00:00 2024-09-16T14:46:41.329 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:46:41.345 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:46:41.345 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:46:41.420 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-fuse 2024-09-16T14:46:41.441 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:46:41.443 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:46:41.476 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T14:46:41.969 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:46:41.993 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:20 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:42.048 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:17 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:42.124 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:42.125 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:42.125 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-16T14:46:42.125 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:42.125 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:46:42.125 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 852 k 2024-09-16T14:46:42.125 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-16T14:46:42.125 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T14:46:42.125 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:42.125 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:46:42.125 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:42.125 INFO:teuthology.orchestra.run.smithi111.stdout:Install 2 Packages 2024-09-16T14:46:42.125 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:42.125 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 932 k 2024-09-16T14:46:42.126 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 2.7 M 2024-09-16T14:46:42.126 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:46:42.139 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T14:46:42.181 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:42.182 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:42.182 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-16T14:46:42.183 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:42.183 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:46:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 139 k 2024-09-16T14:46:42.183 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-16T14:46:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T14:46:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T14:46:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T14:46:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T14:46:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T14:46:42.183 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:42.184 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:46:42.184 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:42.184 INFO:teuthology.orchestra.run.smithi114.stdout:Install 6 Packages 2024-09-16T14:46:42.184 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:42.184 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 7.0 M 2024-09-16T14:46:42.184 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 33 M 2024-09-16T14:46:42.184 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:46:42.196 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T14:46:42.252 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T14:46:42.287 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T14:46:42.330 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T14:46:42.435 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T14:46:42.546 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T14:46:42.551 INFO:teuthology.orchestra.run.smithi111.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 547 kB/s | 80 kB 00:00 2024-09-16T14:46:42.599 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T14:46:42.684 INFO:teuthology.orchestra.run.smithi111.stdout:(2/2): ceph-fuse-18.2.4-845.ge4e227b3.el9.x86_6 3.0 MB/s | 852 kB 00:00 2024-09-16T14:46:42.685 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:42.685 INFO:teuthology.orchestra.run.smithi111.stdout:Total 1.6 MB/s | 932 kB 00:00 2024-09-16T14:46:42.697 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:46:42.719 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:46:42.720 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:46:42.800 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:46:42.801 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:46:42.909 INFO:teuthology.orchestra.run.smithi114.stdout:(1/6): ceph-mgr-cephadm-18.2.4-845.ge4e227b3.el 713 kB/s | 139 kB 00:00 2024-09-16T14:46:42.934 INFO:teuthology.orchestra.run.smithi114.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-09-16T14:46:42.984 INFO:teuthology.orchestra.run.smithi114.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T14:46:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:46:43.018 INFO:teuthology.orchestra.run.smithi114.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-16T14:46:43.043 INFO:teuthology.orchestra.run.smithi114.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.0 MB/s | 548 kB 00:00 2024-09-16T14:46:43.124 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T14:46:43.201 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2/2 2024-09-16T14:46:43.391 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T14:46:43.444 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T14:46:43.478 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T14:46:43.527 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T14:46:43.556 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 13/13 2024-09-16T14:46:43.860 INFO:teuthology.orchestra.run.smithi114.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-09-16T14:46:43.861 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:43.862 INFO:teuthology.orchestra.run.smithi114.stdout:Total 4.2 MB/s | 7.0 MB 00:01 2024-09-16T14:46:43.918 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2/2 2024-09-16T14:46:43.918 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/2 2024-09-16T14:46:43.932 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:46:43.943 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:46:43.943 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:46:44.041 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:46:44.041 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:46:44.191 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T14:46:44.191 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:44.191 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:46:44.191 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T14:46:44.191 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:44.192 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:44.352 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:46:44.369 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 13/13 2024-09-16T14:46:44.369 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 1/13 2024-09-16T14:46:44.369 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T14:46:44.369 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T14:46:44.369 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T14:46:44.369 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T14:46:44.369 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T14:46:44.369 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T14:46:44.369 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T14:46:44.369 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T14:46:44.369 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T14:46:44.370 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T14:46:44.370 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T14:46:44.494 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-volume 2024-09-16T14:46:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T14:46:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T14:46:44.717 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T14:46:44.717 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:44.717 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:46:44.717 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T14:46:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T14:46:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T14:46:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T14:46:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T14:46:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T14:46:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T14:46:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T14:46:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T14:46:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T14:46:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T14:46:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T14:46:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:44.718 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:46:44.943 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T14:46:45.077 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:24 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:45.102 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T14:46:45.189 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T14:46:45.212 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:45.213 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:45.213 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-16T14:46:45.213 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:45.213 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:46:45.213 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 264 k 2024-09-16T14:46:45.213 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-16T14:46:45.213 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T14:46:45.213 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:45.213 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:46:45.213 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:45.214 INFO:teuthology.orchestra.run.smithi111.stdout:Install 2 Packages 2024-09-16T14:46:45.214 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:45.214 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 342 k 2024-09-16T14:46:45.214 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 1.5 M 2024-09-16T14:46:45.214 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:46:45.250 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T14:46:45.273 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 6/6 2024-09-16T14:46:45.535 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:24 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:45.673 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:46:45.674 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:45.674 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-16T14:46:45.674 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:45.675 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:46:45.675 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 139 k 2024-09-16T14:46:45.675 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-16T14:46:45.675 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T14:46:45.675 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T14:46:45.675 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T14:46:45.675 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T14:46:45.675 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T14:46:45.675 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:45.675 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:46:45.675 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:45.675 INFO:teuthology.orchestra.run.smithi040.stdout:Install 6 Packages 2024-09-16T14:46:45.675 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:45.676 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 7.0 M 2024-09-16T14:46:45.676 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 33 M 2024-09-16T14:46:45.676 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:46:45.698 INFO:teuthology.orchestra.run.smithi111.stdout:(1/2): ceph-volume-18.2.4-845.ge4e227b3.el9.noa 1.3 MB/s | 264 kB 00:00 2024-09-16T14:46:45.873 INFO:teuthology.orchestra.run.smithi111.stdout:(2/2): python3-packaging-20.9-5.el9.noarch.rpm 203 kB/s | 77 kB 00:00 2024-09-16T14:46:45.873 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:45.873 INFO:teuthology.orchestra.run.smithi111.stdout:Total 518 kB/s | 342 kB 00:00 2024-09-16T14:46:45.885 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:46:45.895 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:46:45.895 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:46:45.934 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:46:45.934 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:46:46.070 INFO:teuthology.orchestra.run.smithi040.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-09-16T14:46:46.080 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 6/6 2024-09-16T14:46:46.080 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/6 2024-09-16T14:46:46.080 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T14:46:46.080 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T14:46:46.080 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T14:46:46.080 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T14:46:46.080 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:46:46.196 INFO:teuthology.orchestra.run.smithi040.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-09-16T14:46:46.212 INFO:teuthology.orchestra.run.smithi040.stdout:(3/6): ceph-mgr-cephadm-18.2.4-845.ge4e227b3.el 445 kB/s | 139 kB 00:00 2024-09-16T14:46:46.255 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-16T14:46:46.263 INFO:teuthology.orchestra.run.smithi040.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 863 kB/s | 58 kB 00:00 2024-09-16T14:46:46.288 INFO:teuthology.orchestra.run.smithi040.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.5 MB/s | 548 kB 00:00 2024-09-16T14:46:46.292 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2/2 2024-09-16T14:46:46.304 INFO:teuthology.orchestra.run.smithi040.stdout:(6/6): python3-typing-extensions-4.12.2-2.el9.n 881 kB/s | 74 kB 00:00 2024-09-16T14:46:46.308 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:46.308 INFO:teuthology.orchestra.run.smithi040.stdout:Total 11 MB/s | 7.0 MB 00:00 2024-09-16T14:46:46.308 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2/2 2024-09-16T14:46:46.309 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:46.309 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T14:46:46.309 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:46.337 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T14:46:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:46.338 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:46:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T14:46:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T14:46:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T14:46:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T14:46:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T14:46:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:46.338 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:46.381 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:46:46.393 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:46:46.393 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:46:46.499 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:46:46.499 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:46:46.615 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-fuse 2024-09-16T14:46:46.818 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:46:46.995 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T14:46:47.050 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T14:46:47.058 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 1/2 2024-09-16T14:46:47.185 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:22 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:47.315 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:47.316 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:47.316 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-16T14:46:47.316 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:47.316 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:46:47.316 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 852 k 2024-09-16T14:46:47.316 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-16T14:46:47.316 INFO:teuthology.orchestra.run.smithi114.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T14:46:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:47.317 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:46:47.317 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:47.317 INFO:teuthology.orchestra.run.smithi114.stdout:Install 2 Packages 2024-09-16T14:46:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:47.317 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 932 k 2024-09-16T14:46:47.317 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 2.7 M 2024-09-16T14:46:47.317 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:46:47.368 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-16T14:46:47.368 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:47.368 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:46:47.368 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:47.368 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T14:46:47.368 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:47.368 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:47.436 INFO:teuthology.orchestra.run.smithi114.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 815 kB/s | 80 kB 00:00 2024-09-16T14:46:47.603 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T14:46:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:(2/2): ceph-fuse-18.2.4-845.ge4e227b3.el9.x86_6 2.8 MB/s | 852 kB 00:00 2024-09-16T14:46:47.637 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:47.638 INFO:teuthology.orchestra.run.smithi114.stdout:Total 2.8 MB/s | 932 kB 00:00 2024-09-16T14:46:47.649 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:46:47.662 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install librados-devel 2024-09-16T14:46:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:46:47.671 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:46:47.696 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T14:46:47.750 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:46:47.750 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:46:47.766 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T14:46:47.793 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 6/6 2024-09-16T14:46:47.944 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:46:48.085 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T14:46:48.152 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2/2 2024-09-16T14:46:48.241 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:27 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:48.372 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:48.373 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repo Size 2024-09-16T14:46:48.374 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:48.374 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:46:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 127 k 2024-09-16T14:46:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:48.374 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:46:48.374 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:48.374 INFO:teuthology.orchestra.run.smithi111.stdout:Install 1 Package 2024-09-16T14:46:48.374 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:48.374 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 127 k 2024-09-16T14:46:48.374 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 456 k 2024-09-16T14:46:48.374 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:46:48.518 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 6/6 2024-09-16T14:46:48.518 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 1/6 2024-09-16T14:46:48.518 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T14:46:48.518 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T14:46:48.519 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T14:46:48.519 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T14:46:48.589 INFO:teuthology.orchestra.run.smithi111.stdout:librados-devel-18.2.4-845.ge4e227b3.el9.x86_64. 596 kB/s | 127 kB 00:00 2024-09-16T14:46:48.589 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:48.589 INFO:teuthology.orchestra.run.smithi111.stdout:Total 592 kB/s | 127 kB 00:00 2024-09-16T14:46:48.589 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:46:48.593 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:46:48.593 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:46:48.657 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:46:48.658 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:46:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T14:46:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:48.791 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:46:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T14:46:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T14:46:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T14:46:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T14:46:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T14:46:48.791 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:48.791 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:46:48.808 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:46:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2/2 2024-09-16T14:46:48.846 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/2 2024-09-16T14:46:48.909 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:49.028 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-fuse 2024-09-16T14:46:49.121 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T14:46:49.121 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:49.121 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:46:49.121 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T14:46:49.121 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:49.122 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:49.357 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:49.437 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-volume 2024-09-16T14:46:49.598 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:49.599 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:49.599 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:46:49.599 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:49.599 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:49.599 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:49.620 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:28 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:49.759 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:46:49.760 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:49.760 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-16T14:46:49.760 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:49.760 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:46:49.760 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 852 k 2024-09-16T14:46:49.760 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-16T14:46:49.760 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T14:46:49.760 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:49.760 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:46:49.760 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:49.760 INFO:teuthology.orchestra.run.smithi040.stdout:Install 2 Packages 2024-09-16T14:46:49.761 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:49.761 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 932 k 2024-09-16T14:46:49.761 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 2.7 M 2024-09-16T14:46:49.761 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:46:49.813 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install libcephfs2 2024-09-16T14:46:50.017 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:25 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:50.131 INFO:teuthology.orchestra.run.smithi040.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 863 kB/s | 80 kB 00:00 2024-09-16T14:46:50.156 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:50.157 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-16T14:46:50.157 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:50.157 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:46:50.157 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 264 k 2024-09-16T14:46:50.158 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-16T14:46:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T14:46:50.158 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:50.158 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:46:50.158 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:50.158 INFO:teuthology.orchestra.run.smithi114.stdout:Install 2 Packages 2024-09-16T14:46:50.159 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:50.159 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 342 k 2024-09-16T14:46:50.159 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 1.5 M 2024-09-16T14:46:50.159 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:46:50.315 INFO:teuthology.orchestra.run.smithi040.stdout:(2/2): ceph-fuse-18.2.4-845.ge4e227b3.el9.x86_6 3.0 MB/s | 852 kB 00:00 2024-09-16T14:46:50.315 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:50.316 INFO:teuthology.orchestra.run.smithi040.stdout:Total 1.6 MB/s | 932 kB 00:00 2024-09-16T14:46:50.328 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:46:50.352 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:46:50.352 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:46:50.391 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:29 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:50.434 INFO:teuthology.orchestra.run.smithi114.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 506 kB/s | 77 kB 00:00 2024-09-16T14:46:50.437 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:46:50.437 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:46:50.483 INFO:teuthology.orchestra.run.smithi111.stdout:Package libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:50.509 INFO:teuthology.orchestra.run.smithi114.stdout:(2/2): ceph-volume-18.2.4-845.ge4e227b3.el9.noa 1.1 MB/s | 264 kB 00:00 2024-09-16T14:46:50.510 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:50.510 INFO:teuthology.orchestra.run.smithi114.stdout:Total 967 kB/s | 342 kB 00:00 2024-09-16T14:46:50.521 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:46:50.531 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:50.531 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:46:50.531 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:46:50.532 INFO:teuthology.orchestra.run.smithi111.stdout:Nothing to do. 2024-09-16T14:46:50.532 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:50.570 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:46:50.570 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:46:50.599 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install libcephfs-devel 2024-09-16T14:46:50.635 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:46:50.722 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:46:50.768 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T14:46:50.842 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2/2 2024-09-16T14:46:50.910 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-16T14:46:50.945 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2/2 2024-09-16T14:46:50.961 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2/2 2024-09-16T14:46:50.962 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:50.962 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T14:46:50.962 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:51.160 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:30 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:51.293 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:51.294 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:51.294 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repo Size 2024-09-16T14:46:51.295 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:51.295 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:46:51.295 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 31 k 2024-09-16T14:46:51.295 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:51.295 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:46:51.295 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:51.295 INFO:teuthology.orchestra.run.smithi111.stdout:Install 1 Package 2024-09-16T14:46:51.295 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:51.295 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 31 k 2024-09-16T14:46:51.295 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 139 k 2024-09-16T14:46:51.295 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:46:51.440 INFO:teuthology.orchestra.run.smithi111.stdout:libcephfs-devel-18.2.4-845.ge4e227b3.el9.x86_64 214 kB/s | 31 kB 00:00 2024-09-16T14:46:51.441 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:51.441 INFO:teuthology.orchestra.run.smithi111.stdout:Total 213 kB/s | 31 kB 00:00 2024-09-16T14:46:51.441 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:46:51.445 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:46:51.445 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:46:51.467 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:46:51.467 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:46:51.495 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2/2 2024-09-16T14:46:51.495 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/2 2024-09-16T14:46:51.556 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:46:51.669 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:51.726 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 1/2 2024-09-16T14:46:51.816 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T14:46:51.817 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:51.817 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:46:51.817 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T14:46:51.817 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:51.817 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:46:52.025 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-16T14:46:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:52.040 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:46:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T14:46:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:52.040 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:52.134 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-volume 2024-09-16T14:46:52.263 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:52.263 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:52.263 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:46:52.263 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:52.263 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:52.263 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:52.353 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install librados-devel 2024-09-16T14:46:52.436 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install librados2 2024-09-16T14:46:52.732 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:31 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:52.871 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:46:52.872 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:52.872 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-09-16T14:46:52.872 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:52.872 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:46:52.872 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume noarch 2:18.2.4-845.ge4e227b3.el9 ceph-noarch 264 k 2024-09-16T14:46:52.872 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-09-16T14:46:52.873 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T14:46:52.873 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:52.873 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:46:52.873 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:52.873 INFO:teuthology.orchestra.run.smithi040.stdout:Install 2 Packages 2024-09-16T14:46:52.873 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:52.873 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 342 k 2024-09-16T14:46:52.873 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 1.5 M 2024-09-16T14:46:52.873 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:46:52.932 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:27 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:53.001 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:31 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:53.089 INFO:teuthology.orchestra.run.smithi040.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 632 kB/s | 77 kB 00:00 2024-09-16T14:46:53.090 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:53.090 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:53.090 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repo Size 2024-09-16T14:46:53.090 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:53.090 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:46:53.090 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 127 k 2024-09-16T14:46:53.090 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:53.090 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:46:53.090 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:53.090 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-16T14:46:53.090 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:53.091 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 127 k 2024-09-16T14:46:53.091 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 456 k 2024-09-16T14:46:53.091 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:46:53.091 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:53.133 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:53.135 INFO:teuthology.orchestra.run.smithi111.stdout:Nothing to do. 2024-09-16T14:46:53.135 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:53.136 INFO:teuthology.orchestra.run.smithi040.stdout:(2/2): ceph-volume-18.2.4-845.ge4e227b3.el9.noa 1.1 MB/s | 264 kB 00:00 2024-09-16T14:46:53.137 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:53.137 INFO:teuthology.orchestra.run.smithi040.stdout:Total 1.3 MB/s | 342 kB 00:00 2024-09-16T14:46:53.149 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:46:53.159 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:46:53.160 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:46:53.198 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:46:53.198 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:46:53.205 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install librbd1 2024-09-16T14:46:53.290 INFO:teuthology.orchestra.run.smithi114.stdout:librados-devel-18.2.4-845.ge4e227b3.el9.x86_64. 586 kB/s | 127 kB 00:00 2024-09-16T14:46:53.290 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:53.291 INFO:teuthology.orchestra.run.smithi114.stdout:Total 581 kB/s | 127 kB 00:00 2024-09-16T14:46:53.291 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:46:53.296 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:46:53.296 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:46:53.352 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:46:53.358 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:46:53.359 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:46:53.512 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:46:53.536 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-16T14:46:53.564 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2/2 2024-09-16T14:46:53.581 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2/2 2024-09-16T14:46:53.581 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:46:53.581 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T14:46:53.581 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:53.654 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:53.768 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:53.858 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:53.902 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:53.902 INFO:teuthology.orchestra.run.smithi111.stdout:Nothing to do. 2024-09-16T14:46:53.903 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:53.979 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install python3-rados 2024-09-16T14:46:54.097 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 1/2 2024-09-16T14:46:54.350 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:54.350 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:54.350 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:46:54.350 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:54.350 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:54.350 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:54.547 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:33 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:54.598 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-16T14:46:54.599 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:54.599 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:46:54.599 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-volume-2:18.2.4-845.ge4e227b3.el9.noarch 2024-09-16T14:46:54.599 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T14:46:54.599 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:54.599 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:46:54.616 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install libcephfs2 2024-09-16T14:46:54.637 INFO:teuthology.orchestra.run.smithi111.stdout:Package python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:54.680 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:54.681 INFO:teuthology.orchestra.run.smithi111.stdout:Nothing to do. 2024-09-16T14:46:54.681 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:54.751 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install python3-rgw 2024-09-16T14:46:54.876 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librados-devel 2024-09-16T14:46:55.191 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:30 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:55.283 INFO:teuthology.orchestra.run.smithi114.stdout:Package libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:55.316 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:55.327 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:55.329 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-16T14:46:55.329 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:55.401 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install libcephfs-devel 2024-09-16T14:46:55.407 INFO:teuthology.orchestra.run.smithi111.stdout:Package python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:55.450 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:55.451 INFO:teuthology.orchestra.run.smithi111.stdout:Nothing to do. 2024-09-16T14:46:55.451 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:55.470 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:55.518 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install python3-cephfs 2024-09-16T14:46:55.608 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:46:55.609 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:55.609 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-09-16T14:46:55.609 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:55.609 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:46:55.609 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 127 k 2024-09-16T14:46:55.609 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:55.610 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:46:55.610 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:55.610 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-16T14:46:55.610 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:55.610 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 127 k 2024-09-16T14:46:55.610 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 456 k 2024-09-16T14:46:55.610 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:46:55.834 INFO:teuthology.orchestra.run.smithi040.stdout:librados-devel-18.2.4-845.ge4e227b3.el9.x86_64. 566 kB/s | 127 kB 00:00 2024-09-16T14:46:55.835 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:55.836 INFO:teuthology.orchestra.run.smithi040.stdout:Total 562 kB/s | 127 kB 00:00 2024-09-16T14:46:55.836 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:46:55.841 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:46:55.841 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:46:55.907 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:46:55.907 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:46:55.965 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:30 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:56.066 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:46:56.078 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:56.098 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:56.099 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:56.099 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repo Size 2024-09-16T14:46:56.099 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:56.099 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:46:56.099 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 31 k 2024-09-16T14:46:56.099 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:56.099 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:46:56.099 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:46:56.099 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-16T14:46:56.099 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:56.100 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 31 k 2024-09-16T14:46:56.100 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 139 k 2024-09-16T14:46:56.100 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:46:56.167 INFO:teuthology.orchestra.run.smithi111.stdout:Package python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:56.176 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:56.210 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:56.211 INFO:teuthology.orchestra.run.smithi111.stdout:Nothing to do. 2024-09-16T14:46:56.211 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:56.260 INFO:teuthology.orchestra.run.smithi114.stdout:libcephfs-devel-18.2.4-845.ge4e227b3.el9.x86_64 194 kB/s | 31 kB 00:00 2024-09-16T14:46:56.260 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:56.261 INFO:teuthology.orchestra.run.smithi114.stdout:Total 192 kB/s | 31 kB 00:00 2024-09-16T14:46:56.261 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:46:56.264 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:46:56.264 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:46:56.274 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install python3-rbd 2024-09-16T14:46:56.290 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:46:56.291 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:46:56.377 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:46:56.499 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:56.628 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:56.840 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:56.889 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:56.890 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:56.890 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:46:56.890 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:56.890 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:56.890 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:46:56.903 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:56.929 INFO:teuthology.orchestra.run.smithi111.stdout:Package python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:56.972 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:56.973 INFO:teuthology.orchestra.run.smithi111.stdout:Nothing to do. 2024-09-16T14:46:56.973 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:57.037 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install rbd-fuse 2024-09-16T14:46:57.124 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install libcephfs2 2024-09-16T14:46:57.126 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:57.127 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:57.127 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:46:57.127 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:57.127 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:46:57.127 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:57.323 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install librados2 2024-09-16T14:46:57.600 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:36 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:57.721 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:36 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:57.734 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:57.734 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:57.734 INFO:teuthology.orchestra.run.smithi111.stdout: Package Architecture Version Repository Size 2024-09-16T14:46:57.735 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:57.735 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:46:57.735 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 87 k 2024-09-16T14:46:57.735 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:57.735 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:46:57.735 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:57.735 INFO:teuthology.orchestra.run.smithi111.stdout:Install 1 Package 2024-09-16T14:46:57.735 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:57.736 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 87 k 2024-09-16T14:46:57.736 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 227 k 2024-09-16T14:46:57.736 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:46:57.816 INFO:teuthology.orchestra.run.smithi040.stdout:Package libcephfs2-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:57.862 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:46:57.863 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-16T14:46:57.863 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:46:57.889 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:57.923 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install libcephfs-devel 2024-09-16T14:46:57.935 INFO:teuthology.orchestra.run.smithi111.stdout:rbd-fuse-18.2.4-845.ge4e227b3.el9.x86_64.rpm 436 kB/s | 87 kB 00:00 2024-09-16T14:46:57.935 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:57.936 INFO:teuthology.orchestra.run.smithi111.stdout:Total 433 kB/s | 87 kB 00:00 2024-09-16T14:46:57.936 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:46:57.942 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:46:57.943 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:46:57.980 INFO:teuthology.orchestra.run.smithi114.stdout:Package librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:58.010 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:46:58.010 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:46:58.022 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:58.024 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-16T14:46:58.024 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:58.097 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install librbd1 2024-09-16T14:46:58.157 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:46:58.275 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:58.514 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:37 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:58.653 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:46:58.655 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:58.655 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-09-16T14:46:58.655 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:58.655 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:46:58.655 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 31 k 2024-09-16T14:46:58.656 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:58.656 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:46:58.656 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:46:58.656 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-16T14:46:58.656 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:58.656 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 31 k 2024-09-16T14:46:58.656 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 139 k 2024-09-16T14:46:58.657 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:46:58.662 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:33 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:58.751 INFO:teuthology.orchestra.run.smithi114.stdout:Package librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:58.794 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:58.794 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-16T14:46:58.795 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:58.804 INFO:teuthology.orchestra.run.smithi040.stdout:libcephfs-devel-18.2.4-845.ge4e227b3.el9.x86_64 208 kB/s | 31 kB 00:00 2024-09-16T14:46:58.805 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:46:58.805 INFO:teuthology.orchestra.run.smithi040.stdout:Total 206 kB/s | 31 kB 00:00 2024-09-16T14:46:58.805 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:46:58.809 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:46:58.809 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:46:58.834 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:46:58.835 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:46:58.863 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-rados 2024-09-16T14:46:58.933 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:58.934 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:58.934 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:46:58.934 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:58.934 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:58.934 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:46:59.052 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:46:59.170 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:59.203 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install rbd-mirror 2024-09-16T14:46:59.421 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:46:59.509 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:46:59.524 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:59.552 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:46:59.553 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-16T14:46:59.553 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:46:59.629 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-rgw 2024-09-16T14:46:59.777 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:38 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:46:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:46:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:59.780 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:46:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:46:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:46:59.780 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:46:59.911 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:46:59.912 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:59.912 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repo Size 2024-09-16T14:46:59.912 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:59.913 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:46:59.913 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.0 M 2024-09-16T14:46:59.913 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:59.913 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:46:59.913 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:46:59.913 INFO:teuthology.orchestra.run.smithi111.stdout:Install 1 Package 2024-09-16T14:46:59.913 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:46:59.913 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 3.0 M 2024-09-16T14:46:59.913 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 12 M 2024-09-16T14:46:59.913 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:46:59.957 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librados2 2024-09-16T14:47:00.185 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:47:00.255 INFO:teuthology.orchestra.run.smithi111.stdout:rbd-mirror-18.2.4-845.ge4e227b3.el9.x86_64.rpm 8.7 MB/s | 3.0 MB 00:00 2024-09-16T14:47:00.256 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:47:00.256 INFO:teuthology.orchestra.run.smithi111.stdout:Total 8.7 MB/s | 3.0 MB 00:00 2024-09-16T14:47:00.256 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:47:00.265 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:47:00.265 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:47:00.273 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:47:00.316 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:47:00.317 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-16T14:47:00.317 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:47:00.344 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:47:00.344 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:47:00.388 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-cephfs 2024-09-16T14:47:00.547 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:39 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:47:00.644 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:47:00.689 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:47:00.690 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-16T14:47:00.690 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:47:00.752 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librbd1 2024-09-16T14:47:00.775 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:47:00.839 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:00.864 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:00.864 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:47:00.865 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T14:47:00.865 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T14:47:00.865 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T14:47:00.865 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:00.948 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:47:01.037 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:47:01.079 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:47:01.080 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-16T14:47:01.080 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:47:01.146 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install python3-rbd 2024-09-16T14:47:01.343 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:40 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:47:01.436 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:47:01.482 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:47:01.483 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-16T14:47:01.483 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:47:01.542 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rados 2024-09-16T14:47:01.705 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:36 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:47:01.795 INFO:teuthology.orchestra.run.smithi114.stdout:Package python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:47:01.838 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:47:01.839 INFO:teuthology.orchestra.run.smithi114.stdout:Nothing to do. 2024-09-16T14:47:01.839 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:47:01.914 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install rbd-fuse 2024-09-16T14:47:01.923 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:01.923 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:01.923 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:47:01.923 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:47:01.923 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:01.923 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:47:02.137 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:41 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:47:02.160 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install rbd-nbd 2024-09-16T14:47:02.230 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rados-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:47:02.276 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:47:02.277 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-16T14:47:02.277 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:47:02.335 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rgw 2024-09-16T14:47:02.477 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:37 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:47:02.609 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:47:02.610 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:47:02.610 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-09-16T14:47:02.610 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:47:02.610 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:47:02.610 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 87 k 2024-09-16T14:47:02.610 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:02.610 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:47:02.610 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:47:02.610 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-16T14:47:02.611 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:02.611 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 87 k 2024-09-16T14:47:02.611 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 227 k 2024-09-16T14:47:02.611 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:47:02.732 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:01:41 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:47:02.797 INFO:teuthology.orchestra.run.smithi114.stdout:rbd-fuse-18.2.4-845.ge4e227b3.el9.x86_64.rpm 469 kB/s | 87 kB 00:00 2024-09-16T14:47:02.797 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:47:02.798 INFO:teuthology.orchestra.run.smithi114.stdout:Total 465 kB/s | 87 kB 00:00 2024-09-16T14:47:02.798 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:47:02.805 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:47:02.805 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:47:02.869 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-16T14:47:02.869 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:47:02.869 INFO:teuthology.orchestra.run.smithi111.stdout: Package Architecture Version Repository Size 2024-09-16T14:47:02.869 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:47:02.870 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-16T14:47:02.870 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 172 k 2024-09-16T14:47:02.870 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:02.870 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-16T14:47:02.870 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2024-09-16T14:47:02.870 INFO:teuthology.orchestra.run.smithi111.stdout:Install 1 Package 2024-09-16T14:47:02.870 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:02.871 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 172 k 2024-09-16T14:47:02.871 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 494 k 2024-09-16T14:47:02.871 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-16T14:47:02.872 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:47:02.872 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:47:02.926 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:41 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:47:03.019 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rgw-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:47:03.042 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:47:03.064 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:47:03.065 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-16T14:47:03.065 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:47:03.092 INFO:teuthology.orchestra.run.smithi111.stdout:rbd-nbd-18.2.4-845.ge4e227b3.el9.x86_64.rpm 779 kB/s | 172 kB 00:00 2024-09-16T14:47:03.092 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:47:03.093 INFO:teuthology.orchestra.run.smithi111.stdout:Total 773 kB/s | 172 kB 00:00 2024-09-16T14:47:03.093 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-16T14:47:03.100 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-16T14:47:03.100 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-16T14:47:03.125 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-cephfs 2024-09-16T14:47:03.144 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:03.169 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-16T14:47:03.169 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-16T14:47:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-16T14:47:03.439 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:03.515 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:03.707 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:42 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:47:03.765 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:03.765 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:03.765 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:47:03.765 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:47:03.765 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:03.765 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:47:03.800 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-cephfs-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:47:03.845 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:47:03.847 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-16T14:47:03.847 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:47:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:03.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rbd 2024-09-16T14:47:04.037 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install rbd-mirror 2024-09-16T14:47:04.099 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:04.099 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:04.099 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-16T14:47:04.099 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:47:04.099 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:04.099 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-16T14:47:04.336 DEBUG:teuthology.parallel:result is None 2024-09-16T14:47:04.499 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:43 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:47:04.591 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rbd-2:18.2.4-845.ge4e227b3.el9.x86_64 is already installed. 2024-09-16T14:47:04.610 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:39 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:47:04.636 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:47:04.637 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-09-16T14:47:04.637 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:47:04.701 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-fuse 2024-09-16T14:47:04.745 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:47:04.746 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:47:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repo Size 2024-09-16T14:47:04.746 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:47:04.746 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:47:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.0 M 2024-09-16T14:47:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:04.747 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:47:04.747 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:47:04.747 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-16T14:47:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:04.747 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 3.0 M 2024-09-16T14:47:04.747 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 12 M 2024-09-16T14:47:04.747 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:47:05.101 INFO:teuthology.orchestra.run.smithi114.stdout:rbd-mirror-18.2.4-845.ge4e227b3.el9.x86_64.rpm 8.5 MB/s | 3.0 MB 00:00 2024-09-16T14:47:05.101 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:47:05.102 INFO:teuthology.orchestra.run.smithi114.stdout:Total 8.4 MB/s | 3.0 MB 00:00 2024-09-16T14:47:05.102 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:47:05.111 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:47:05.111 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:47:05.191 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:47:05.191 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:47:05.294 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:44 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:47:05.433 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:47:05.434 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:47:05.434 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-09-16T14:47:05.434 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:47:05.434 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:47:05.434 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 87 k 2024-09-16T14:47:05.434 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:05.434 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:47:05.434 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:47:05.434 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-16T14:47:05.434 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:05.435 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 87 k 2024-09-16T14:47:05.435 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 227 k 2024-09-16T14:47:05.435 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:47:05.608 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:47:05.640 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-fuse-18.2.4-845.ge4e227b3.el9.x86_64.rpm 424 kB/s | 87 kB 00:00 2024-09-16T14:47:05.640 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:47:05.641 INFO:teuthology.orchestra.run.smithi040.stdout:Total 421 kB/s | 87 kB 00:00 2024-09-16T14:47:05.641 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:47:05.649 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:47:05.649 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:47:05.668 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:05.693 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:05.693 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:47:05.693 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T14:47:05.693 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T14:47:05.693 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T14:47:05.693 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:05.720 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:47:05.720 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:47:05.887 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:47:05.982 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:06.340 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:06.607 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:06.608 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:06.608 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:47:06.608 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:47:06.608 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:06.608 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:47:06.800 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:06.800 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:06.800 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:47:06.800 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:47:06.800 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:06.801 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:47:06.850 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-mirror 2024-09-16T14:47:07.080 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install rbd-nbd 2024-09-16T14:47:07.468 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:46 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:47:07.609 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:47:07.610 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:47:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-09-16T14:47:07.610 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:47:07.610 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:47:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 3.0 M 2024-09-16T14:47:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:07.610 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:47:07.610 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:47:07.610 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-16T14:47:07.610 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:07.611 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 3.0 M 2024-09-16T14:47:07.611 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 12 M 2024-09-16T14:47:07.611 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:47:07.657 INFO:teuthology.orchestra.run.smithi114.stdout:Last metadata expiration check: 0:01:42 ago on Mon 16 Sep 2024 02:45:25 PM UTC. 2024-09-16T14:47:07.793 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-16T14:47:07.794 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:47:07.794 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-09-16T14:47:07.795 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:47:07.795 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-16T14:47:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 172 k 2024-09-16T14:47:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:07.795 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-16T14:47:07.795 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-16T14:47:07.795 INFO:teuthology.orchestra.run.smithi114.stdout:Install 1 Package 2024-09-16T14:47:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:07.795 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 172 k 2024-09-16T14:47:07.795 INFO:teuthology.orchestra.run.smithi114.stdout:Installed size: 494 k 2024-09-16T14:47:07.795 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-16T14:47:08.013 INFO:teuthology.orchestra.run.smithi114.stdout:rbd-nbd-18.2.4-845.ge4e227b3.el9.x86_64.rpm 791 kB/s | 172 kB 00:00 2024-09-16T14:47:08.013 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:47:08.014 INFO:teuthology.orchestra.run.smithi114.stdout:Total 785 kB/s | 172 kB 00:00 2024-09-16T14:47:08.014 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-16T14:47:08.019 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-mirror-18.2.4-845.ge4e227b3.el9.x86_64.rpm 7.3 MB/s | 3.0 MB 00:00 2024-09-16T14:47:08.020 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:47:08.020 INFO:teuthology.orchestra.run.smithi040.stdout:Total 7.3 MB/s | 3.0 MB 00:00 2024-09-16T14:47:08.021 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:47:08.021 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-16T14:47:08.021 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-16T14:47:08.031 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:47:08.031 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:47:08.091 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-16T14:47:08.091 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-16T14:47:08.117 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:47:08.117 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:47:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-16T14:47:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:08.556 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:47:08.620 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:08.648 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:08.648 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T14:47:08.648 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T14:47:08.648 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T14:47:08.648 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T14:47:08.648 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:08.850 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:09.132 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:09.132 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:09.132 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-16T14:47:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:47:09.133 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:09.133 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-16T14:47:09.405 DEBUG:teuthology.parallel:result is None 2024-09-16T14:47:09.771 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:09.771 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:09.771 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:47:09.771 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:47:09.771 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:09.771 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:47:10.000 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-nbd 2024-09-16T14:47:10.595 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:49 ago on Mon 16 Sep 2024 02:45:21 PM UTC. 2024-09-16T14:47:10.735 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-09-16T14:47:10.736 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:47:10.736 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-09-16T14:47:10.736 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:47:10.736 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-09-16T14:47:10.736 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_64 2:18.2.4-845.ge4e227b3.el9 ceph 172 k 2024-09-16T14:47:10.736 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:10.736 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-09-16T14:47:10.736 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-09-16T14:47:10.736 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-09-16T14:47:10.736 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:10.737 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 172 k 2024-09-16T14:47:10.737 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 494 k 2024-09-16T14:47:10.737 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-09-16T14:47:10.956 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-nbd-18.2.4-845.ge4e227b3.el9.x86_64.rpm 784 kB/s | 172 kB 00:00 2024-09-16T14:47:10.957 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-09-16T14:47:10.957 INFO:teuthology.orchestra.run.smithi040.stdout:Total 779 kB/s | 172 kB 00:00 2024-09-16T14:47:10.957 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-09-16T14:47:10.965 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-09-16T14:47:10.965 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-09-16T14:47:11.039 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-09-16T14:47:11.039 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-09-16T14:47:11.212 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-09-16T14:47:11.306 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:11.732 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:11.996 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 1/1 2024-09-16T14:47:11.996 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:11.996 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-09-16T14:47:11.996 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:18.2.4-845.ge4e227b3.el9.x86_64 2024-09-16T14:47:11.996 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:11.996 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-09-16T14:47:12.219 DEBUG:teuthology.parallel:result is None 2024-09-16T14:47:12.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:47:12.363 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T14:47:12.385 INFO:teuthology.orchestra.run.smithi040.stdout:18.2.4-845.ge4e227b3.el9 2024-09-16T14:47:12.385 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-845.ge4e227b3.el9 2024-09-16T14:47:12.385 INFO:teuthology.task.install:The correct ceph version 18.2.4-845.ge4e227b3 is installed. 2024-09-16T14:47:12.388 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:47:12.528 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T14:47:12.553 INFO:teuthology.orchestra.run.smithi111.stdout:18.2.4-845.ge4e227b3.el9 2024-09-16T14:47:12.554 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-845.ge4e227b3.el9 2024-09-16T14:47:12.554 INFO:teuthology.task.install:The correct ceph version 18.2.4-845.ge4e227b3 is installed. 2024-09-16T14:47:12.556 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-16T14:47:12.700 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T14:47:12.726 INFO:teuthology.orchestra.run.smithi114.stdout:18.2.4-845.ge4e227b3.el9 2024-09-16T14:47:12.726 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-845.ge4e227b3.el9 2024-09-16T14:47:12.726 INFO:teuthology.task.install:The correct ceph version 18.2.4-845.ge4e227b3 is installed. 2024-09-16T14:47:12.728 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T14:47:12.729 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:47:12.729 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T14:47:12.758 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:47:12.758 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T14:47:12.788 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:47:12.789 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T14:47:12.820 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T14:47:12.820 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:47:12.820 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T14:47:12.847 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T14:47:12.914 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:47:12.914 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T14:47:12.943 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T14:47:13.012 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:47:13.013 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T14:47:13.042 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T14:47:13.112 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T14:47:13.112 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:47:13.112 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T14:47:13.139 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T14:47:13.206 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:47:13.206 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T14:47:13.236 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T14:47:13.303 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:47:13.304 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T14:47:13.333 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T14:47:13.402 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T14:47:13.402 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:47:13.403 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T14:47:13.431 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T14:47:13.498 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:47:13.498 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T14:47:13.523 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T14:47:13.587 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:47:13.588 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T14:47:13.618 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T14:47:13.688 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T14:47:13.767 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T14:47:13.767 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /var/log/ceph 2024-09-16T14:47:13.770 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /var/log/ceph 2024-09-16T14:47:13.772 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /var/log/ceph 2024-09-16T14:47:13.801 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T14:47:13.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T14:47:13.841 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T14:47:13.843 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T14:47:13.871 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T14:47:13.872 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T14:47:13.911 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T14:47:13.913 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T14:47:13.952 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T14:47:13.952 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T14:47:13.952 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764', 'branch': 'reef', 'description': 'rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7906764', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'ae7f37330c1a72493684eb1fcaed6f2a01195bb6'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 931, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a'], ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x'], ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c']], 'seed': 1390, 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'sleep_before_teardown': 0, 'subset': '5319/100000', 'suite': 'rados', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_ae7f37330c1a72493684eb1fcaed6f2a01195bb6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'ae7f37330c1a72493684eb1fcaed6f2a01195bb6', 'targets': {'smithi040.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFh27YyxVqFLxa26xlc0MXHu8TQcmlM5mzKb+6IJL+5SX5FmxZpYD5tJ0Zg2QmbXS6qsnc52ZeyjJyWcg3GijD8=', 'smithi111.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA7VsOOTMTAokhrlWnkWq1YXVXoL8Ajq6J8LAAXEnE78TtiIfL4rQzlyobhKhV6jQAmKBnzd0whMZIGJZMv2Gpc=', 'smithi114.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDAXtHXk2eiVWvBYb5Ctqzzm6lcQQ/gJDA46C9tOu43pC/bkUgPPNSQcWYZPmDg9ig0jaRBff65WMtESNXuJajs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'cluster': 'ceph'}}, {'thrashosds': {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200}}, {'rados': {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 4, 'l': 3, 'm': 2, 'name': 'lrcprofile', 'plugin': 'lrc'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_22:00:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T14:47:13.953 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T14:47:13.982 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T14:47:13.984 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T14:47:14.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T14:47:14.040 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T14:47:14.042 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T14:47:14.070 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:47:14.070 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T14:47:14.127 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T14:47:14.128 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2024-09-16T14:47:14.186 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T14:47:14.186 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:47:14.187 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T14:47:14.187 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:47:14.187 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:47:14.187 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-09-16 14:47:08.908726439 +0000 2024-09-16T14:47:14.187 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-09-16 14:44:43.831243010 +0000 2024-09-16T14:47:14.187 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-09-16 14:44:43.831243010 +0000 2024-09-16T14:47:14.187 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-09-16 14:44:43.831243010 +0000 2024-09-16T14:47:14.187 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T14:47:14.256 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-09-16T14:47:14.256 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-09-16T14:47:14.256 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000280262 s, 1.8 MB/s 2024-09-16T14:47:14.257 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T14:47:14.317 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2024-09-16T14:47:14.375 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T14:47:14.375 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:47:14.375 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T14:47:14.375 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:47:14.375 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:47:14.375 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-09-16 14:47:08.908726439 +0000 2024-09-16T14:47:14.375 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-09-16 14:44:44.178238596 +0000 2024-09-16T14:47:14.375 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-09-16 14:44:44.178238596 +0000 2024-09-16T14:47:14.375 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-09-16 14:44:44.178238596 +0000 2024-09-16T14:47:14.375 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T14:47:14.442 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-09-16T14:47:14.442 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-09-16T14:47:14.442 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000215667 s, 2.4 MB/s 2024-09-16T14:47:14.443 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T14:47:14.501 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2024-09-16T14:47:14.561 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T14:47:14.561 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:47:14.561 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-16T14:47:14.562 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:47:14.562 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:47:14.562 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-09-16 14:47:08.908726439 +0000 2024-09-16T14:47:14.562 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-09-16 14:44:44.506234425 +0000 2024-09-16T14:47:14.562 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-09-16 14:44:44.506234425 +0000 2024-09-16T14:47:14.562 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-09-16 14:44:44.506234425 +0000 2024-09-16T14:47:14.562 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T14:47:14.629 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-09-16T14:47:14.629 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-09-16T14:47:14.629 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000227592 s, 2.2 MB/s 2024-09-16T14:47:14.630 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T14:47:14.688 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2024-09-16T14:47:14.748 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T14:47:14.749 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:47:14.749 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-16T14:47:14.749 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:47:14.749 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:47:14.749 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-09-16 14:47:08.909726426 +0000 2024-09-16T14:47:14.749 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-09-16 14:44:44.828230329 +0000 2024-09-16T14:47:14.749 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-09-16 14:44:44.828230329 +0000 2024-09-16T14:47:14.749 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-09-16 14:44:44.828230329 +0000 2024-09-16T14:47:14.749 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T14:47:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-09-16T14:47:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-09-16T14:47:14.815 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000206024 s, 2.5 MB/s 2024-09-16T14:47:14.817 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T14:47:14.875 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:14.876 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:47:14.876 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T14:47:14.893 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T14:47:14.893 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2024-09-16T14:47:14.952 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T14:47:14.952 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:47:14.952 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-16T14:47:14.953 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:47:14.953 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:47:14.953 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-16 14:47:01.110239843 +0000 2024-09-16T14:47:14.953 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-16 14:44:40.379387899 +0000 2024-09-16T14:47:14.953 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-16 14:44:40.379387899 +0000 2024-09-16T14:47:14.953 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-16 14:44:40.379387899 +0000 2024-09-16T14:47:14.953 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T14:47:15.014 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-16T14:47:15.014 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-16T14:47:15.014 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000267671 s, 1.9 MB/s 2024-09-16T14:47:15.015 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T14:47:15.073 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2024-09-16T14:47:15.130 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T14:47:15.130 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:47:15.130 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T14:47:15.130 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:47:15.130 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:47:15.130 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-16 14:47:01.111239825 +0000 2024-09-16T14:47:15.130 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-16 14:44:40.700382313 +0000 2024-09-16T14:47:15.130 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-16 14:44:40.700382313 +0000 2024-09-16T14:47:15.131 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-16 14:44:40.700382313 +0000 2024-09-16T14:47:15.131 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T14:47:15.194 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-16T14:47:15.194 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-16T14:47:15.195 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000200496 s, 2.6 MB/s 2024-09-16T14:47:15.196 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T14:47:15.252 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2024-09-16T14:47:15.309 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T14:47:15.309 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:47:15.309 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-16T14:47:15.309 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:47:15.309 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:47:15.309 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-16 14:47:01.111239825 +0000 2024-09-16T14:47:15.309 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-16 14:44:41.023376693 +0000 2024-09-16T14:47:15.309 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-16 14:44:41.023376693 +0000 2024-09-16T14:47:15.309 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-16 14:44:41.023376693 +0000 2024-09-16T14:47:15.310 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T14:47:15.372 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-16T14:47:15.372 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-16T14:47:15.372 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000191371 s, 2.7 MB/s 2024-09-16T14:47:15.373 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T14:47:15.430 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2024-09-16T14:47:15.485 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T14:47:15.485 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:47:15.485 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-16T14:47:15.485 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:47:15.485 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:47:15.485 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-16 14:47:01.111239825 +0000 2024-09-16T14:47:15.485 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-16 14:44:41.343371124 +0000 2024-09-16T14:47:15.485 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-16 14:44:41.343371124 +0000 2024-09-16T14:47:15.485 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-16 14:44:41.343371124 +0000 2024-09-16T14:47:15.486 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T14:47:15.548 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-16T14:47:15.548 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-16T14:47:15.548 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000160488 s, 3.2 MB/s 2024-09-16T14:47:15.549 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T14:47:15.604 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:15.605 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:47:15.605 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T14:47:15.624 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T14:47:15.625 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 2024-09-16T14:47:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T14:47:15.682 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:47:15.682 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-16T14:47:15.682 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:47:15.682 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:47:15.683 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-16 14:47:05.943570137 +0000 2024-09-16T14:47:15.683 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-16 14:44:40.125106503 +0000 2024-09-16T14:47:15.683 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-16 14:44:40.125106503 +0000 2024-09-16T14:47:15.683 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-16 14:44:40.125106503 +0000 2024-09-16T14:47:15.683 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T14:47:15.749 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-16T14:47:15.749 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-16T14:47:15.749 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000202343 s, 2.5 MB/s 2024-09-16T14:47:15.751 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T14:47:15.810 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 2024-09-16T14:47:15.867 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T14:47:15.867 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:47:15.867 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T14:47:15.868 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:47:15.868 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:47:15.868 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-16 14:47:05.943570137 +0000 2024-09-16T14:47:15.868 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-16 14:44:40.453099509 +0000 2024-09-16T14:47:15.868 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-16 14:44:40.453099509 +0000 2024-09-16T14:47:15.868 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-16 14:44:40.453099509 +0000 2024-09-16T14:47:15.868 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T14:47:15.933 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-16T14:47:15.934 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-16T14:47:15.934 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000227731 s, 2.2 MB/s 2024-09-16T14:47:15.935 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T14:47:15.993 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 2024-09-16T14:47:16.050 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T14:47:16.050 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:47:16.050 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-16T14:47:16.050 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:47:16.051 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:47:16.051 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-16 14:47:05.944570115 +0000 2024-09-16T14:47:16.051 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-16 14:44:40.779092557 +0000 2024-09-16T14:47:16.051 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-16 14:44:40.779092557 +0000 2024-09-16T14:47:16.051 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-16 14:44:40.779092557 +0000 2024-09-16T14:47:16.051 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T14:47:16.115 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-16T14:47:16.115 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-16T14:47:16.116 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000235061 s, 2.2 MB/s 2024-09-16T14:47:16.117 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T14:47:16.177 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 2024-09-16T14:47:16.233 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T14:47:16.233 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T14:47:16.234 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-16T14:47:16.234 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T14:47:16.234 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T14:47:16.234 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-16 14:47:05.944570115 +0000 2024-09-16T14:47:16.234 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-16 14:44:41.114085413 +0000 2024-09-16T14:47:16.234 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-16 14:44:41.114085413 +0000 2024-09-16T14:47:16.234 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-16 14:44:41.114085413 +0000 2024-09-16T14:47:16.234 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T14:47:16.298 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-16T14:47:16.298 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-16T14:47:16.298 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.00019292 s, 2.7 MB/s 2024-09-16T14:47:16.299 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T14:47:16.356 INFO:tasks.ceph:osd dev map: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:16.356 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi111.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'}} 2024-09-16T14:47:16.356 INFO:tasks.ceph:Generating config... 2024-09-16T14:47:16.358 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-16T14:47:16.358 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-16T14:47:16.358 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-16T14:47:16.358 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T14:47:16.358 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T14:47:16.358 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-16T14:47:16.359 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-16T14:47:16.359 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-16T14:47:16.359 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T14:47:16.359 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T14:47:16.359 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-16T14:47:16.359 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T14:47:16.359 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T14:47:16.359 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T14:47:16.359 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-16T14:47:16.359 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.1 2024-09-16T14:47:16.359 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-16T14:47:16.359 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-16T14:47:16.359 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T14:47:16.359 INFO:tasks.ceph:[osd] osd max backfills = 6 2024-09-16T14:47:16.360 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-16T14:47:16.360 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-16T14:47:16.360 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-16T14:47:16.360 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-09-16T14:47:16.360 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T14:47:16.360 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-16T14:47:16.360 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-16T14:47:16.360 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-09-16T14:47:16.360 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-09-16T14:47:16.360 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-16T14:47:16.360 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-16T14:47:16.360 INFO:tasks.ceph:[global] ms inject socket failures = 2500 2024-09-16T14:47:16.360 INFO:tasks.ceph:[global] ms tcp read timeout = 5 2024-09-16T14:47:16.360 INFO:tasks.ceph:[global] osd_object_clean_region_max_num_intervals = 1000 2024-09-16T14:47:16.361 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T14:47:16.361 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T14:47:16.361 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T14:47:16.361 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T14:47:16.361 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T14:47:16.361 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-16T14:47:16.361 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T14:47:16.361 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T14:47:16.416 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T14:47:16.420 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T14:47:16.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T14:47:16.495 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.40'), ('mon.b', '172.21.15.111'), ('mon.c', '172.21.15.114')] 2024-09-16T14:47:16.496 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.40,172.21.15.111,172.21.15.114', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'osd_object_clean_region_max_num_intervals': 1000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-16T14:47:16.497 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:47:16.497 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T14:47:16.552 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.40 --add b 172.21.15.111 --add c 172.21.15.114 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:16.641 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:16.641 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: generated fsid aaaa1c0f-de8b-48c8-9412-815c8b70b7b8 2024-09-16T14:47:16.641 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = pacific 2024-09-16T14:47:16.641 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 0 2024-09-16T14:47:16.641 INFO:teuthology.orchestra.run.smithi040.stdout:fsid aaaa1c0f-de8b-48c8-9412-815c8b70b7b8 2024-09-16T14:47:16.641 INFO:teuthology.orchestra.run.smithi040.stdout:last_changed 2024-09-16T14:47:16.640290+0000 2024-09-16T14:47:16.641 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-09-16T14:47:16.640290+0000 2024-09-16T14:47:16.641 INFO:teuthology.orchestra.run.smithi040.stdout:min_mon_release 16 (pacific) 2024-09-16T14:47:16.641 INFO:teuthology.orchestra.run.smithi040.stdout:election_strategy: 1 2024-09-16T14:47:16.642 INFO:teuthology.orchestra.run.smithi040.stdout:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-09-16T14:47:16.642 INFO:teuthology.orchestra.run.smithi040.stdout:1: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-09-16T14:47:16.642 INFO:teuthology.orchestra.run.smithi040.stdout:2: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.c 2024-09-16T14:47:16.642 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-16T14:47:16.643 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T14:47:16.659 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID aaaa1c0f-de8b-48c8-9412-815c8b70b7b8... 2024-09-16T14:47:16.660 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T14:47:16.702 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T14:47:16.705 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T14:47:16.745 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2024-09-16T14:47:16.745 INFO:teuthology.orchestra.run.smithi040.stdout: chdir = "" 2024-09-16T14:47:16.745 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T14:47:16.745 INFO:teuthology.orchestra.run.smithi040.stdout: auth supported = cephx 2024-09-16T14:47:16.745 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.745 INFO:teuthology.orchestra.run.smithi040.stdout: filestore xattr use omap = true 2024-09-16T14:47:16.745 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.745 INFO:teuthology.orchestra.run.smithi040.stdout: mon clock drift allowed = 1.000 2024-09-16T14:47:16.745 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.746 INFO:teuthology.orchestra.run.smithi040.stdout: osd crush chooseleaf type = 0 2024-09-16T14:47:16.746 INFO:teuthology.orchestra.run.smithi040.stdout: auth debug = true 2024-09-16T14:47:16.746 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.746 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on old message = true 2024-09-16T14:47:16.746 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on bug = true 2024-09-16T14:47:16.746 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.746 INFO:teuthology.orchestra.run.smithi040.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T14:47:16.746 INFO:teuthology.orchestra.run.smithi040.stdout: mon pg warn max object skew = 0 2024-09-16T14:47:16.746 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.746 INFO:teuthology.orchestra.run.smithi040.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T14:47:16.746 INFO:teuthology.orchestra.run.smithi040.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T14:47:16.746 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.747 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default size = 2 2024-09-16T14:47:16.747 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.747 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow primary affinity = true 2024-09-16T14:47:16.747 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow pg remap = true 2024-09-16T14:47:16.747 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on legacy crush tunables = false 2024-09-16T14:47:16.747 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on crush straw calc version zero = false 2024-09-16T14:47:16.747 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on no sortbitwise = false 2024-09-16T14:47:16.747 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on osd down out interval zero = false 2024-09-16T14:47:16.747 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on too few osds = false 2024-09-16T14:47:16.747 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T14:47:16.747 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T14:47:16.748 INFO:teuthology.orchestra.run.smithi040.stdout: mon_allow_pool_size_one = true 2024-09-16T14:47:16.748 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.748 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T14:47:16.748 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.748 INFO:teuthology.orchestra.run.smithi040.stdout: osd default data pool replay window = 5 2024-09-16T14:47:16.748 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.748 INFO:teuthology.orchestra.run.smithi040.stdout: mon allow pool delete = true 2024-09-16T14:47:16.748 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.748 INFO:teuthology.orchestra.run.smithi040.stdout: mon cluster log file level = debug 2024-09-16T14:47:16.748 INFO:teuthology.orchestra.run.smithi040.stdout: debug asserts on shutdown = true 2024-09-16T14:47:16.748 INFO:teuthology.orchestra.run.smithi040.stdout: mon health detail to clog = false 2024-09-16T14:47:16.749 INFO:teuthology.orchestra.run.smithi040.stdout: mon host = "172.21.15.40,172.21.15.111,172.21.15.114" 2024-09-16T14:47:16.749 INFO:teuthology.orchestra.run.smithi040.stdout: mon client directed command retry = 5 2024-09-16T14:47:16.749 INFO:teuthology.orchestra.run.smithi040.stdout: mon election default strategy = 1 2024-09-16T14:47:16.749 INFO:teuthology.orchestra.run.smithi040.stdout: ms inject socket failures = 2500 2024-09-16T14:47:16.749 INFO:teuthology.orchestra.run.smithi040.stdout: ms tcp read timeout = 5 2024-09-16T14:47:16.749 INFO:teuthology.orchestra.run.smithi040.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-16T14:47:16.749 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = aaaa1c0f-de8b-48c8-9412-815c8b70b7b8 2024-09-16T14:47:16.749 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.749 INFO:teuthology.orchestra.run.smithi040.stdout:[osd] 2024-09-16T14:47:16.749 INFO:teuthology.orchestra.run.smithi040.stdout: osd journal size = 100 2024-09-16T14:47:16.749 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub load threshold = 5.0 2024-09-16T14:47:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub max interval = 120 2024-09-16T14:47:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock profile = high_recovery_ops 2024-09-16T14:47:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: osd recover clone overlap = true 2024-09-16T14:47:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: osd recovery max chunk = 1048576 2024-09-16T14:47:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug shutdown = true 2024-09-16T14:47:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug op order = true 2024-09-16T14:47:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify stray on activate = true 2024-09-16T14:47:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.750 INFO:teuthology.orchestra.run.smithi040.stdout: osd open classes on start = true 2024-09-16T14:47:16.751 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug pg log writeout = true 2024-09-16T14:47:16.751 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.751 INFO:teuthology.orchestra.run.smithi040.stdout: osd deep scrub update digest min age = 30 2024-09-16T14:47:16.751 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.751 INFO:teuthology.orchestra.run.smithi040.stdout: osd map max advance = 10 2024-09-16T14:47:16.751 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.751 INFO:teuthology.orchestra.run.smithi040.stdout: journal zero on create = true 2024-09-16T14:47:16.751 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.751 INFO:teuthology.orchestra.run.smithi040.stdout: filestore ondisk finisher threads = 3 2024-09-16T14:47:16.751 INFO:teuthology.orchestra.run.smithi040.stdout: filestore apply finisher threads = 3 2024-09-16T14:47:16.751 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.751 INFO:teuthology.orchestra.run.smithi040.stdout: bdev debug aio = true 2024-09-16T14:47:16.752 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug misdirected ops = true 2024-09-16T14:47:16.752 INFO:teuthology.orchestra.run.smithi040.stdout: debug monc = 20 2024-09-16T14:47:16.752 INFO:teuthology.orchestra.run.smithi040.stdout: bdev async discard = True 2024-09-16T14:47:16.752 INFO:teuthology.orchestra.run.smithi040.stdout: bdev enable discard = True 2024-09-16T14:47:16.752 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore block size = 96636764160 2024-09-16T14:47:16.752 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore fsck on mount = True 2024-09-16T14:47:16.752 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore zero block detection = True 2024-09-16T14:47:16.752 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 20 2024-09-16T14:47:16.752 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 20 2024-09-16T14:47:16.752 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-09-16T14:47:16.752 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 20 2024-09-16T14:47:16.752 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 10 2024-09-16T14:47:16.753 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T14:47:16.753 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.9 2024-09-16T14:47:16.753 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.8 2024-09-16T14:47:16.753 INFO:teuthology.orchestra.run.smithi040.stdout: osd blocked scrub grace period = 3600 2024-09-16T14:47:16.753 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug reject backfill probability = 0.1 2024-09-16T14:47:16.753 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify cached snaps = True 2024-09-16T14:47:16.753 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify missing on start = True 2024-09-16T14:47:16.753 INFO:teuthology.orchestra.run.smithi040.stdout: osd failsafe full ratio = 0.95 2024-09-16T14:47:16.753 INFO:teuthology.orchestra.run.smithi040.stdout: osd max backfills = 6 2024-09-16T14:47:16.753 INFO:teuthology.orchestra.run.smithi040.stdout: osd max markdown count = 1000 2024-09-16T14:47:16.753 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock override recovery settings = True 2024-09-16T14:47:16.754 INFO:teuthology.orchestra.run.smithi040.stdout: osd memory target = 1610612736 2024-09-16T14:47:16.754 INFO:teuthology.orchestra.run.smithi040.stdout: osd objectstore = bluestore 2024-09-16T14:47:16.754 INFO:teuthology.orchestra.run.smithi040.stdout: osd op queue = debug_random 2024-09-16T14:47:16.754 INFO:teuthology.orchestra.run.smithi040.stdout: osd op queue cut off = debug_random 2024-09-16T14:47:16.754 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub min interval = 60 2024-09-16T14:47:16.754 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.754 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr] 2024-09-16T14:47:16.754 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-09-16T14:47:16.754 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 20 2024-09-16T14:47:16.754 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-09-16T14:47:16.754 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-09-16T14:47:16.754 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-09-16T14:47:16.755 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-09-16T14:47:16.755 INFO:teuthology.orchestra.run.smithi040.stdout: mgr/telemetry/nag = false 2024-09-16T14:47:16.755 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.755 INFO:teuthology.orchestra.run.smithi040.stdout:[mon] 2024-09-16T14:47:16.755 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-09-16T14:47:16.755 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-09-16T14:47:16.755 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 20 2024-09-16T14:47:16.755 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-09-16T14:47:16.755 INFO:teuthology.orchestra.run.smithi040.stdout: mon data avail warn = 5 2024-09-16T14:47:16.755 INFO:teuthology.orchestra.run.smithi040.stdout: mon mgr mkfs grace = 240 2024-09-16T14:47:16.755 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-09-16T14:47:16.755 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd reporter subtree level = osd 2024-09-16T14:47:16.756 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd prime pg temp = true 2024-09-16T14:47:16.756 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-09-16T14:47:16.756 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.756 INFO:teuthology.orchestra.run.smithi040.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T14:47:16.756 INFO:teuthology.orchestra.run.smithi040.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T14:47:16.756 INFO:teuthology.orchestra.run.smithi040.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T14:47:16.756 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.756 INFO:teuthology.orchestra.run.smithi040.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T14:47:16.756 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T14:47:16.756 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T14:47:16.756 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.756 INFO:teuthology.orchestra.run.smithi040.stdout: # 1m isn't quite enough 2024-09-16T14:47:16.757 INFO:teuthology.orchestra.run.smithi040.stdout: mon_down_mkfs_grace = 2m 2024-09-16T14:47:16.757 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.757 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_filestore_osds = false 2024-09-16T14:47:16.757 INFO:teuthology.orchestra.run.smithi040.stdout: mon scrub interval = 300 2024-09-16T14:47:16.757 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:47:16.757 INFO:teuthology.orchestra.run.smithi040.stdout:[client] 2024-09-16T14:47:16.757 INFO:teuthology.orchestra.run.smithi040.stdout: rgw cache enabled = true 2024-09-16T14:47:16.757 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable ops log = true 2024-09-16T14:47:16.757 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable usage log = true 2024-09-16T14:47:16.757 INFO:teuthology.orchestra.run.smithi040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T14:47:16.757 INFO:teuthology.orchestra.run.smithi040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T14:47:16.757 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.a] 2024-09-16T14:47:16.758 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.b] 2024-09-16T14:47:16.758 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.c] 2024-09-16T14:47:16.758 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2024-09-16T14:47:16.759 INFO:teuthology.orchestra.run.smithi114.stdout: chdir = "" 2024-09-16T14:47:16.759 INFO:teuthology.orchestra.run.smithi114.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T14:47:16.759 INFO:teuthology.orchestra.run.smithi114.stdout: auth supported = cephx 2024-09-16T14:47:16.759 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.759 INFO:teuthology.orchestra.run.smithi114.stdout: filestore xattr use omap = true 2024-09-16T14:47:16.759 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.759 INFO:teuthology.orchestra.run.smithi114.stdout: mon clock drift allowed = 1.000 2024-09-16T14:47:16.759 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.759 INFO:teuthology.orchestra.run.smithi114.stdout: osd crush chooseleaf type = 0 2024-09-16T14:47:16.759 INFO:teuthology.orchestra.run.smithi114.stdout: auth debug = true 2024-09-16T14:47:16.759 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.760 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on old message = true 2024-09-16T14:47:16.760 INFO:teuthology.orchestra.run.smithi114.stdout: ms die on bug = true 2024-09-16T14:47:16.760 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.760 INFO:teuthology.orchestra.run.smithi114.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T14:47:16.760 INFO:teuthology.orchestra.run.smithi114.stdout: mon pg warn max object skew = 0 2024-09-16T14:47:16.760 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.760 INFO:teuthology.orchestra.run.smithi114.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T14:47:16.760 INFO:teuthology.orchestra.run.smithi114.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T14:47:16.760 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.760 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default size = 2 2024-09-16T14:47:16.760 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.761 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow primary affinity = true 2024-09-16T14:47:16.761 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd allow pg remap = true 2024-09-16T14:47:16.761 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on legacy crush tunables = false 2024-09-16T14:47:16.761 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on crush straw calc version zero = false 2024-09-16T14:47:16.761 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on no sortbitwise = false 2024-09-16T14:47:16.761 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on osd down out interval zero = false 2024-09-16T14:47:16.761 INFO:teuthology.orchestra.run.smithi114.stdout: mon warn on too few osds = false 2024-09-16T14:47:16.761 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T14:47:16.761 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T14:47:16.761 INFO:teuthology.orchestra.run.smithi114.stdout: mon_allow_pool_size_one = true 2024-09-16T14:47:16.761 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.761 INFO:teuthology.orchestra.run.smithi114.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T14:47:16.762 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.762 INFO:teuthology.orchestra.run.smithi114.stdout: osd default data pool replay window = 5 2024-09-16T14:47:16.762 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.762 INFO:teuthology.orchestra.run.smithi114.stdout: mon allow pool delete = true 2024-09-16T14:47:16.762 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.762 INFO:teuthology.orchestra.run.smithi114.stdout: mon cluster log file level = debug 2024-09-16T14:47:16.762 INFO:teuthology.orchestra.run.smithi114.stdout: debug asserts on shutdown = true 2024-09-16T14:47:16.762 INFO:teuthology.orchestra.run.smithi114.stdout: mon health detail to clog = false 2024-09-16T14:47:16.762 INFO:teuthology.orchestra.run.smithi114.stdout: mon host = "172.21.15.40,172.21.15.111,172.21.15.114" 2024-09-16T14:47:16.762 INFO:teuthology.orchestra.run.smithi114.stdout: mon client directed command retry = 5 2024-09-16T14:47:16.762 INFO:teuthology.orchestra.run.smithi114.stdout: mon election default strategy = 1 2024-09-16T14:47:16.762 INFO:teuthology.orchestra.run.smithi114.stdout: ms inject socket failures = 2500 2024-09-16T14:47:16.763 INFO:teuthology.orchestra.run.smithi114.stdout: ms tcp read timeout = 5 2024-09-16T14:47:16.763 INFO:teuthology.orchestra.run.smithi114.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-16T14:47:16.763 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = aaaa1c0f-de8b-48c8-9412-815c8b70b7b8 2024-09-16T14:47:16.763 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.763 INFO:teuthology.orchestra.run.smithi114.stdout:[osd] 2024-09-16T14:47:16.763 INFO:teuthology.orchestra.run.smithi114.stdout: osd journal size = 100 2024-09-16T14:47:16.763 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.763 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub load threshold = 5.0 2024-09-16T14:47:16.763 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub max interval = 120 2024-09-16T14:47:16.763 INFO:teuthology.orchestra.run.smithi114.stdout: osd mclock profile = high_recovery_ops 2024-09-16T14:47:16.763 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.763 INFO:teuthology.orchestra.run.smithi114.stdout: osd recover clone overlap = true 2024-09-16T14:47:16.764 INFO:teuthology.orchestra.run.smithi114.stdout: osd recovery max chunk = 1048576 2024-09-16T14:47:16.764 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.764 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug shutdown = true 2024-09-16T14:47:16.764 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug op order = true 2024-09-16T14:47:16.764 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify stray on activate = true 2024-09-16T14:47:16.764 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.764 INFO:teuthology.orchestra.run.smithi114.stdout: osd open classes on start = true 2024-09-16T14:47:16.764 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug pg log writeout = true 2024-09-16T14:47:16.764 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.764 INFO:teuthology.orchestra.run.smithi114.stdout: osd deep scrub update digest min age = 30 2024-09-16T14:47:16.764 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.765 INFO:teuthology.orchestra.run.smithi114.stdout: osd map max advance = 10 2024-09-16T14:47:16.765 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.765 INFO:teuthology.orchestra.run.smithi114.stdout: journal zero on create = true 2024-09-16T14:47:16.765 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.765 INFO:teuthology.orchestra.run.smithi114.stdout: filestore ondisk finisher threads = 3 2024-09-16T14:47:16.765 INFO:teuthology.orchestra.run.smithi114.stdout: filestore apply finisher threads = 3 2024-09-16T14:47:16.765 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.765 INFO:teuthology.orchestra.run.smithi114.stdout: bdev debug aio = true 2024-09-16T14:47:16.765 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug misdirected ops = true 2024-09-16T14:47:16.765 INFO:teuthology.orchestra.run.smithi114.stdout: debug monc = 20 2024-09-16T14:47:16.765 INFO:teuthology.orchestra.run.smithi114.stdout: bdev async discard = True 2024-09-16T14:47:16.766 INFO:teuthology.orchestra.run.smithi114.stdout: bdev enable discard = True 2024-09-16T14:47:16.766 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore block size = 96636764160 2024-09-16T14:47:16.766 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore fsck on mount = True 2024-09-16T14:47:16.766 INFO:teuthology.orchestra.run.smithi114.stdout: bluestore zero block detection = True 2024-09-16T14:47:16.766 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluefs = 20 2024-09-16T14:47:16.766 INFO:teuthology.orchestra.run.smithi114.stdout: debug bluestore = 20 2024-09-16T14:47:16.766 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-09-16T14:47:16.766 INFO:teuthology.orchestra.run.smithi114.stdout: debug osd = 20 2024-09-16T14:47:16.767 INFO:teuthology.orchestra.run.smithi114.stdout: debug rocksdb = 10 2024-09-16T14:47:16.767 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T14:47:16.767 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd full ratio = 0.9 2024-09-16T14:47:16.767 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd nearfull ratio = 0.8 2024-09-16T14:47:16.767 INFO:teuthology.orchestra.run.smithi114.stdout: osd blocked scrub grace period = 3600 2024-09-16T14:47:16.767 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug reject backfill probability = 0.1 2024-09-16T14:47:16.767 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify cached snaps = True 2024-09-16T14:47:16.767 INFO:teuthology.orchestra.run.smithi114.stdout: osd debug verify missing on start = True 2024-09-16T14:47:16.767 INFO:teuthology.orchestra.run.smithi114.stdout: osd failsafe full ratio = 0.95 2024-09-16T14:47:16.767 INFO:teuthology.orchestra.run.smithi114.stdout: osd max backfills = 6 2024-09-16T14:47:16.767 INFO:teuthology.orchestra.run.smithi114.stdout: osd max markdown count = 1000 2024-09-16T14:47:16.768 INFO:teuthology.orchestra.run.smithi114.stdout: osd mclock override recovery settings = True 2024-09-16T14:47:16.768 INFO:teuthology.orchestra.run.smithi114.stdout: osd memory target = 1610612736 2024-09-16T14:47:16.768 INFO:teuthology.orchestra.run.smithi114.stdout: osd objectstore = bluestore 2024-09-16T14:47:16.768 INFO:teuthology.orchestra.run.smithi114.stdout: osd op queue = debug_random 2024-09-16T14:47:16.768 INFO:teuthology.orchestra.run.smithi114.stdout: osd op queue cut off = debug_random 2024-09-16T14:47:16.768 INFO:teuthology.orchestra.run.smithi114.stdout: osd scrub min interval = 60 2024-09-16T14:47:16.768 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.768 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr] 2024-09-16T14:47:16.768 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-09-16T14:47:16.768 INFO:teuthology.orchestra.run.smithi114.stdout: debug mgr = 20 2024-09-16T14:47:16.768 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-09-16T14:47:16.769 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-09-16T14:47:16.769 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-09-16T14:47:16.769 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-09-16T14:47:16.769 INFO:teuthology.orchestra.run.smithi114.stdout: mgr/telemetry/nag = false 2024-09-16T14:47:16.769 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.769 INFO:teuthology.orchestra.run.smithi114.stdout:[mon] 2024-09-16T14:47:16.769 INFO:teuthology.orchestra.run.smithi114.stdout: debug ms = 1 2024-09-16T14:47:16.769 INFO:teuthology.orchestra.run.smithi114.stdout: debug mon = 20 2024-09-16T14:47:16.769 INFO:teuthology.orchestra.run.smithi114.stdout: debug paxos = 20 2024-09-16T14:47:16.769 INFO:teuthology.orchestra.run.smithi114.stdout: debug auth = 20 2024-09-16T14:47:16.769 INFO:teuthology.orchestra.run.smithi114.stdout: mon data avail warn = 5 2024-09-16T14:47:16.769 INFO:teuthology.orchestra.run.smithi114.stdout: mon mgr mkfs grace = 240 2024-09-16T14:47:16.770 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min pgs per osd = 4 2024-09-16T14:47:16.770 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd reporter subtree level = osd 2024-09-16T14:47:16.770 INFO:teuthology.orchestra.run.smithi114.stdout: mon osd prime pg temp = true 2024-09-16T14:47:16.770 INFO:teuthology.orchestra.run.smithi114.stdout: mon reweight min bytes per osd = 10 2024-09-16T14:47:16.770 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.770 INFO:teuthology.orchestra.run.smithi114.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T14:47:16.770 INFO:teuthology.orchestra.run.smithi114.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T14:47:16.770 INFO:teuthology.orchestra.run.smithi114.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T14:47:16.770 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.770 INFO:teuthology.orchestra.run.smithi114.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T14:47:16.770 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T14:47:16.770 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T14:47:16.771 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.771 INFO:teuthology.orchestra.run.smithi114.stdout: # 1m isn't quite enough 2024-09-16T14:47:16.771 INFO:teuthology.orchestra.run.smithi114.stdout: mon_down_mkfs_grace = 2m 2024-09-16T14:47:16.771 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.771 INFO:teuthology.orchestra.run.smithi114.stdout: mon_warn_on_filestore_osds = false 2024-09-16T14:47:16.771 INFO:teuthology.orchestra.run.smithi114.stdout: mon scrub interval = 300 2024-09-16T14:47:16.771 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-16T14:47:16.771 INFO:teuthology.orchestra.run.smithi114.stdout:[client] 2024-09-16T14:47:16.771 INFO:teuthology.orchestra.run.smithi114.stdout: rgw cache enabled = true 2024-09-16T14:47:16.771 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable ops log = true 2024-09-16T14:47:16.771 INFO:teuthology.orchestra.run.smithi114.stdout: rgw enable usage log = true 2024-09-16T14:47:16.772 INFO:teuthology.orchestra.run.smithi114.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T14:47:16.772 INFO:teuthology.orchestra.run.smithi114.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T14:47:16.772 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.a] 2024-09-16T14:47:16.772 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.b] 2024-09-16T14:47:16.772 INFO:teuthology.orchestra.run.smithi114.stdout:[mon.c] 2024-09-16T14:47:16.772 INFO:teuthology.orchestra.run.smithi111.stdout:[global] 2024-09-16T14:47:16.772 INFO:teuthology.orchestra.run.smithi111.stdout: chdir = "" 2024-09-16T14:47:16.772 INFO:teuthology.orchestra.run.smithi111.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T14:47:16.772 INFO:teuthology.orchestra.run.smithi111.stdout: auth supported = cephx 2024-09-16T14:47:16.773 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.773 INFO:teuthology.orchestra.run.smithi111.stdout: filestore xattr use omap = true 2024-09-16T14:47:16.773 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.773 INFO:teuthology.orchestra.run.smithi111.stdout: mon clock drift allowed = 1.000 2024-09-16T14:47:16.773 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.773 INFO:teuthology.orchestra.run.smithi111.stdout: osd crush chooseleaf type = 0 2024-09-16T14:47:16.773 INFO:teuthology.orchestra.run.smithi111.stdout: auth debug = true 2024-09-16T14:47:16.773 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.773 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on old message = true 2024-09-16T14:47:16.773 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on bug = true 2024-09-16T14:47:16.773 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.773 INFO:teuthology.orchestra.run.smithi111.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T14:47:16.774 INFO:teuthology.orchestra.run.smithi111.stdout: mon pg warn max object skew = 0 2024-09-16T14:47:16.774 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.774 INFO:teuthology.orchestra.run.smithi111.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T14:47:16.774 INFO:teuthology.orchestra.run.smithi111.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T14:47:16.774 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.774 INFO:teuthology.orchestra.run.smithi111.stdout: osd pool default size = 2 2024-09-16T14:47:16.774 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.774 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd allow primary affinity = true 2024-09-16T14:47:16.774 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd allow pg remap = true 2024-09-16T14:47:16.774 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on legacy crush tunables = false 2024-09-16T14:47:16.774 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on crush straw calc version zero = false 2024-09-16T14:47:16.774 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on no sortbitwise = false 2024-09-16T14:47:16.775 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on osd down out interval zero = false 2024-09-16T14:47:16.775 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on too few osds = false 2024-09-16T14:47:16.775 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T14:47:16.775 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T14:47:16.775 INFO:teuthology.orchestra.run.smithi111.stdout: mon_allow_pool_size_one = true 2024-09-16T14:47:16.775 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.775 INFO:teuthology.orchestra.run.smithi111.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T14:47:16.775 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.775 INFO:teuthology.orchestra.run.smithi111.stdout: osd default data pool replay window = 5 2024-09-16T14:47:16.775 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.775 INFO:teuthology.orchestra.run.smithi111.stdout: mon allow pool delete = true 2024-09-16T14:47:16.775 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.776 INFO:teuthology.orchestra.run.smithi111.stdout: mon cluster log file level = debug 2024-09-16T14:47:16.776 INFO:teuthology.orchestra.run.smithi111.stdout: debug asserts on shutdown = true 2024-09-16T14:47:16.776 INFO:teuthology.orchestra.run.smithi111.stdout: mon health detail to clog = false 2024-09-16T14:47:16.776 INFO:teuthology.orchestra.run.smithi111.stdout: mon host = "172.21.15.40,172.21.15.111,172.21.15.114" 2024-09-16T14:47:16.776 INFO:teuthology.orchestra.run.smithi111.stdout: mon client directed command retry = 5 2024-09-16T14:47:16.776 INFO:teuthology.orchestra.run.smithi111.stdout: mon election default strategy = 1 2024-09-16T14:47:16.776 INFO:teuthology.orchestra.run.smithi111.stdout: ms inject socket failures = 2500 2024-09-16T14:47:16.776 INFO:teuthology.orchestra.run.smithi111.stdout: ms tcp read timeout = 5 2024-09-16T14:47:16.776 INFO:teuthology.orchestra.run.smithi111.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-09-16T14:47:16.776 INFO:teuthology.orchestra.run.smithi111.stdout: fsid = aaaa1c0f-de8b-48c8-9412-815c8b70b7b8 2024-09-16T14:47:16.776 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.776 INFO:teuthology.orchestra.run.smithi111.stdout:[osd] 2024-09-16T14:47:16.777 INFO:teuthology.orchestra.run.smithi111.stdout: osd journal size = 100 2024-09-16T14:47:16.777 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.777 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub load threshold = 5.0 2024-09-16T14:47:16.777 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub max interval = 120 2024-09-16T14:47:16.777 INFO:teuthology.orchestra.run.smithi111.stdout: osd mclock profile = high_recovery_ops 2024-09-16T14:47:16.777 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.777 INFO:teuthology.orchestra.run.smithi111.stdout: osd recover clone overlap = true 2024-09-16T14:47:16.777 INFO:teuthology.orchestra.run.smithi111.stdout: osd recovery max chunk = 1048576 2024-09-16T14:47:16.777 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.777 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug shutdown = true 2024-09-16T14:47:16.777 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug op order = true 2024-09-16T14:47:16.777 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug verify stray on activate = true 2024-09-16T14:47:16.778 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.778 INFO:teuthology.orchestra.run.smithi111.stdout: osd open classes on start = true 2024-09-16T14:47:16.778 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug pg log writeout = true 2024-09-16T14:47:16.778 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.778 INFO:teuthology.orchestra.run.smithi111.stdout: osd deep scrub update digest min age = 30 2024-09-16T14:47:16.778 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.778 INFO:teuthology.orchestra.run.smithi111.stdout: osd map max advance = 10 2024-09-16T14:47:16.778 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.778 INFO:teuthology.orchestra.run.smithi111.stdout: journal zero on create = true 2024-09-16T14:47:16.778 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.778 INFO:teuthology.orchestra.run.smithi111.stdout: filestore ondisk finisher threads = 3 2024-09-16T14:47:16.778 INFO:teuthology.orchestra.run.smithi111.stdout: filestore apply finisher threads = 3 2024-09-16T14:47:16.779 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.779 INFO:teuthology.orchestra.run.smithi111.stdout: bdev debug aio = true 2024-09-16T14:47:16.779 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug misdirected ops = true 2024-09-16T14:47:16.779 INFO:teuthology.orchestra.run.smithi111.stdout: debug monc = 20 2024-09-16T14:47:16.779 INFO:teuthology.orchestra.run.smithi111.stdout: bdev async discard = True 2024-09-16T14:47:16.779 INFO:teuthology.orchestra.run.smithi111.stdout: bdev enable discard = True 2024-09-16T14:47:16.779 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore block size = 96636764160 2024-09-16T14:47:16.779 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore fsck on mount = True 2024-09-16T14:47:16.779 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore zero block detection = True 2024-09-16T14:47:16.779 INFO:teuthology.orchestra.run.smithi111.stdout: debug bluefs = 20 2024-09-16T14:47:16.779 INFO:teuthology.orchestra.run.smithi111.stdout: debug bluestore = 20 2024-09-16T14:47:16.780 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-09-16T14:47:16.780 INFO:teuthology.orchestra.run.smithi111.stdout: debug osd = 20 2024-09-16T14:47:16.780 INFO:teuthology.orchestra.run.smithi111.stdout: debug rocksdb = 10 2024-09-16T14:47:16.780 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T14:47:16.780 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd full ratio = 0.9 2024-09-16T14:47:16.780 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd nearfull ratio = 0.8 2024-09-16T14:47:16.780 INFO:teuthology.orchestra.run.smithi111.stdout: osd blocked scrub grace period = 3600 2024-09-16T14:47:16.780 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug reject backfill probability = 0.1 2024-09-16T14:47:16.780 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug verify cached snaps = True 2024-09-16T14:47:16.780 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug verify missing on start = True 2024-09-16T14:47:16.780 INFO:teuthology.orchestra.run.smithi111.stdout: osd failsafe full ratio = 0.95 2024-09-16T14:47:16.780 INFO:teuthology.orchestra.run.smithi111.stdout: osd max backfills = 6 2024-09-16T14:47:16.781 INFO:teuthology.orchestra.run.smithi111.stdout: osd max markdown count = 1000 2024-09-16T14:47:16.781 INFO:teuthology.orchestra.run.smithi111.stdout: osd mclock override recovery settings = True 2024-09-16T14:47:16.781 INFO:teuthology.orchestra.run.smithi111.stdout: osd memory target = 1610612736 2024-09-16T14:47:16.781 INFO:teuthology.orchestra.run.smithi111.stdout: osd objectstore = bluestore 2024-09-16T14:47:16.781 INFO:teuthology.orchestra.run.smithi111.stdout: osd op queue = debug_random 2024-09-16T14:47:16.781 INFO:teuthology.orchestra.run.smithi111.stdout: osd op queue cut off = debug_random 2024-09-16T14:47:16.781 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub min interval = 60 2024-09-16T14:47:16.781 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.781 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr] 2024-09-16T14:47:16.781 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-09-16T14:47:16.781 INFO:teuthology.orchestra.run.smithi111.stdout: debug mgr = 20 2024-09-16T14:47:16.782 INFO:teuthology.orchestra.run.smithi111.stdout: debug mon = 20 2024-09-16T14:47:16.782 INFO:teuthology.orchestra.run.smithi111.stdout: debug auth = 20 2024-09-16T14:47:16.782 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min pgs per osd = 4 2024-09-16T14:47:16.782 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min bytes per osd = 10 2024-09-16T14:47:16.782 INFO:teuthology.orchestra.run.smithi111.stdout: mgr/telemetry/nag = false 2024-09-16T14:47:16.782 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.782 INFO:teuthology.orchestra.run.smithi111.stdout:[mon] 2024-09-16T14:47:16.782 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2024-09-16T14:47:16.782 INFO:teuthology.orchestra.run.smithi111.stdout: debug mon = 20 2024-09-16T14:47:16.782 INFO:teuthology.orchestra.run.smithi111.stdout: debug paxos = 20 2024-09-16T14:47:16.782 INFO:teuthology.orchestra.run.smithi111.stdout: debug auth = 20 2024-09-16T14:47:16.783 INFO:teuthology.orchestra.run.smithi111.stdout: mon data avail warn = 5 2024-09-16T14:47:16.783 INFO:teuthology.orchestra.run.smithi111.stdout: mon mgr mkfs grace = 240 2024-09-16T14:47:16.783 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min pgs per osd = 4 2024-09-16T14:47:16.783 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd reporter subtree level = osd 2024-09-16T14:47:16.783 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd prime pg temp = true 2024-09-16T14:47:16.783 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min bytes per osd = 10 2024-09-16T14:47:16.783 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.783 INFO:teuthology.orchestra.run.smithi111.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T14:47:16.783 INFO:teuthology.orchestra.run.smithi111.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T14:47:16.783 INFO:teuthology.orchestra.run.smithi111.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T14:47:16.783 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.783 INFO:teuthology.orchestra.run.smithi111.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T14:47:16.783 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T14:47:16.784 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T14:47:16.784 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.784 INFO:teuthology.orchestra.run.smithi111.stdout: # 1m isn't quite enough 2024-09-16T14:47:16.784 INFO:teuthology.orchestra.run.smithi111.stdout: mon_down_mkfs_grace = 2m 2024-09-16T14:47:16.784 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.784 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_filestore_osds = false 2024-09-16T14:47:16.784 INFO:teuthology.orchestra.run.smithi111.stdout: mon scrub interval = 300 2024-09-16T14:47:16.784 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-16T14:47:16.784 INFO:teuthology.orchestra.run.smithi111.stdout:[client] 2024-09-16T14:47:16.784 INFO:teuthology.orchestra.run.smithi111.stdout: rgw cache enabled = true 2024-09-16T14:47:16.784 INFO:teuthology.orchestra.run.smithi111.stdout: rgw enable ops log = true 2024-09-16T14:47:16.784 INFO:teuthology.orchestra.run.smithi111.stdout: rgw enable usage log = true 2024-09-16T14:47:16.784 INFO:teuthology.orchestra.run.smithi111.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T14:47:16.785 INFO:teuthology.orchestra.run.smithi111.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T14:47:16.785 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.a] 2024-09-16T14:47:16.785 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.b] 2024-09-16T14:47:16.785 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.c] 2024-09-16T14:47:16.787 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T14:47:16.787 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T14:47:16.882 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T14:47:16.883 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:47:16.883 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T14:47:16.899 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:47:16.899 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T14:47:16.954 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi040.front.sepia.ceph.com 2024-09-16T14:47:16.955 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:47:16.955 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T14:47:16.955 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T14:47:17.031 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:47:17.031 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:17.087 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi111.front.sepia.ceph.com 2024-09-16T14:47:17.087 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:47:17.087 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T14:47:17.087 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T14:47:17.127 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:47:17.127 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:17.186 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi114.front.sepia.ceph.com 2024-09-16T14:47:17.186 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:47:17.186 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T14:47:17.186 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T14:47:17.227 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:47:17.227 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:17.284 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T14:47:17.285 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T14:47:17.285 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T14:47:17.350 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T14:47:17.353 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T14:47:17.353 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T14:47:17.353 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T14:47:17.397 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T14:47:17.410 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T14:47:17.410 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi111.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi114.front.sepia.ceph.com'): {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'}} 2024-09-16T14:47:17.410 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T14:47:17.476 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:17.476 INFO:tasks.ceph:role: osd.0 2024-09-16T14:47:17.476 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-16T14:47:17.476 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T14:47:17.549 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:47:17.550 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:47:17.550 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:47:17.550 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T14:47:17.550 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:47:17.550 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:47:17.550 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:47:17.550 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T14:47:17.550 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:47:17.550 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:47:18.053 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-09-16T14:47:18.055 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-09-16T14:47:18.055 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T14:47:18.151 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T14:47:18.182 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T14:47:18.250 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:18.250 INFO:tasks.ceph:role: osd.1 2024-09-16T14:47:18.250 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-16T14:47:18.250 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T14:47:18.323 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:47:18.323 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:47:18.323 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:47:18.323 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T14:47:18.323 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:47:18.323 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:47:18.323 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:47:18.323 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T14:47:18.324 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:47:18.324 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:47:18.791 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-09-16T14:47:18.793 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-09-16T14:47:18.793 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T14:47:18.886 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T14:47:18.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T14:47:18.993 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:18.993 INFO:tasks.ceph:role: osd.2 2024-09-16T14:47:18.993 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-16T14:47:18.993 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T14:47:19.086 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:47:19.086 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:47:19.086 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:47:19.086 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T14:47:19.086 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:47:19.086 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:47:19.086 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:47:19.087 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T14:47:19.087 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:47:19.087 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:47:19.601 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-09-16T14:47:19.605 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-09-16T14:47:19.605 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T14:47:19.728 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T14:47:19.765 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-16T14:47:19.831 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:19.832 INFO:tasks.ceph:role: osd.3 2024-09-16T14:47:19.832 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-16T14:47:19.832 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T14:47:19.905 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:47:19.906 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:47:19.906 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:47:19.906 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T14:47:19.906 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:47:19.906 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:47:19.906 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:47:19.906 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T14:47:19.907 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:47:19.907 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:47:20.383 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2024-09-16T14:47:20.385 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-09-16T14:47:20.386 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-16T14:47:20.501 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-16T14:47:20.534 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:20.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:20.632+0000 7fe6776e2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T14:47:20.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:20.632+0000 7fe6776e2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T14:47:20.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:20.632+0000 7fe6776e2640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T14:47:20.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:20.632+0000 7fe6776e2640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T14:47:20.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:20.632+0000 7fe6776e2640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T14:47:20.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:20.633+0000 7fe6776e2640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T14:47:23.488 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T14:47:23.518 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:23.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:23.609+0000 7fbdce6ed640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T14:47:23.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:23.610+0000 7fbdce6ed640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T14:47:23.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:23.610+0000 7fbdce6ed640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T14:47:23.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:23.610+0000 7fbdce6ed640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T14:47:23.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:23.610+0000 7fbdce6ed640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T14:47:23.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:23.610+0000 7fbdce6ed640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T14:47:26.463 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T14:47:26.493 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:26.587+0000 7f4ced7bd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T14:47:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:26.588+0000 7f4ced7bd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T14:47:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:26.588+0000 7f4ced7bd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T14:47:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:26.588+0000 7f4ced7bd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T14:47:26.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:26.588+0000 7f4ced7bd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T14:47:26.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:26.588+0000 7f4ced7bd640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T14:47:29.443 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T14:47:29.474 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:29.564+0000 7f541e02f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-16T14:47:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:29.565+0000 7f541e02f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-16T14:47:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:29.565+0000 7f541e02f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T14:47:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:29.565+0000 7f541e02f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T14:47:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:29.565+0000 7f541e02f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-16T14:47:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:47:29.565+0000 7f541e02f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-16T14:47:32.419 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T14:47:32.450 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-16T14:47:32.482 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:32.482 INFO:tasks.ceph:role: osd.4 2024-09-16T14:47:32.482 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-16T14:47:32.482 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T14:47:32.558 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:47:32.559 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:47:32.559 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:47:32.559 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T14:47:32.559 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:47:32.559 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:47:32.559 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:47:32.559 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T14:47:32.559 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:47:32.559 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:47:33.051 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-09-16T14:47:33.053 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-09-16T14:47:33.053 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-16T14:47:33.187 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-16T14:47:33.221 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-16T14:47:33.288 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:33.288 INFO:tasks.ceph:role: osd.5 2024-09-16T14:47:33.288 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-16T14:47:33.288 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T14:47:33.362 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:47:33.362 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:47:33.362 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:47:33.362 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T14:47:33.362 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:47:33.362 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:47:33.362 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:47:33.362 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T14:47:33.362 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:47:33.363 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:47:33.813 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-09-16T14:47:33.814 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-09-16T14:47:33.815 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-16T14:47:33.903 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-16T14:47:33.932 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-16T14:47:33.996 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:33.997 INFO:tasks.ceph:role: osd.6 2024-09-16T14:47:33.997 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-16T14:47:33.997 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T14:47:34.069 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:47:34.069 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:47:34.069 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:47:34.069 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T14:47:34.069 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:47:34.070 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:47:34.070 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:47:34.070 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T14:47:34.070 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:47:34.070 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:47:34.552 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-09-16T14:47:34.554 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-09-16T14:47:34.554 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-16T14:47:34.665 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-16T14:47:34.696 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-16T14:47:34.761 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:34.761 INFO:tasks.ceph:role: osd.7 2024-09-16T14:47:34.761 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-16T14:47:34.761 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T14:47:34.833 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:47:34.833 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:47:34.833 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:47:34.833 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T14:47:34.833 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:47:34.833 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:47:34.833 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:47:34.834 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T14:47:34.834 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:47:34.834 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:47:35.345 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2024-09-16T14:47:35.347 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2024-09-16T14:47:35.347 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-16T14:47:35.471 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-16T14:47:35.505 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:35.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:35.602+0000 7f7a95332640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-16T14:47:35.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:35.603+0000 7f7a95332640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-16T14:47:35.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:35.603+0000 7f7a95332640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T14:47:35.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:35.603+0000 7f7a95332640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T14:47:35.605 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:35.603+0000 7f7a95332640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-16T14:47:35.606 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:35.603+0000 7f7a95332640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-16T14:47:38.451 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T14:47:38.481 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:38.571 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:38.569+0000 7f1fb140e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-16T14:47:38.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:38.569+0000 7f1fb140e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-16T14:47:38.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:38.569+0000 7f1fb140e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T14:47:38.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:38.569+0000 7f1fb140e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T14:47:38.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:38.569+0000 7f1fb140e640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-16T14:47:38.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:38.570+0000 7f1fb140e640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-16T14:47:41.423 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T14:47:41.453 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:41.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:41.544+0000 7f5e02a13640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-16T14:47:41.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:41.544+0000 7f5e02a13640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-16T14:47:41.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:41.544+0000 7f5e02a13640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T14:47:41.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:41.544+0000 7f5e02a13640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T14:47:41.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:41.544+0000 7f5e02a13640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-16T14:47:41.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:41.544+0000 7f5e02a13640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-16T14:47:44.394 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-16T14:47:44.423 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:44.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:44.513+0000 7fda36d32640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-16T14:47:44.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:44.513+0000 7fda36d32640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-16T14:47:44.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:44.513+0000 7fda36d32640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T14:47:44.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:44.513+0000 7fda36d32640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T14:47:44.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:44.513+0000 7fda36d32640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-16T14:47:44.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-16T14:47:44.514+0000 7fda36d32640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-16T14:47:47.362 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-16T14:47:47.394 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2024-09-16T14:47:47.424 INFO:tasks.ceph:roles_to_devs: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:47.425 INFO:tasks.ceph:role: osd.8 2024-09-16T14:47:47.425 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-16T14:47:47.425 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T14:47:47.500 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:47:47.501 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:47:47.501 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:47:47.501 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T14:47:47.501 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:47:47.501 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:47:47.501 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:47:47.501 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T14:47:47.501 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:47:47.501 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:47:48.006 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-16T14:47:48.009 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-16T14:47:48.009 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-8 2024-09-16T14:47:48.124 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2024-09-16T14:47:48.156 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2024-09-16T14:47:48.221 INFO:tasks.ceph:roles_to_devs: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:48.221 INFO:tasks.ceph:role: osd.9 2024-09-16T14:47:48.221 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-16T14:47:48.221 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T14:47:48.295 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:47:48.295 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:47:48.295 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:47:48.295 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T14:47:48.295 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:47:48.295 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:47:48.296 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:47:48.296 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T14:47:48.296 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:47:48.296 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:47:48.755 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-16T14:47:48.873 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-16T14:47:48.873 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-9 2024-09-16T14:47:48.955 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2024-09-16T14:47:49.026 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2024-09-16T14:47:49.093 INFO:tasks.ceph:roles_to_devs: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:49.094 INFO:tasks.ceph:role: osd.10 2024-09-16T14:47:49.094 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-16T14:47:49.094 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T14:47:49.165 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:47:49.165 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:47:49.165 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:47:49.165 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T14:47:49.165 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:47:49.165 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:47:49.166 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:47:49.166 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T14:47:49.166 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:47:49.166 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:47:49.686 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-16T14:47:49.688 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-16T14:47:49.689 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2024-09-16T14:47:49.817 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2024-09-16T14:47:49.847 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2024-09-16T14:47:49.912 INFO:tasks.ceph:roles_to_devs: {'osd.8': '/dev/vg_nvme/lv_1', 'osd.9': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.11': '/dev/vg_nvme/lv_4'} 2024-09-16T14:47:49.912 INFO:tasks.ceph:role: osd.11 2024-09-16T14:47:49.913 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-16T14:47:49.913 DEBUG:teuthology.orchestra.run.smithi114:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T14:47:49.984 INFO:teuthology.orchestra.run.smithi114.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T14:47:49.984 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T14:47:49.984 INFO:teuthology.orchestra.run.smithi114.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T14:47:49.985 INFO:teuthology.orchestra.run.smithi114.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T14:47:49.985 INFO:teuthology.orchestra.run.smithi114.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T14:47:49.985 INFO:teuthology.orchestra.run.smithi114.stdout: = sunit=0 swidth=0 blks 2024-09-16T14:47:49.985 INFO:teuthology.orchestra.run.smithi114.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T14:47:49.985 INFO:teuthology.orchestra.run.smithi114.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T14:47:49.985 INFO:teuthology.orchestra.run.smithi114.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T14:47:49.985 INFO:teuthology.orchestra.run.smithi114.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T14:47:50.497 INFO:teuthology.orchestra.run.smithi114.stdout:Discarding blocks...Done. 2024-09-16T14:47:50.499 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi114.front.sepia.ceph.com -o noatime 2024-09-16T14:47:50.499 DEBUG:teuthology.orchestra.run.smithi114:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-11 2024-09-16T14:47:50.582 DEBUG:teuthology.orchestra.run.smithi114:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2024-09-16T14:47:50.654 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:50.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:50.762+0000 7fd664818640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2024-09-16T14:47:50.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:50.762+0000 7fd664818640 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2024-09-16T14:47:50.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:50.762+0000 7fd664818640 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-16T14:47:50.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:50.762+0000 7fd664818640 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-16T14:47:50.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:50.762+0000 7fd664818640 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-16T14:47:50.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:50.763+0000 7fd664818640 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2024-09-16T14:47:53.617 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2024-09-16T14:47:53.646 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:53.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:53.736+0000 7f256ee18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2024-09-16T14:47:53.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:53.736+0000 7f256ee18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2024-09-16T14:47:53.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:53.736+0000 7f256ee18640 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-16T14:47:53.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:53.736+0000 7f256ee18640 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-16T14:47:53.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:53.736+0000 7f256ee18640 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-16T14:47:53.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:53.737+0000 7f256ee18640 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2024-09-16T14:47:56.594 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2024-09-16T14:47:56.622 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:56.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:56.710+0000 7efd1da0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2024-09-16T14:47:56.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:56.710+0000 7efd1da0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2024-09-16T14:47:56.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:56.710+0000 7efd1da0d640 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-09-16T14:47:56.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:56.710+0000 7efd1da0d640 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-09-16T14:47:56.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:56.710+0000 7efd1da0d640 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-09-16T14:47:56.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:56.710+0000 7efd1da0d640 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2024-09-16T14:47:59.570 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2024-09-16T14:47:59.599 DEBUG:teuthology.orchestra.run.smithi114:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:47:59.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:59.685+0000 7f248bbb2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2024-09-16T14:47:59.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:59.686+0000 7f248bbb2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2024-09-16T14:47:59.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:59.686+0000 7f248bbb2640 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-09-16T14:47:59.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:59.686+0000 7f248bbb2640 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-09-16T14:47:59.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:59.686+0000 7f248bbb2640 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-09-16T14:47:59.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-16T14:47:59.686+0000 7f248bbb2640 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2024-09-16T14:48:02.540 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2024-09-16T14:48:02.569 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T14:48:02.570 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:48:02.570 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T14:48:02.602 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:48:02.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T14:48:02.672 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:48:02.672 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T14:48:02.741 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:48:02.741 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-16T14:48:02.809 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:48:02.809 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T14:48:02.839 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:48:02.839 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-16T14:48:02.907 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:48:02.907 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-16T14:48:02.973 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:48:02.974 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-16T14:48:03.040 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:48:03.040 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-16T14:48:03.105 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:48:03.106 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2024-09-16T14:48:03.135 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:48:03.135 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2024-09-16T14:48:03.205 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:48:03.205 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2024-09-16T14:48:03.273 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:48:03.273 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2024-09-16T14:48:03.340 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:48:03.340 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T14:48:03.360 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T14:48:03.361 DEBUG:teuthology.orchestra.run.smithi040:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T14:48:03.402 DEBUG:teuthology.orchestra.run.smithi111:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T14:48:03.405 DEBUG:teuthology.orchestra.run.smithi114:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T14:48:03.427 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.0] 2024-09-16T14:48:03.428 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD4ROhmDuG4JRAAD/nWq2hQ1SJ2etF7P8CHsA== 2024-09-16T14:48:03.428 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.1] 2024-09-16T14:48:03.428 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD7ROhm3BhmJBAAED2lUNHJdrF8ZJrhocsn4g== 2024-09-16T14:48:03.428 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.2] 2024-09-16T14:48:03.428 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD+ROhm5z0TIxAAK5xtBRJgFRF2el9Twxfw8Q== 2024-09-16T14:48:03.428 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.3] 2024-09-16T14:48:03.428 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQABRehmem20IRAAAIUQYb3NJ7xRddFfs8IwDg== 2024-09-16T14:48:03.428 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr.x] 2024-09-16T14:48:03.428 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD1ROhmtWfgFBAAZuUmT+suGY0tcLrZ3/87zg== 2024-09-16T14:48:03.428 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.4] 2024-09-16T14:48:03.429 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAHRehmjDwIJBAAl6HejDfmPHmfG0Lotvxvrg== 2024-09-16T14:48:03.429 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.5] 2024-09-16T14:48:03.429 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAKRehmcLIKIhAAcyrol9uQG7I0gJh9axi6MQ== 2024-09-16T14:48:03.429 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.6] 2024-09-16T14:48:03.429 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQANRehml2h/IBAAL9CFyewPpW2nfeztc2z+Wg== 2024-09-16T14:48:03.429 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.7] 2024-09-16T14:48:03.429 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAQRehmbYWhHhAA9Z1Aax8mQyMRXf1oZHaXYQ== 2024-09-16T14:48:03.429 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.8] 2024-09-16T14:48:03.429 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAWRehm0+eBLRAAgE1QIFj9JCYbbsglIcfvLg== 2024-09-16T14:48:03.429 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.9] 2024-09-16T14:48:03.429 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAZRehmFCb2KxAAl1mUMMT6cNsqKnt4Tgh7PQ== 2024-09-16T14:48:03.429 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.10] 2024-09-16T14:48:03.429 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAcRehm/15hKhAAv1untvw/NVh1f25PZ3a6EA== 2024-09-16T14:48:03.430 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.11] 2024-09-16T14:48:03.430 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQAfRehmtwfxKBAA/S+pg5HhNWskAEruAHeUxA== 2024-09-16T14:48:03.430 INFO:teuthology.orchestra.run.smithi111.stdout:[client.0] 2024-09-16T14:48:03.430 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQD1ROhmcEinFxAA0vvNHW+WGvTuHUGruNoJZA== 2024-09-16T14:48:03.431 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.0] 2024-09-16T14:48:03.431 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQD4ROhmDuG4JRAAD/nWq2hQ1SJ2etF7P8CHsA== 2024-09-16T14:48:03.431 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.1] 2024-09-16T14:48:03.431 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQD7ROhm3BhmJBAAED2lUNHJdrF8ZJrhocsn4g== 2024-09-16T14:48:03.431 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.2] 2024-09-16T14:48:03.432 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQD+ROhm5z0TIxAAK5xtBRJgFRF2el9Twxfw8Q== 2024-09-16T14:48:03.432 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.3] 2024-09-16T14:48:03.432 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQABRehmem20IRAAAIUQYb3NJ7xRddFfs8IwDg== 2024-09-16T14:48:03.432 INFO:teuthology.orchestra.run.smithi114.stdout:[mgr.x] 2024-09-16T14:48:03.432 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQD1ROhmtWfgFBAAZuUmT+suGY0tcLrZ3/87zg== 2024-09-16T14:48:03.432 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.4] 2024-09-16T14:48:03.432 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAHRehmjDwIJBAAl6HejDfmPHmfG0Lotvxvrg== 2024-09-16T14:48:03.432 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.5] 2024-09-16T14:48:03.432 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAKRehmcLIKIhAAcyrol9uQG7I0gJh9axi6MQ== 2024-09-16T14:48:03.432 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.6] 2024-09-16T14:48:03.432 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQANRehml2h/IBAAL9CFyewPpW2nfeztc2z+Wg== 2024-09-16T14:48:03.432 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.7] 2024-09-16T14:48:03.432 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAQRehmbYWhHhAA9Z1Aax8mQyMRXf1oZHaXYQ== 2024-09-16T14:48:03.433 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.8] 2024-09-16T14:48:03.433 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAWRehm0+eBLRAAgE1QIFj9JCYbbsglIcfvLg== 2024-09-16T14:48:03.433 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.9] 2024-09-16T14:48:03.433 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAZRehmFCb2KxAAl1mUMMT6cNsqKnt4Tgh7PQ== 2024-09-16T14:48:03.433 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.10] 2024-09-16T14:48:03.433 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAcRehm/15hKhAAv1untvw/NVh1f25PZ3a6EA== 2024-09-16T14:48:03.433 INFO:teuthology.orchestra.run.smithi114.stdout:[osd.11] 2024-09-16T14:48:03.434 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAfRehmtwfxKBAA/S+pg5HhNWskAEruAHeUxA== 2024-09-16T14:48:03.434 INFO:teuthology.orchestra.run.smithi114.stdout:[client.0] 2024-09-16T14:48:03.434 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQD1ROhmcEinFxAA0vvNHW+WGvTuHUGruNoJZA== 2024-09-16T14:48:03.435 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2024-09-16T14:48:03.435 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQD4ROhmDuG4JRAAD/nWq2hQ1SJ2etF7P8CHsA== 2024-09-16T14:48:03.435 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2024-09-16T14:48:03.436 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQD7ROhm3BhmJBAAED2lUNHJdrF8ZJrhocsn4g== 2024-09-16T14:48:03.436 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2024-09-16T14:48:03.436 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQD+ROhm5z0TIxAAK5xtBRJgFRF2el9Twxfw8Q== 2024-09-16T14:48:03.436 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.3] 2024-09-16T14:48:03.436 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQABRehmem20IRAAAIUQYb3NJ7xRddFfs8IwDg== 2024-09-16T14:48:03.436 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.x] 2024-09-16T14:48:03.436 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQD1ROhmtWfgFBAAZuUmT+suGY0tcLrZ3/87zg== 2024-09-16T14:48:03.436 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.4] 2024-09-16T14:48:03.436 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAHRehmjDwIJBAAl6HejDfmPHmfG0Lotvxvrg== 2024-09-16T14:48:03.436 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.5] 2024-09-16T14:48:03.436 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAKRehmcLIKIhAAcyrol9uQG7I0gJh9axi6MQ== 2024-09-16T14:48:03.436 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.6] 2024-09-16T14:48:03.437 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQANRehml2h/IBAAL9CFyewPpW2nfeztc2z+Wg== 2024-09-16T14:48:03.437 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.7] 2024-09-16T14:48:03.437 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAQRehmbYWhHhAA9Z1Aax8mQyMRXf1oZHaXYQ== 2024-09-16T14:48:03.437 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.8] 2024-09-16T14:48:03.437 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAWRehm0+eBLRAAgE1QIFj9JCYbbsglIcfvLg== 2024-09-16T14:48:03.437 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.9] 2024-09-16T14:48:03.437 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAZRehmFCb2KxAAl1mUMMT6cNsqKnt4Tgh7PQ== 2024-09-16T14:48:03.437 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.10] 2024-09-16T14:48:03.437 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAcRehm/15hKhAAv1untvw/NVh1f25PZ3a6EA== 2024-09-16T14:48:03.437 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.11] 2024-09-16T14:48:03.437 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAfRehmtwfxKBAA/S+pg5HhNWskAEruAHeUxA== 2024-09-16T14:48:03.437 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-09-16T14:48:03.437 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQD1ROhmcEinFxAA0vvNHW+WGvTuHUGruNoJZA== 2024-09-16T14:48:03.438 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.478 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.481 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.534 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.536 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.538 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.589 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.591 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.594 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.643 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.646 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.648 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.698 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T14:48:03.700 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T14:48:03.702 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T14:48:03.752 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.755 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.757 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.808 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.810 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.813 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.865 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.868 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.870 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.921 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.923 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.925 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.978 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:03.980 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:04.030 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:04.033 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:04.036 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:04.086 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:04.089 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:04.091 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:04.140 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:04.142 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:04.145 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T14:48:04.195 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T14:48:04.198 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T14:48:04.201 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T14:48:04.253 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T14:48:04.254 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T14:48:04.280 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T14:48:04.377 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T14:48:04.403 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T14:48:04.428 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T14:48:04.524 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T14:48:04.549 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T14:48:04.574 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T14:48:04.671 DEBUG:teuthology.orchestra.run.smithi114:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T14:48:04.695 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:48:04.698 DEBUG:teuthology.orchestra.run.smithi111:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:48:04.700 DEBUG:teuthology.orchestra.run.smithi114:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T14:48:04.750 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T14:48:04.751 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T14:48:04.751 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T14:48:04.755 INFO:tasks.ceph.mon.a:Started 2024-09-16T14:48:04.756 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T14:48:04.756 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T14:48:04.758 INFO:tasks.ceph.mon.b:Started 2024-09-16T14:48:04.758 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T14:48:04.758 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T14:48:04.792 INFO:tasks.ceph.mon.c:Started 2024-09-16T14:48:04.793 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T14:48:04.793 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T14:48:04.793 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T14:48:04.795 INFO:tasks.ceph.mgr.x:Started 2024-09-16T14:48:04.795 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T14:48:04.795 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T14:48:10.162 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2024-09-16T14:48:10.174 INFO:tasks.ceph:check_enable_crimson: False 2024-09-16T14:48:10.175 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T14:48:10.175 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:48:10.175 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T14:48:10.186 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:10.185+0000 7f5ee39ef040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T14:48:10.202 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:48:10.202 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T14:48:10.227 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:10.226+0000 7f5ee39ef040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T14:48:10.268 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:48:10.268 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T14:48:10.334 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-16T14:48:10.334 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-16T14:48:10.365 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:10.364+0000 7f5ee39ef040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T14:48:10.400 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:48:10.400 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-16T14:48:10.415 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:10.415+0000 7f5ee39ef040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T14:48:10.425 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:48:10.425 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-16T14:48:10.489 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:48:10.489 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-16T14:48:10.553 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-16T14:48:10.554 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-16T14:48:10.591 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:10.590+0000 7f5ee39ef040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:48:10.615 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:48:10.616 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2024-09-16T14:48:10.646 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:48:10.646 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2024-09-16T14:48:10.703 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:10.702+0000 7f5ee39ef040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T14:48:10.715 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:48:10.715 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2024-09-16T14:48:10.783 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-16T14:48:10.783 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2024-09-16T14:48:10.850 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new ffcfdeb7-eb75-4b3a-bc2a-f1834271654c 0 2024-09-16T14:48:11.010 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:11.009+0000 7f5ee39ef040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T14:48:11.063 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:11.062+0000 7f5ee39ef040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T14:48:11.069 INFO:teuthology.orchestra.run.smithi114.stdout:0 2024-09-16T14:48:11.081 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 6fb8c93f-5e3c-492f-bd0c-6d97764467f5 1 2024-09-16T14:48:11.209 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:11.207+0000 7f5ee39ef040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T14:48:11.257 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-09-16T14:48:11.268 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 500457a9-0d59-451c-b2ff-28aa363056bf 2 2024-09-16T14:48:11.446 INFO:teuthology.orchestra.run.smithi114.stdout:2 2024-09-16T14:48:11.457 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new f6e48d98-5d87-4cd5-8bd8-6e4c9dc56a39 3 2024-09-16T14:48:11.634 INFO:teuthology.orchestra.run.smithi114.stdout:3 2024-09-16T14:48:11.645 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 474dd40c-2b73-44ea-b6a9-48605b78125c 4 2024-09-16T14:48:11.822 INFO:teuthology.orchestra.run.smithi114.stdout:4 2024-09-16T14:48:11.833 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 479bfe9b-00f6-4763-9740-9c9fd11e2587 5 2024-09-16T14:48:12.009 INFO:teuthology.orchestra.run.smithi114.stdout:5 2024-09-16T14:48:12.020 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 6ac36041-045e-4813-ad35-ca4e224811fa 6 2024-09-16T14:48:12.140 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:12.138+0000 7f5ee39ef040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T14:48:12.191 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:12.189+0000 7f5ee39ef040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T14:48:12.193 INFO:teuthology.orchestra.run.smithi114.stdout:6 2024-09-16T14:48:12.204 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 129006cf-b82b-4e11-a007-63deb355d007 7 2024-09-16T14:48:12.377 INFO:teuthology.orchestra.run.smithi114.stdout:7 2024-09-16T14:48:12.387 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 27e746ee-f67a-4a73-8976-86d012469657 8 2024-09-16T14:48:12.554 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:12.552+0000 7f5ee39ef040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T14:48:12.559 INFO:teuthology.orchestra.run.smithi114.stdout:8 2024-09-16T14:48:12.570 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 90975d4f-24b3-4c84-916e-3138bc52a076 9 2024-09-16T14:48:12.637 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:12.635+0000 7f5ee39ef040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T14:48:12.741 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:12.739+0000 7f5ee39ef040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T14:48:12.745 INFO:teuthology.orchestra.run.smithi114.stdout:9 2024-09-16T14:48:12.756 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 89c8157b-52ce-4c1c-9df7-e214df30a827 10 2024-09-16T14:48:12.786 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:12.784+0000 7f5ee39ef040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T14:48:12.895 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:12.893+0000 7f5ee39ef040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T14:48:12.927 INFO:teuthology.orchestra.run.smithi114.stdout:10 2024-09-16T14:48:12.938 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --cluster ceph osd new 0dfbd657-c9f0-42f9-9b43-323c45d89c1f 11 2024-09-16T14:48:12.947 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:12.945+0000 7f5ee39ef040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T14:48:12.991 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:12.989+0000 7f5ee39ef040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T14:48:13.102 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:13.100+0000 7f5ee39ef040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T14:48:13.111 INFO:teuthology.orchestra.run.smithi114.stdout:11 2024-09-16T14:48:13.122 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T14:48:13.122 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T14:48:13.125 INFO:tasks.ceph.osd.0:Started 2024-09-16T14:48:13.125 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T14:48:13.125 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T14:48:13.127 INFO:tasks.ceph.osd.1:Started 2024-09-16T14:48:13.127 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T14:48:13.127 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T14:48:13.129 INFO:tasks.ceph.osd.2:Started 2024-09-16T14:48:13.130 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T14:48:13.130 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T14:48:13.132 INFO:tasks.ceph.osd.3:Started 2024-09-16T14:48:13.132 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-16T14:48:13.132 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-16T14:48:13.134 INFO:tasks.ceph.osd.4:Started 2024-09-16T14:48:13.134 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-16T14:48:13.134 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-16T14:48:13.136 INFO:tasks.ceph.osd.5:Started 2024-09-16T14:48:13.137 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-16T14:48:13.137 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-16T14:48:13.139 INFO:tasks.ceph.osd.6:Started 2024-09-16T14:48:13.139 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-16T14:48:13.139 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-16T14:48:13.142 INFO:tasks.ceph.osd.7:Started 2024-09-16T14:48:13.142 INFO:tasks.ceph.osd.8:Restarting daemon 2024-09-16T14:48:13.142 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2024-09-16T14:48:13.145 INFO:tasks.ceph.osd.8:Started 2024-09-16T14:48:13.145 INFO:tasks.ceph.osd.9:Restarting daemon 2024-09-16T14:48:13.145 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2024-09-16T14:48:13.147 INFO:tasks.ceph.osd.9:Started 2024-09-16T14:48:13.147 INFO:tasks.ceph.osd.10:Restarting daemon 2024-09-16T14:48:13.147 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 10 2024-09-16T14:48:13.149 INFO:tasks.ceph.osd.10:Started 2024-09-16T14:48:13.149 INFO:tasks.ceph.osd.11:Restarting daemon 2024-09-16T14:48:13.149 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 11 2024-09-16T14:48:13.152 INFO:tasks.ceph.osd.11:Started 2024-09-16T14:48:13.152 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:48:13.159 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:13.157+0000 7f5ee39ef040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T14:48:13.226 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:13.224+0000 7f5ee39ef040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T14:48:13.296 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:13.294+0000 7f5ee39ef040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T14:48:13.340 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:13.338+0000 7f5ee39ef040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T14:48:13.351 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:48:13.351 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"fsid":"aaaa1c0f-de8b-48c8-9412-815c8b70b7b8","created":"2024-09-16T14:48:10.063465+0000","modified":"2024-09-16T14:48:13.108012+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T14:48:13.108012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ffcfdeb7-eb75-4b3a-bc2a-f1834271654c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6fb8c93f-5e3c-492f-bd0c-6d97764467f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"500457a9-0d59-451c-b2ff-28aa363056bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f6e48d98-5d87-4cd5-8bd8-6e4c9dc56a39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"474dd40c-2b73-44ea-b6a9-48605b78125c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"479bfe9b-00f6-4763-9740-9c9fd11e2587","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6ac36041-045e-4813-ad35-ca4e224811fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"129006cf-b82b-4e11-a007-63deb355d007","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"27e746ee-f67a-4a73-8976-86d012469657","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"90975d4f-24b3-4c84-916e-3138bc52a076","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"89c8157b-52ce-4c1c-9df7-e214df30a827","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"0dfbd657-c9f0-42f9-9b43-323c45d89c1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:48:13.362 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T14:48:13.362 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T14:48:13.383 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:13.381+0000 7f5ee39ef040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T14:48:13.733 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:13.731+0000 7f552c9a6640 -1 Falling back to public interface 2024-09-16T14:48:13.737 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:13.735+0000 7fda9a40f640 -1 Falling back to public interface 2024-09-16T14:48:13.738 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:13.736+0000 7fe1f3e15640 -1 Falling back to public interface 2024-09-16T14:48:13.741 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:13.739+0000 7fbcdf9b2640 -1 Falling back to public interface 2024-09-16T14:48:13.742 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:13.740+0000 7f95e8614640 -1 Falling back to public interface 2024-09-16T14:48:13.745 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:13.742+0000 7f18afba6640 -1 Falling back to public interface 2024-09-16T14:48:13.745 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:13.743+0000 7f6890a16640 -1 Falling back to public interface 2024-09-16T14:48:13.747 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:13.745+0000 7fe31141e640 -1 Falling back to public interface 2024-09-16T14:48:13.751 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:13.749+0000 7f438b8ed640 -1 Falling back to public interface 2024-09-16T14:48:13.751 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:13.749+0000 7f40a2026640 -1 Falling back to public interface 2024-09-16T14:48:13.752 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:13.750+0000 7f7e00e14640 -1 Falling back to public interface 2024-09-16T14:48:13.753 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:13.751+0000 7f7cd2e2f640 -1 Falling back to public interface 2024-09-16T14:48:14.020 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:14.018+0000 7f5ee39ef040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T14:48:14.571 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T14:48:14.743 INFO:teuthology.misc.health.smithi040.stdout: 2024-09-16T14:48:14.743 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":14,"fsid":"aaaa1c0f-de8b-48c8-9412-815c8b70b7b8","created":"2024-09-16T14:48:10.063465+0000","modified":"2024-09-16T14:48:13.108012+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T14:48:13.108012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ffcfdeb7-eb75-4b3a-bc2a-f1834271654c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6fb8c93f-5e3c-492f-bd0c-6d97764467f5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"500457a9-0d59-451c-b2ff-28aa363056bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f6e48d98-5d87-4cd5-8bd8-6e4c9dc56a39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"474dd40c-2b73-44ea-b6a9-48605b78125c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"479bfe9b-00f6-4763-9740-9c9fd11e2587","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6ac36041-045e-4813-ad35-ca4e224811fa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"129006cf-b82b-4e11-a007-63deb355d007","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"27e746ee-f67a-4a73-8976-86d012469657","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"90975d4f-24b3-4c84-916e-3138bc52a076","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"89c8157b-52ce-4c1c-9df7-e214df30a827","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"0dfbd657-c9f0-42f9-9b43-323c45d89c1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:48:14.752 DEBUG:teuthology.misc:0 of 12 OSDs are up 2024-09-16T14:48:15.089 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:15.087+0000 7f5e84c82640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T14:48:15.089 INFO:tasks.ceph.mgr.x.smithi111.stderr:2024-09-16T14:48:15.087+0000 7f5e84c82640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T14:48:17.164 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:17.162+0000 7fda9a40f640 -1 osd.9 0 log_to_monitors true 2024-09-16T14:48:17.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:17.170+0000 7f552c9a6640 -1 osd.0 0 log_to_monitors true 2024-09-16T14:48:17.184 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:17.182+0000 7fe1f3e15640 -1 osd.5 0 log_to_monitors true 2024-09-16T14:48:17.211 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:17.210+0000 7f95e8614640 -1 osd.10 0 log_to_monitors true 2024-09-16T14:48:17.213 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:17.211+0000 7f18afba6640 -1 osd.2 0 log_to_monitors true 2024-09-16T14:48:17.223 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:17.221+0000 7f40a2026640 -1 osd.4 0 log_to_monitors true 2024-09-16T14:48:17.255 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:17.253+0000 7f7cd2e2f640 -1 osd.1 0 log_to_monitors true 2024-09-16T14:48:17.257 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:17.255+0000 7fe31141e640 -1 osd.11 0 log_to_monitors true 2024-09-16T14:48:17.258 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:17.256+0000 7fbcdf9b2640 -1 osd.7 0 log_to_monitors true 2024-09-16T14:48:17.258 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:17.257+0000 7f6890a16640 -1 osd.8 0 log_to_monitors true 2024-09-16T14:48:17.259 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:17.257+0000 7f438b8ed640 -1 osd.6 0 log_to_monitors true 2024-09-16T14:48:17.278 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:17.276+0000 7f7e00e14640 -1 osd.3 0 log_to_monitors true 2024-09-16T14:48:19.116 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:19.114+0000 7f18ab310640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T14:48:19.117 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:19.116+0000 7fda9556d640 -1 osd.9 0 waiting for initial osdmap 2024-09-16T14:48:19.120 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:19.119+0000 7fbcdb11c640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T14:48:19.125 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:19.123+0000 7f7dfbe79640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T14:48:19.140 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:19.139+0000 7f688ba79640 -1 osd.8 0 waiting for initial osdmap 2024-09-16T14:48:19.140 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:19.138+0000 7f409d790640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T14:48:19.152 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:19.150+0000 7f7df84b4640 -1 osd.3 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:48:19.153 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:19.150+0000 7f18a794b640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:48:19.164 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:19.161+0000 7fbcd7757640 -1 osd.7 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:48:19.172 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:19.170+0000 7f4099dcb640 -1 osd.4 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:48:19.206 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:19.205+0000 7fda91ba8640 -1 osd.9 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:48:19.217 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:19.216+0000 7f68880b4640 -1 osd.8 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:48:20.331 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:20.328+0000 7fe30c638640 -1 osd.11 0 waiting for initial osdmap 2024-09-16T14:48:20.338 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:20.336+0000 7f95e4eb4640 -1 osd.10 0 waiting for initial osdmap 2024-09-16T14:48:20.341 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:20.339+0000 7f4387057640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T14:48:20.344 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:20.342+0000 7fe304c6b640 -1 osd.11 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:48:20.351 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:20.349+0000 7f95dbcac640 -1 osd.10 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:48:20.355 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:20.353+0000 7f437f68a640 -1 osd.6 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:48:20.358 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:20.356+0000 7f7cce599640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T14:48:20.359 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:20.357+0000 7fe1eef00640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T14:48:20.359 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:20.357+0000 7fe1f073b640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T14:48:20.363 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:20.361+0000 7fe1e7533640 -1 osd.5 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:48:20.374 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:20.372+0000 7f7cc6bcc640 -1 osd.1 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:48:20.375 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:20.373+0000 7f5528110640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T14:48:20.416 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:20.414+0000 7f5520743640 -1 osd.0 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:48:21.963 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T14:48:22.257 INFO:teuthology.misc.health.smithi040.stdout: 2024-09-16T14:48:22.257 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":19,"fsid":"aaaa1c0f-de8b-48c8-9412-815c8b70b7b8","created":"2024-09-16T14:48:10.063465+0000","modified":"2024-09-16T14:48:22.128378+0000","last_up_change":"2024-09-16T14:48:21.118926+0000","last_in_change":"2024-09-16T14:48:13.108012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:48:20.279244+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffcfdeb7-eb75-4b3a-bc2a-f1834271654c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":725262},{"type":"v1","addr":"172.21.15.40:6801","nonce":725262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":725262},{"type":"v1","addr":"172.21.15.40:6803","nonce":725262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":725262},{"type":"v1","addr":"172.21.15.40:6807","nonce":725262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":725262},{"type":"v1","addr":"172.21.15.40:6805","nonce":725262}]},"public_addr":"172.21.15.40:6801/725262","cluster_addr":"172.21.15.40:6803/725262","heartbeat_back_addr":"172.21.15.40:6807/725262","heartbeat_front_addr":"172.21.15.40:6805/725262","state":["exists","up"]},{"osd":1,"uuid":"6fb8c93f-5e3c-492f-bd0c-6d97764467f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6821","nonce":501335559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6825","nonce":501335559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6831","nonce":501335559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6829","nonce":501335559}]},"public_addr":"172.21.15.40:6821/501335559","cluster_addr":"172.21.15.40:6825/501335559","heartbeat_back_addr":"172.21.15.40:6831/501335559","heartbeat_front_addr":"172.21.15.40:6829/501335559","state":["exists","up"]},{"osd":2,"uuid":"500457a9-0d59-451c-b2ff-28aa363056bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6809","nonce":1490162929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6811","nonce":1490162929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6815","nonce":1490162929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6813","nonce":1490162929}]},"public_addr":"172.21.15.40:6809/1490162929","cluster_addr":"172.21.15.40:6811/1490162929","heartbeat_back_addr":"172.21.15.40:6815/1490162929","heartbeat_front_addr":"172.21.15.40:6813/1490162929","state":["exists","up"]},{"osd":3,"uuid":"f6e48d98-5d87-4cd5-8bd8-6e4c9dc56a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6817","nonce":682463486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6820","nonce":682463486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6828","nonce":682463486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6824","nonce":682463486}]},"public_addr":"172.21.15.40:6817/682463486","cluster_addr":"172.21.15.40:6820/682463486","heartbeat_back_addr":"172.21.15.40:6828/682463486","heartbeat_front_addr":"172.21.15.40:6824/682463486","state":["exists","up"]},{"osd":4,"uuid":"474dd40c-2b73-44ea-b6a9-48605b78125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6818","nonce":1550824417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6819","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6821","nonce":1550824417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6828","nonce":1550824417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6824","nonce":1550824417}]},"public_addr":"172.21.15.111:6818/1550824417","cluster_addr":"172.21.15.111:6821/1550824417","heartbeat_back_addr":"172.21.15.111:6828/1550824417","heartbeat_front_addr":"172.21.15.111:6824/1550824417","state":["exists","up"]},{"osd":5,"uuid":"479bfe9b-00f6-4763-9740-9c9fd11e2587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6801","nonce":3785556374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6803","nonce":3785556374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6807","nonce":3785556374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6805","nonce":3785556374}]},"public_addr":"172.21.15.111:6801/3785556374","cluster_addr":"172.21.15.111:6803/3785556374","heartbeat_back_addr":"172.21.15.111:6807/3785556374","heartbeat_front_addr":"172.21.15.111:6805/3785556374","state":["exists","up"]},{"osd":6,"uuid":"6ac36041-045e-4813-ad35-ca4e224811fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":370046418},{"type":"v1","addr":"172.21.15.111:6820","nonce":370046418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":370046418},{"type":"v1","addr":"172.21.15.111:6825","nonce":370046418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":370046418},{"type":"v1","addr":"172.21.15.111:6831","nonce":370046418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6827","nonce":370046418},{"type":"v1","addr":"172.21.15.111:6829","nonce":370046418}]},"public_addr":"172.21.15.111:6820/370046418","cluster_addr":"172.21.15.111:6825/370046418","heartbeat_back_addr":"172.21.15.111:6831/370046418","heartbeat_front_addr":"172.21.15.111:6829/370046418","state":["exists","up"]},{"osd":7,"uuid":"129006cf-b82b-4e11-a007-63deb355d007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6809","nonce":3591930331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6811","nonce":3591930331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6815","nonce":3591930331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6813","nonce":3591930331}]},"public_addr":"172.21.15.111:6809/3591930331","cluster_addr":"172.21.15.111:6811/3591930331","heartbeat_back_addr":"172.21.15.111:6815/3591930331","heartbeat_front_addr":"172.21.15.111:6813/3591930331","state":["exists","up"]},{"osd":8,"uuid":"27e746ee-f67a-4a73-8976-86d012469657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6817","nonce":2473986485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6819","nonce":2473986485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6824","nonce":2473986485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6821","nonce":2473986485}]},"public_addr":"172.21.15.114:6817/2473986485","cluster_addr":"172.21.15.114:6819/2473986485","heartbeat_back_addr":"172.21.15.114:6824/2473986485","heartbeat_front_addr":"172.21.15.114:6821/2473986485","state":["exists","up"]},{"osd":9,"uuid":"90975d4f-24b3-4c84-916e-3138bc52a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6801","nonce":2290892207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6803","nonce":2290892207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6807","nonce":2290892207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6805","nonce":2290892207}]},"public_addr":"172.21.15.114:6801/2290892207","cluster_addr":"172.21.15.114:6803/2290892207","heartbeat_back_addr":"172.21.15.114:6807/2290892207","heartbeat_front_addr":"172.21.15.114:6805/2290892207","state":["exists","up"]},{"osd":10,"uuid":"89c8157b-52ce-4c1c-9df7-e214df30a827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6809","nonce":1795470140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6811","nonce":1795470140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6815","nonce":1795470140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6813","nonce":1795470140}]},"public_addr":"172.21.15.114:6809/1795470140","cluster_addr":"172.21.15.114:6811/1795470140","heartbeat_back_addr":"172.21.15.114:6815/1795470140","heartbeat_front_addr":"172.21.15.114:6813/1795470140","state":["exists","up"]},{"osd":11,"uuid":"0dfbd657-c9f0-42f9-9b43-323c45d89c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6825","nonce":1823608150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6827","nonce":1823608150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6831","nonce":1823608150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6829","nonce":1823608150}]},"public_addr":"172.21.15.114:6825/1823608150","cluster_addr":"172.21.15.114:6827/1823608150","heartbeat_back_addr":"172.21.15.114:6831/1823608150","heartbeat_front_addr":"172.21.15.114:6829/1823608150","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.220714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.257765+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.258268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.280034+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.251971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.143236+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.279302+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.295724+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.270893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.194896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.199423+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.292698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:48:22.268 DEBUG:teuthology.misc:12 of 12 OSDs are up 2024-09-16T14:48:22.268 INFO:tasks.ceph:Creating RBD pool 2024-09-16T14:48:22.268 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-16T14:48:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'rbd' created 2024-09-16T14:48:23.157 DEBUG:teuthology.orchestra.run.smithi040:> rbd --cluster ceph pool init rbd 2024-09-16T14:48:26.167 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T14:48:26.167 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T14:48:26.167 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T14:48:26.168 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T14:48:26.168 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T14:48:26.510 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:48:26.521 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3450467473},{"type":"v1","addr":"172.21.15.111:6833","nonce":3450467473}]},"active_addr":"172.21.15.111:6833/3450467473","active_change":"2024-09-16T14:48:14.022158+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":3214371635}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":1404405087}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":3151804581}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":3532763039}]}]} 2024-09-16T14:48:26.523 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T14:48:26.523 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T14:48:26.523 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:48:26.808 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:48:26.808 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"aaaa1c0f-de8b-48c8-9412-815c8b70b7b8","created":"2024-09-16T14:48:10.063465+0000","modified":"2024-09-16T14:48:26.150001+0000","last_up_change":"2024-09-16T14:48:21.118926+0000","last_in_change":"2024-09-16T14:48:13.108012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:48:20.279244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T14:48:22.580854+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":0.75,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffcfdeb7-eb75-4b3a-bc2a-f1834271654c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":725262},{"type":"v1","addr":"172.21.15.40:6801","nonce":725262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":725262},{"type":"v1","addr":"172.21.15.40:6803","nonce":725262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":725262},{"type":"v1","addr":"172.21.15.40:6807","nonce":725262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":725262},{"type":"v1","addr":"172.21.15.40:6805","nonce":725262}]},"public_addr":"172.21.15.40:6801/725262","cluster_addr":"172.21.15.40:6803/725262","heartbeat_back_addr":"172.21.15.40:6807/725262","heartbeat_front_addr":"172.21.15.40:6805/725262","state":["exists","up"]},{"osd":1,"uuid":"6fb8c93f-5e3c-492f-bd0c-6d97764467f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6821","nonce":501335559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6825","nonce":501335559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6831","nonce":501335559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6829","nonce":501335559}]},"public_addr":"172.21.15.40:6821/501335559","cluster_addr":"172.21.15.40:6825/501335559","heartbeat_back_addr":"172.21.15.40:6831/501335559","heartbeat_front_addr":"172.21.15.40:6829/501335559","state":["exists","up"]},{"osd":2,"uuid":"500457a9-0d59-451c-b2ff-28aa363056bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6809","nonce":1490162929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6811","nonce":1490162929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6815","nonce":1490162929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6813","nonce":1490162929}]},"public_addr":"172.21.15.40:6809/1490162929","cluster_addr":"172.21.15.40:6811/1490162929","heartbeat_back_addr":"172.21.15.40:6815/1490162929","heartbeat_front_addr":"172.21.15.40:6813/1490162929","state":["exists","up"]},{"osd":3,"uuid":"f6e48d98-5d87-4cd5-8bd8-6e4c9dc56a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6817","nonce":682463486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6820","nonce":682463486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6828","nonce":682463486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6824","nonce":682463486}]},"public_addr":"172.21.15.40:6817/682463486","cluster_addr":"172.21.15.40:6820/682463486","heartbeat_back_addr":"172.21.15.40:6828/682463486","heartbeat_front_addr":"172.21.15.40:6824/682463486","state":["exists","up"]},{"osd":4,"uuid":"474dd40c-2b73-44ea-b6a9-48605b78125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6818","nonce":1550824417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6819","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6821","nonce":1550824417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6828","nonce":1550824417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6824","nonce":1550824417}]},"public_addr":"172.21.15.111:6818/1550824417","cluster_addr":"172.21.15.111:6821/1550824417","heartbeat_back_addr":"172.21.15.111:6828/1550824417","heartbeat_front_addr":"172.21.15.111:6824/1550824417","state":["exists","up"]},{"osd":5,"uuid":"479bfe9b-00f6-4763-9740-9c9fd11e2587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6801","nonce":3785556374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6803","nonce":3785556374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6807","nonce":3785556374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6805","nonce":3785556374}]},"public_addr":"172.21.15.111:6801/3785556374","cluster_addr":"172.21.15.111:6803/3785556374","heartbeat_back_addr":"172.21.15.111:6807/3785556374","heartbeat_front_addr":"172.21.15.111:6805/3785556374","state":["exists","up"]},{"osd":6,"uuid":"6ac36041-045e-4813-ad35-ca4e224811fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":370046418},{"type":"v1","addr":"172.21.15.111:6820","nonce":370046418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":370046418},{"type":"v1","addr":"172.21.15.111:6825","nonce":370046418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":370046418},{"type":"v1","addr":"172.21.15.111:6831","nonce":370046418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6827","nonce":370046418},{"type":"v1","addr":"172.21.15.111:6829","nonce":370046418}]},"public_addr":"172.21.15.111:6820/370046418","cluster_addr":"172.21.15.111:6825/370046418","heartbeat_back_addr":"172.21.15.111:6831/370046418","heartbeat_front_addr":"172.21.15.111:6829/370046418","state":["exists","up"]},{"osd":7,"uuid":"129006cf-b82b-4e11-a007-63deb355d007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6809","nonce":3591930331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6811","nonce":3591930331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6815","nonce":3591930331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6813","nonce":3591930331}]},"public_addr":"172.21.15.111:6809/3591930331","cluster_addr":"172.21.15.111:6811/3591930331","heartbeat_back_addr":"172.21.15.111:6815/3591930331","heartbeat_front_addr":"172.21.15.111:6813/3591930331","state":["exists","up"]},{"osd":8,"uuid":"27e746ee-f67a-4a73-8976-86d012469657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6817","nonce":2473986485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6819","nonce":2473986485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6824","nonce":2473986485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6821","nonce":2473986485}]},"public_addr":"172.21.15.114:6817/2473986485","cluster_addr":"172.21.15.114:6819/2473986485","heartbeat_back_addr":"172.21.15.114:6824/2473986485","heartbeat_front_addr":"172.21.15.114:6821/2473986485","state":["exists","up"]},{"osd":9,"uuid":"90975d4f-24b3-4c84-916e-3138bc52a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6801","nonce":2290892207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6803","nonce":2290892207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6807","nonce":2290892207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6805","nonce":2290892207}]},"public_addr":"172.21.15.114:6801/2290892207","cluster_addr":"172.21.15.114:6803/2290892207","heartbeat_back_addr":"172.21.15.114:6807/2290892207","heartbeat_front_addr":"172.21.15.114:6805/2290892207","state":["exists","up"]},{"osd":10,"uuid":"89c8157b-52ce-4c1c-9df7-e214df30a827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6809","nonce":1795470140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6811","nonce":1795470140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6815","nonce":1795470140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6813","nonce":1795470140}]},"public_addr":"172.21.15.114:6809/1795470140","cluster_addr":"172.21.15.114:6811/1795470140","heartbeat_back_addr":"172.21.15.114:6815/1795470140","heartbeat_front_addr":"172.21.15.114:6813/1795470140","state":["exists","up"]},{"osd":11,"uuid":"0dfbd657-c9f0-42f9-9b43-323c45d89c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6825","nonce":1823608150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6827","nonce":1823608150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6831","nonce":1823608150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6829","nonce":1823608150}]},"public_addr":"172.21.15.114:6825/1823608150","cluster_addr":"172.21.15.114:6827/1823608150","heartbeat_back_addr":"172.21.15.114:6831/1823608150","heartbeat_front_addr":"172.21.15.114:6829/1823608150","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.220714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.257765+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.258268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.280034+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.251971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.143236+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.279302+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.295724+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.270893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.194896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.199423+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.292698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:48:26.821 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T14:48:26.821 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:48:27.106 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:48:27.106 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":23,"fsid":"aaaa1c0f-de8b-48c8-9412-815c8b70b7b8","created":"2024-09-16T14:48:10.063465+0000","modified":"2024-09-16T14:48:26.150001+0000","last_up_change":"2024-09-16T14:48:21.118926+0000","last_in_change":"2024-09-16T14:48:13.108012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:48:20.279244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T14:48:22.580854+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":0.75,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffcfdeb7-eb75-4b3a-bc2a-f1834271654c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":725262},{"type":"v1","addr":"172.21.15.40:6801","nonce":725262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":725262},{"type":"v1","addr":"172.21.15.40:6803","nonce":725262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":725262},{"type":"v1","addr":"172.21.15.40:6807","nonce":725262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":725262},{"type":"v1","addr":"172.21.15.40:6805","nonce":725262}]},"public_addr":"172.21.15.40:6801/725262","cluster_addr":"172.21.15.40:6803/725262","heartbeat_back_addr":"172.21.15.40:6807/725262","heartbeat_front_addr":"172.21.15.40:6805/725262","state":["exists","up"]},{"osd":1,"uuid":"6fb8c93f-5e3c-492f-bd0c-6d97764467f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6821","nonce":501335559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6825","nonce":501335559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6831","nonce":501335559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6829","nonce":501335559}]},"public_addr":"172.21.15.40:6821/501335559","cluster_addr":"172.21.15.40:6825/501335559","heartbeat_back_addr":"172.21.15.40:6831/501335559","heartbeat_front_addr":"172.21.15.40:6829/501335559","state":["exists","up"]},{"osd":2,"uuid":"500457a9-0d59-451c-b2ff-28aa363056bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6809","nonce":1490162929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6811","nonce":1490162929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6815","nonce":1490162929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6813","nonce":1490162929}]},"public_addr":"172.21.15.40:6809/1490162929","cluster_addr":"172.21.15.40:6811/1490162929","heartbeat_back_addr":"172.21.15.40:6815/1490162929","heartbeat_front_addr":"172.21.15.40:6813/1490162929","state":["exists","up"]},{"osd":3,"uuid":"f6e48d98-5d87-4cd5-8bd8-6e4c9dc56a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6817","nonce":682463486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6820","nonce":682463486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6828","nonce":682463486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6824","nonce":682463486}]},"public_addr":"172.21.15.40:6817/682463486","cluster_addr":"172.21.15.40:6820/682463486","heartbeat_back_addr":"172.21.15.40:6828/682463486","heartbeat_front_addr":"172.21.15.40:6824/682463486","state":["exists","up"]},{"osd":4,"uuid":"474dd40c-2b73-44ea-b6a9-48605b78125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6818","nonce":1550824417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6819","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6821","nonce":1550824417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6828","nonce":1550824417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6824","nonce":1550824417}]},"public_addr":"172.21.15.111:6818/1550824417","cluster_addr":"172.21.15.111:6821/1550824417","heartbeat_back_addr":"172.21.15.111:6828/1550824417","heartbeat_front_addr":"172.21.15.111:6824/1550824417","state":["exists","up"]},{"osd":5,"uuid":"479bfe9b-00f6-4763-9740-9c9fd11e2587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6801","nonce":3785556374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6803","nonce":3785556374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6807","nonce":3785556374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6805","nonce":3785556374}]},"public_addr":"172.21.15.111:6801/3785556374","cluster_addr":"172.21.15.111:6803/3785556374","heartbeat_back_addr":"172.21.15.111:6807/3785556374","heartbeat_front_addr":"172.21.15.111:6805/3785556374","state":["exists","up"]},{"osd":6,"uuid":"6ac36041-045e-4813-ad35-ca4e224811fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":370046418},{"type":"v1","addr":"172.21.15.111:6820","nonce":370046418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":370046418},{"type":"v1","addr":"172.21.15.111:6825","nonce":370046418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":370046418},{"type":"v1","addr":"172.21.15.111:6831","nonce":370046418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6827","nonce":370046418},{"type":"v1","addr":"172.21.15.111:6829","nonce":370046418}]},"public_addr":"172.21.15.111:6820/370046418","cluster_addr":"172.21.15.111:6825/370046418","heartbeat_back_addr":"172.21.15.111:6831/370046418","heartbeat_front_addr":"172.21.15.111:6829/370046418","state":["exists","up"]},{"osd":7,"uuid":"129006cf-b82b-4e11-a007-63deb355d007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6809","nonce":3591930331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6811","nonce":3591930331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6815","nonce":3591930331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6813","nonce":3591930331}]},"public_addr":"172.21.15.111:6809/3591930331","cluster_addr":"172.21.15.111:6811/3591930331","heartbeat_back_addr":"172.21.15.111:6815/3591930331","heartbeat_front_addr":"172.21.15.111:6813/3591930331","state":["exists","up"]},{"osd":8,"uuid":"27e746ee-f67a-4a73-8976-86d012469657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6817","nonce":2473986485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6819","nonce":2473986485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6824","nonce":2473986485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6821","nonce":2473986485}]},"public_addr":"172.21.15.114:6817/2473986485","cluster_addr":"172.21.15.114:6819/2473986485","heartbeat_back_addr":"172.21.15.114:6824/2473986485","heartbeat_front_addr":"172.21.15.114:6821/2473986485","state":["exists","up"]},{"osd":9,"uuid":"90975d4f-24b3-4c84-916e-3138bc52a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6801","nonce":2290892207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6803","nonce":2290892207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6807","nonce":2290892207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6805","nonce":2290892207}]},"public_addr":"172.21.15.114:6801/2290892207","cluster_addr":"172.21.15.114:6803/2290892207","heartbeat_back_addr":"172.21.15.114:6807/2290892207","heartbeat_front_addr":"172.21.15.114:6805/2290892207","state":["exists","up"]},{"osd":10,"uuid":"89c8157b-52ce-4c1c-9df7-e214df30a827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6809","nonce":1795470140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6811","nonce":1795470140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6815","nonce":1795470140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6813","nonce":1795470140}]},"public_addr":"172.21.15.114:6809/1795470140","cluster_addr":"172.21.15.114:6811/1795470140","heartbeat_back_addr":"172.21.15.114:6815/1795470140","heartbeat_front_addr":"172.21.15.114:6813/1795470140","state":["exists","up"]},{"osd":11,"uuid":"0dfbd657-c9f0-42f9-9b43-323c45d89c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6825","nonce":1823608150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6827","nonce":1823608150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6831","nonce":1823608150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6829","nonce":1823608150}]},"public_addr":"172.21.15.114:6825/1823608150","cluster_addr":"172.21.15.114:6827/1823608150","heartbeat_back_addr":"172.21.15.114:6831/1823608150","heartbeat_front_addr":"172.21.15.114:6829/1823608150","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.220714+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.257765+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.258268+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.280034+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.251971+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.143236+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.279302+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.295724+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.270893+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.194896+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.199423+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:48:18.292698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:48:27.119 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T14:48:27.120 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T14:48:27.120 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T14:48:27.120 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T14:48:27.121 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T14:48:27.121 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T14:48:27.121 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T14:48:27.122 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T14:48:27.122 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-16T14:48:27.123 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-16T14:48:27.123 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-16T14:48:27.124 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-16T14:48:27.460 INFO:teuthology.orchestra.run.smithi040.stdout:73014444036 2024-09-16T14:48:27.460 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-16T14:48:27.463 INFO:teuthology.orchestra.run.smithi040.stdout:77309411332 2024-09-16T14:48:27.464 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T14:48:27.467 INFO:teuthology.orchestra.run.smithi040.stdout:77309411333 2024-09-16T14:48:27.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T14:48:27.470 INFO:teuthology.orchestra.run.smithi040.stdout:77309411333 2024-09-16T14:48:27.470 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T14:48:27.476 INFO:teuthology.orchestra.run.smithi040.stdout:73014444036 2024-09-16T14:48:27.476 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T14:48:27.483 INFO:teuthology.orchestra.run.smithi040.stdout:73014444035 2024-09-16T14:48:27.483 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T14:48:27.486 INFO:teuthology.orchestra.run.smithi040.stdout:77309411331 2024-09-16T14:48:27.487 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T14:48:27.496 INFO:teuthology.orchestra.run.smithi040.stdout:73014444036 2024-09-16T14:48:27.496 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T14:48:27.501 INFO:teuthology.orchestra.run.smithi040.stdout:73014444036 2024-09-16T14:48:27.502 INFO:teuthology.orchestra.run.smithi040.stdout:77309411331 2024-09-16T14:48:27.502 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-16T14:48:27.502 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-16T14:48:27.512 INFO:teuthology.orchestra.run.smithi040.stdout:73014444036 2024-09-16T14:48:27.512 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T14:48:27.563 INFO:teuthology.orchestra.run.smithi040.stdout:77309411333 2024-09-16T14:48:27.563 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-16T14:48:28.256 INFO:teuthology.orchestra.run.smithi040.stdout:77309411333 2024-09-16T14:48:28.265 INFO:teuthology.orchestra.run.smithi040.stdout:73014444036 2024-09-16T14:48:28.286 INFO:teuthology.orchestra.run.smithi040.stdout:77309411332 2024-09-16T14:48:28.286 INFO:teuthology.orchestra.run.smithi040.stdout:77309411331 2024-09-16T14:48:28.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444036 for osd.8 2024-09-16T14:48:28.290 DEBUG:teuthology.parallel:result is None 2024-09-16T14:48:28.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411333 got 77309411333 for osd.6 2024-09-16T14:48:28.300 DEBUG:teuthology.parallel:result is None 2024-09-16T14:48:28.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411331 got 77309411331 for osd.0 2024-09-16T14:48:28.307 DEBUG:teuthology.parallel:result is None 2024-09-16T14:48:28.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411332 got 77309411332 for osd.5 2024-09-16T14:48:28.325 DEBUG:teuthology.parallel:result is None 2024-09-16T14:48:28.328 INFO:teuthology.orchestra.run.smithi040.stdout:77309411331 2024-09-16T14:48:28.332 INFO:teuthology.orchestra.run.smithi040.stdout:73014444035 2024-09-16T14:48:28.342 INFO:teuthology.orchestra.run.smithi040.stdout:73014444036 2024-09-16T14:48:28.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411331 got 77309411331 for osd.11 2024-09-16T14:48:28.346 DEBUG:teuthology.parallel:result is None 2024-09-16T14:48:28.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444035 got 73014444035 for osd.4 2024-09-16T14:48:28.350 DEBUG:teuthology.parallel:result is None 2024-09-16T14:48:28.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444036 for osd.7 2024-09-16T14:48:28.359 DEBUG:teuthology.parallel:result is None 2024-09-16T14:48:28.373 INFO:teuthology.orchestra.run.smithi040.stdout:73014444036 2024-09-16T14:48:28.380 INFO:teuthology.orchestra.run.smithi040.stdout:77309411333 2024-09-16T14:48:28.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444036 for osd.9 2024-09-16T14:48:28.389 DEBUG:teuthology.parallel:result is None 2024-09-16T14:48:28.391 INFO:teuthology.orchestra.run.smithi040.stdout:73014444036 2024-09-16T14:48:28.394 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411333 got 77309411333 for osd.10 2024-09-16T14:48:28.394 DEBUG:teuthology.parallel:result is None 2024-09-16T14:48:28.404 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444036 for osd.3 2024-09-16T14:48:28.404 DEBUG:teuthology.parallel:result is None 2024-09-16T14:48:28.426 INFO:teuthology.orchestra.run.smithi040.stdout:73014444036 2024-09-16T14:48:28.435 INFO:teuthology.orchestra.run.smithi040.stdout:77309411333 2024-09-16T14:48:28.440 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444036 got 73014444036 for osd.2 2024-09-16T14:48:28.440 DEBUG:teuthology.parallel:result is None 2024-09-16T14:48:28.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411333 got 77309411333 for osd.1 2024-09-16T14:48:28.448 DEBUG:teuthology.parallel:result is None 2024-09-16T14:48:28.448 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T14:48:28.449 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:48:28.770 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:48:28.771 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:48:28.785 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-16T14:48:28.030237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":183,"num_write_kb":2940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":328860,"kb_used_data":3672,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1132133220,"statfs":{"total":1159641169920,"available":1159304417280,"internally_reserved":0,"allocated":3760128,"data_stored":2093182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.875778"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.563974+0000","last_change":"2024-09-16T14:48:26.563974+0000","last_active":"2024-09-16T14:48:26.563974+0000","last_peered":"2024-09-16T14:48:26.563974+0000","last_clean":"2024-09-16T14:48:26.563974+0000","last_became_active":"2024-09-16T14:48:24.165741+0000","last_became_peered":"2024-09-16T14:48:24.165741+0000","last_unstale":"2024-09-16T14:48:26.563974+0000","last_undegraded":"2024-09-16T14:48:26.563974+0000","last_fullsized":"2024-09-16T14:48:26.563974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:47.376916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000285531,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.161840+0000","last_change":"2024-09-16T14:48:26.161840+0000","last_active":"2024-09-16T14:48:26.161840+0000","last_peered":"2024-09-16T14:48:26.161840+0000","last_clean":"2024-09-16T14:48:26.161840+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:48:26.161840+0000","last_undegraded":"2024-09-16T14:48:26.161840+0000","last_fullsized":"2024-09-16T14:48:26.161840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.436189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.156275+0000","last_change":"2024-09-16T14:48:26.156486+0000","last_active":"2024-09-16T14:48:26.156275+0000","last_peered":"2024-09-16T14:48:26.156275+0000","last_clean":"2024-09-16T14:48:26.156275+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:48:26.156275+0000","last_undegraded":"2024-09-16T14:48:26.156275+0000","last_fullsized":"2024-09-16T14:48:26.156275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.015966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.161822+0000","last_change":"2024-09-16T14:48:26.161822+0000","last_active":"2024-09-16T14:48:26.161822+0000","last_peered":"2024-09-16T14:48:26.161822+0000","last_clean":"2024-09-16T14:48:26.161822+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:48:26.161822+0000","last_undegraded":"2024-09-16T14:48:26.161822+0000","last_fullsized":"2024-09-16T14:48:26.161822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:48.998147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.161039+0000","last_change":"2024-09-16T14:48:26.159919+0000","last_active":"2024-09-16T14:48:26.161039+0000","last_peered":"2024-09-16T14:48:26.161039+0000","last_clean":"2024-09-16T14:48:26.161039+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:48:26.161039+0000","last_undegraded":"2024-09-16T14:48:26.161039+0000","last_fullsized":"2024-09-16T14:48:26.161039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:25.005776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.156257+0000","last_change":"2024-09-16T14:48:26.156468+0000","last_active":"2024-09-16T14:48:26.156257+0000","last_peered":"2024-09-16T14:48:26.156257+0000","last_clean":"2024-09-16T14:48:26.156257+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:48:26.156257+0000","last_undegraded":"2024-09-16T14:48:26.156257+0000","last_fullsized":"2024-09-16T14:48:26.156257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:25.252442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.568135+0000","last_change":"2024-09-16T14:48:26.568303+0000","last_active":"2024-09-16T14:48:26.568135+0000","last_peered":"2024-09-16T14:48:26.568135+0000","last_clean":"2024-09-16T14:48:26.568135+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:48:26.568135+0000","last_undegraded":"2024-09-16T14:48:26.568135+0000","last_fullsized":"2024-09-16T14:48:26.568135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:35.481460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.159458+0000","last_change":"2024-09-16T14:48:26.159458+0000","last_active":"2024-09-16T14:48:26.159458+0000","last_peered":"2024-09-16T14:48:26.159458+0000","last_clean":"2024-09-16T14:48:26.159458+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:48:26.159458+0000","last_undegraded":"2024-09-16T14:48:26.159458+0000","last_fullsized":"2024-09-16T14:48:26.159458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:33.086957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"20'137","reported_seq":186,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.112198+0000","last_change":"2024-09-16T14:48:22.359960+0000","last_active":"2024-09-16T14:48:26.112198+0000","last_peered":"2024-09-16T14:48:26.112198+0000","last_clean":"2024-09-16T14:48:26.112198+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:48:26.112198+0000","last_undegraded":"2024-09-16T14:48:26.112198+0000","last_fullsized":"2024-09-16T14:48:26.112198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:48:21.118926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:37.878583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411331,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":757760,"data_stored":615921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":10,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":86037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":6,"up_from":18,"seq":77309411333,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":1,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":87012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":0,"up_from":18,"seq":77309411331,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":5,"up_from":18,"seq":77309411332,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":87012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":9,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":8,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[2,3,4,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]}]},{"osd":7,"up_from":17,"seq":73014444036,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":757760,"data_stored":615921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.251}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]}]},{"osd":4,"up_from":17,"seq":73014444035,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]}]},{"osd":3,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":79321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[2,4,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]},{"osd":2,"up_from":17,"seq":73014444036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:48:28.789 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:48:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:48:29.075 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:48:29.084 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-16T14:48:28.030237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":183,"num_write_kb":2940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":328860,"kb_used_data":3672,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1132133220,"statfs":{"total":1159641169920,"available":1159304417280,"internally_reserved":0,"allocated":3760128,"data_stored":2093182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.875778"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.563974+0000","last_change":"2024-09-16T14:48:26.563974+0000","last_active":"2024-09-16T14:48:26.563974+0000","last_peered":"2024-09-16T14:48:26.563974+0000","last_clean":"2024-09-16T14:48:26.563974+0000","last_became_active":"2024-09-16T14:48:24.165741+0000","last_became_peered":"2024-09-16T14:48:24.165741+0000","last_unstale":"2024-09-16T14:48:26.563974+0000","last_undegraded":"2024-09-16T14:48:26.563974+0000","last_fullsized":"2024-09-16T14:48:26.563974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:47.376916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000285531,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.161840+0000","last_change":"2024-09-16T14:48:26.161840+0000","last_active":"2024-09-16T14:48:26.161840+0000","last_peered":"2024-09-16T14:48:26.161840+0000","last_clean":"2024-09-16T14:48:26.161840+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:48:26.161840+0000","last_undegraded":"2024-09-16T14:48:26.161840+0000","last_fullsized":"2024-09-16T14:48:26.161840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.436189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.156275+0000","last_change":"2024-09-16T14:48:26.156486+0000","last_active":"2024-09-16T14:48:26.156275+0000","last_peered":"2024-09-16T14:48:26.156275+0000","last_clean":"2024-09-16T14:48:26.156275+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:48:26.156275+0000","last_undegraded":"2024-09-16T14:48:26.156275+0000","last_fullsized":"2024-09-16T14:48:26.156275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.015966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.161822+0000","last_change":"2024-09-16T14:48:26.161822+0000","last_active":"2024-09-16T14:48:26.161822+0000","last_peered":"2024-09-16T14:48:26.161822+0000","last_clean":"2024-09-16T14:48:26.161822+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:48:26.161822+0000","last_undegraded":"2024-09-16T14:48:26.161822+0000","last_fullsized":"2024-09-16T14:48:26.161822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:48.998147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.161039+0000","last_change":"2024-09-16T14:48:26.159919+0000","last_active":"2024-09-16T14:48:26.161039+0000","last_peered":"2024-09-16T14:48:26.161039+0000","last_clean":"2024-09-16T14:48:26.161039+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:48:26.161039+0000","last_undegraded":"2024-09-16T14:48:26.161039+0000","last_fullsized":"2024-09-16T14:48:26.161039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:25.005776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.156257+0000","last_change":"2024-09-16T14:48:26.156468+0000","last_active":"2024-09-16T14:48:26.156257+0000","last_peered":"2024-09-16T14:48:26.156257+0000","last_clean":"2024-09-16T14:48:26.156257+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:48:26.156257+0000","last_undegraded":"2024-09-16T14:48:26.156257+0000","last_fullsized":"2024-09-16T14:48:26.156257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:25.252442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.568135+0000","last_change":"2024-09-16T14:48:26.568303+0000","last_active":"2024-09-16T14:48:26.568135+0000","last_peered":"2024-09-16T14:48:26.568135+0000","last_clean":"2024-09-16T14:48:26.568135+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:48:26.568135+0000","last_undegraded":"2024-09-16T14:48:26.568135+0000","last_fullsized":"2024-09-16T14:48:26.568135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:35.481460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.159458+0000","last_change":"2024-09-16T14:48:26.159458+0000","last_active":"2024-09-16T14:48:26.159458+0000","last_peered":"2024-09-16T14:48:26.159458+0000","last_clean":"2024-09-16T14:48:26.159458+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:48:26.159458+0000","last_undegraded":"2024-09-16T14:48:26.159458+0000","last_fullsized":"2024-09-16T14:48:26.159458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:33.086957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"20'137","reported_seq":186,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.112198+0000","last_change":"2024-09-16T14:48:22.359960+0000","last_active":"2024-09-16T14:48:26.112198+0000","last_peered":"2024-09-16T14:48:26.112198+0000","last_clean":"2024-09-16T14:48:26.112198+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:48:26.112198+0000","last_undegraded":"2024-09-16T14:48:26.112198+0000","last_fullsized":"2024-09-16T14:48:26.112198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:48:21.118926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:37.878583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411331,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":757760,"data_stored":615921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":10,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":86037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":6,"up_from":18,"seq":77309411333,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":1,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":87012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":0,"up_from":18,"seq":77309411331,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":5,"up_from":18,"seq":77309411332,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":87012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":9,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":8,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[2,3,4,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]}]},{"osd":7,"up_from":17,"seq":73014444036,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":757760,"data_stored":615921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.251}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]}]},{"osd":4,"up_from":17,"seq":73014444035,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]}]},{"osd":3,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":79321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[2,4,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]},{"osd":2,"up_from":17,"seq":73014444036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:48:29.086 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T14:48:29.086 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T14:48:29.086 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T14:48:29.086 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T14:48:29.403 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:48:29.403 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T14:48:29.415 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T14:48:29.415 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-16T14:48:29.427 INFO:tasks.thrashosds:config is {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-16T14:48:29.427 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'ae7f37330c1a72493684eb1fcaed6f2a01195bb6'}} 2024-09-16T14:48:29.427 INFO:tasks.thrashosds:config is {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-16T14:48:29.427 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-16T14:48:29.726 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-16T14:48:29.727 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T14:48:29.727 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:48:30.010 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:48:30.010 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:48:30.022 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-16T14:48:29.843719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":183,"num_write_kb":2940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":328860,"kb_used_data":3672,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1132133220,"statfs":{"total":1159641169920,"available":1159304417280,"internally_reserved":0,"allocated":3760128,"data_stored":2093182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.689260"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.563974+0000","last_change":"2024-09-16T14:48:26.563974+0000","last_active":"2024-09-16T14:48:26.563974+0000","last_peered":"2024-09-16T14:48:26.563974+0000","last_clean":"2024-09-16T14:48:26.563974+0000","last_became_active":"2024-09-16T14:48:24.165741+0000","last_became_peered":"2024-09-16T14:48:24.165741+0000","last_unstale":"2024-09-16T14:48:26.563974+0000","last_undegraded":"2024-09-16T14:48:26.563974+0000","last_fullsized":"2024-09-16T14:48:26.563974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:47.376916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000285531,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.161840+0000","last_change":"2024-09-16T14:48:26.161840+0000","last_active":"2024-09-16T14:48:26.161840+0000","last_peered":"2024-09-16T14:48:26.161840+0000","last_clean":"2024-09-16T14:48:26.161840+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:48:26.161840+0000","last_undegraded":"2024-09-16T14:48:26.161840+0000","last_fullsized":"2024-09-16T14:48:26.161840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.436189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.156275+0000","last_change":"2024-09-16T14:48:26.156486+0000","last_active":"2024-09-16T14:48:26.156275+0000","last_peered":"2024-09-16T14:48:26.156275+0000","last_clean":"2024-09-16T14:48:26.156275+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:48:26.156275+0000","last_undegraded":"2024-09-16T14:48:26.156275+0000","last_fullsized":"2024-09-16T14:48:26.156275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.015966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.161822+0000","last_change":"2024-09-16T14:48:26.161822+0000","last_active":"2024-09-16T14:48:26.161822+0000","last_peered":"2024-09-16T14:48:26.161822+0000","last_clean":"2024-09-16T14:48:26.161822+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:48:26.161822+0000","last_undegraded":"2024-09-16T14:48:26.161822+0000","last_fullsized":"2024-09-16T14:48:26.161822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:48.998147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.161039+0000","last_change":"2024-09-16T14:48:26.159919+0000","last_active":"2024-09-16T14:48:26.161039+0000","last_peered":"2024-09-16T14:48:26.161039+0000","last_clean":"2024-09-16T14:48:26.161039+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:48:26.161039+0000","last_undegraded":"2024-09-16T14:48:26.161039+0000","last_fullsized":"2024-09-16T14:48:26.161039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:25.005776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.156257+0000","last_change":"2024-09-16T14:48:26.156468+0000","last_active":"2024-09-16T14:48:26.156257+0000","last_peered":"2024-09-16T14:48:26.156257+0000","last_clean":"2024-09-16T14:48:26.156257+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:48:26.156257+0000","last_undegraded":"2024-09-16T14:48:26.156257+0000","last_fullsized":"2024-09-16T14:48:26.156257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:25.252442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.568135+0000","last_change":"2024-09-16T14:48:26.568303+0000","last_active":"2024-09-16T14:48:26.568135+0000","last_peered":"2024-09-16T14:48:26.568135+0000","last_clean":"2024-09-16T14:48:26.568135+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:48:26.568135+0000","last_undegraded":"2024-09-16T14:48:26.568135+0000","last_fullsized":"2024-09-16T14:48:26.568135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:35.481460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.159458+0000","last_change":"2024-09-16T14:48:26.159458+0000","last_active":"2024-09-16T14:48:26.159458+0000","last_peered":"2024-09-16T14:48:26.159458+0000","last_clean":"2024-09-16T14:48:26.159458+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:48:26.159458+0000","last_undegraded":"2024-09-16T14:48:26.159458+0000","last_fullsized":"2024-09-16T14:48:26.159458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:33.086957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"20'137","reported_seq":186,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.112198+0000","last_change":"2024-09-16T14:48:22.359960+0000","last_active":"2024-09-16T14:48:26.112198+0000","last_peered":"2024-09-16T14:48:26.112198+0000","last_clean":"2024-09-16T14:48:26.112198+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:48:26.112198+0000","last_undegraded":"2024-09-16T14:48:26.112198+0000","last_fullsized":"2024-09-16T14:48:26.112198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:48:21.118926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:37.878583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411331,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":757760,"data_stored":615921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":10,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":86037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":6,"up_from":18,"seq":77309411333,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":1,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":87012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":0,"up_from":18,"seq":77309411331,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":5,"up_from":18,"seq":77309411332,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":87012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":9,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":8,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[2,3,4,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]}]},{"osd":7,"up_from":17,"seq":73014444036,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":757760,"data_stored":615921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.251}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]}]},{"osd":4,"up_from":17,"seq":73014444035,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]}]},{"osd":3,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":79321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[2,4,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]},{"osd":2,"up_from":17,"seq":73014444036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:48:30.024 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:48:30.308 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:48:30.308 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:48:30.320 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-16T14:48:30.030671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":183,"num_write_kb":2940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":328860,"kb_used_data":3672,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1132133220,"statfs":{"total":1159641169920,"available":1159304417280,"internally_reserved":0,"allocated":3760128,"data_stored":2093182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.876212"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.563974+0000","last_change":"2024-09-16T14:48:26.563974+0000","last_active":"2024-09-16T14:48:26.563974+0000","last_peered":"2024-09-16T14:48:26.563974+0000","last_clean":"2024-09-16T14:48:26.563974+0000","last_became_active":"2024-09-16T14:48:24.165741+0000","last_became_peered":"2024-09-16T14:48:24.165741+0000","last_unstale":"2024-09-16T14:48:26.563974+0000","last_undegraded":"2024-09-16T14:48:26.563974+0000","last_fullsized":"2024-09-16T14:48:26.563974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:47.376916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000285531,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.161840+0000","last_change":"2024-09-16T14:48:26.161840+0000","last_active":"2024-09-16T14:48:26.161840+0000","last_peered":"2024-09-16T14:48:26.161840+0000","last_clean":"2024-09-16T14:48:26.161840+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:48:26.161840+0000","last_undegraded":"2024-09-16T14:48:26.161840+0000","last_fullsized":"2024-09-16T14:48:26.161840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.436189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.156275+0000","last_change":"2024-09-16T14:48:26.156486+0000","last_active":"2024-09-16T14:48:26.156275+0000","last_peered":"2024-09-16T14:48:26.156275+0000","last_clean":"2024-09-16T14:48:26.156275+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:48:26.156275+0000","last_undegraded":"2024-09-16T14:48:26.156275+0000","last_fullsized":"2024-09-16T14:48:26.156275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.015966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.161822+0000","last_change":"2024-09-16T14:48:26.161822+0000","last_active":"2024-09-16T14:48:26.161822+0000","last_peered":"2024-09-16T14:48:26.161822+0000","last_clean":"2024-09-16T14:48:26.161822+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:48:26.161822+0000","last_undegraded":"2024-09-16T14:48:26.161822+0000","last_fullsized":"2024-09-16T14:48:26.161822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:48.998147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.161039+0000","last_change":"2024-09-16T14:48:26.159919+0000","last_active":"2024-09-16T14:48:26.161039+0000","last_peered":"2024-09-16T14:48:26.161039+0000","last_clean":"2024-09-16T14:48:26.161039+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:48:26.161039+0000","last_undegraded":"2024-09-16T14:48:26.161039+0000","last_fullsized":"2024-09-16T14:48:26.161039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:25.005776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.156257+0000","last_change":"2024-09-16T14:48:26.156468+0000","last_active":"2024-09-16T14:48:26.156257+0000","last_peered":"2024-09-16T14:48:26.156257+0000","last_clean":"2024-09-16T14:48:26.156257+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:48:26.156257+0000","last_undegraded":"2024-09-16T14:48:26.156257+0000","last_fullsized":"2024-09-16T14:48:26.156257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:25.252442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.568135+0000","last_change":"2024-09-16T14:48:26.568303+0000","last_active":"2024-09-16T14:48:26.568135+0000","last_peered":"2024-09-16T14:48:26.568135+0000","last_clean":"2024-09-16T14:48:26.568135+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:48:26.568135+0000","last_undegraded":"2024-09-16T14:48:26.568135+0000","last_fullsized":"2024-09-16T14:48:26.568135+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:35.481460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.159458+0000","last_change":"2024-09-16T14:48:26.159458+0000","last_active":"2024-09-16T14:48:26.159458+0000","last_peered":"2024-09-16T14:48:26.159458+0000","last_clean":"2024-09-16T14:48:26.159458+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:48:26.159458+0000","last_undegraded":"2024-09-16T14:48:26.159458+0000","last_fullsized":"2024-09-16T14:48:26.159458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:33.086957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"20'137","reported_seq":186,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T14:48:26.112198+0000","last_change":"2024-09-16T14:48:22.359960+0000","last_active":"2024-09-16T14:48:26.112198+0000","last_peered":"2024-09-16T14:48:26.112198+0000","last_clean":"2024-09-16T14:48:26.112198+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:48:26.112198+0000","last_undegraded":"2024-09-16T14:48:26.112198+0000","last_fullsized":"2024-09-16T14:48:26.112198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:48:21.118926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:37.878583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411331,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":757760,"data_stored":615921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":10,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":86037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":6,"up_from":18,"seq":77309411333,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":1,"up_from":18,"seq":77309411333,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":87012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":0,"up_from":18,"seq":77309411331,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]}]},{"osd":5,"up_from":18,"seq":77309411332,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":87012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":9,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":8,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[2,3,4,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]}]},{"osd":7,"up_from":17,"seq":73014444036,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":757760,"data_stored":615921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.251}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]}]},{"osd":4,"up_from":17,"seq":73014444035,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]}]},{"osd":3,"up_from":17,"seq":73014444036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":79321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[2,4,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]}]},{"osd":2,"up_from":17,"seq":73014444036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":86993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:48:30.322 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T14:48:30.322 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-16T14:48:30.605 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 24 2024-09-16T14:48:30.605 INFO:teuthology.orchestra.run.smithi040.stdout:fsid aaaa1c0f-de8b-48c8-9412-815c8b70b7b8 2024-09-16T14:48:30.605 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-09-16T14:48:10.063465+0000 2024-09-16T14:48:30.605 INFO:teuthology.orchestra.run.smithi040.stdout:modified 2024-09-16T14:48:29.841476+0000 2024-09-16T14:48:30.605 INFO:teuthology.orchestra.run.smithi040.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T14:48:30.605 INFO:teuthology.orchestra.run.smithi040.stdout:crush_version 4 2024-09-16T14:48:30.605 INFO:teuthology.orchestra.run.smithi040.stdout:full_ratio 0.95 2024-09-16T14:48:30.605 INFO:teuthology.orchestra.run.smithi040.stdout:backfillfull_ratio 0.9 2024-09-16T14:48:30.605 INFO:teuthology.orchestra.run.smithi040.stdout:nearfull_ratio 0.85 2024-09-16T14:48:30.605 INFO:teuthology.orchestra.run.smithi040.stdout:require_min_compat_client luminous 2024-09-16T14:48:30.606 INFO:teuthology.orchestra.run.smithi040.stdout:min_compat_client jewel 2024-09-16T14:48:30.606 INFO:teuthology.orchestra.run.smithi040.stdout:require_osd_release reef 2024-09-16T14:48:30.606 INFO:teuthology.orchestra.run.smithi040.stdout:stretch_mode_enabled false 2024-09-16T14:48:30.606 INFO:teuthology.orchestra.run.smithi040.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 11.76 2024-09-16T14:48:30.606 INFO:teuthology.orchestra.run.smithi040.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 23 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 3.00 2024-09-16T14:48:30.606 INFO:teuthology.orchestra.run.smithi040.stdout:max_osd 12 2024-09-16T14:48:30.606 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6800/725262,v1:172.21.15.40:6801/725262] [v2:172.21.15.40:6802/725262,v1:172.21.15.40:6803/725262] exists,up ffcfdeb7-eb75-4b3a-bc2a-f1834271654c 2024-09-16T14:48:30.606 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6819/501335559,v1:172.21.15.40:6821/501335559] [v2:172.21.15.40:6823/501335559,v1:172.21.15.40:6825/501335559] exists,up 6fb8c93f-5e3c-492f-bd0c-6d97764467f5 2024-09-16T14:48:30.606 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6808/1490162929,v1:172.21.15.40:6809/1490162929] [v2:172.21.15.40:6810/1490162929,v1:172.21.15.40:6811/1490162929] exists,up 500457a9-0d59-451c-b2ff-28aa363056bf 2024-09-16T14:48:30.606 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6816/682463486,v1:172.21.15.40:6817/682463486] [v2:172.21.15.40:6818/682463486,v1:172.21.15.40:6820/682463486] exists,up f6e48d98-5d87-4cd5-8bd8-6e4c9dc56a39 2024-09-16T14:48:30.606 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6816/1550824417,v1:172.21.15.111:6818/1550824417] [v2:172.21.15.111:6819/1550824417,v1:172.21.15.111:6821/1550824417] exists,up 474dd40c-2b73-44ea-b6a9-48605b78125c 2024-09-16T14:48:30.606 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6800/3785556374,v1:172.21.15.111:6801/3785556374] [v2:172.21.15.111:6802/3785556374,v1:172.21.15.111:6803/3785556374] exists,up 479bfe9b-00f6-4763-9740-9c9fd11e2587 2024-09-16T14:48:30.606 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6817/370046418,v1:172.21.15.111:6820/370046418] [v2:172.21.15.111:6822/370046418,v1:172.21.15.111:6825/370046418] exists,up 6ac36041-045e-4813-ad35-ca4e224811fa 2024-09-16T14:48:30.607 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6808/3591930331,v1:172.21.15.111:6809/3591930331] [v2:172.21.15.111:6810/3591930331,v1:172.21.15.111:6811/3591930331] exists,up 129006cf-b82b-4e11-a007-63deb355d007 2024-09-16T14:48:30.607 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6816/2473986485,v1:172.21.15.114:6817/2473986485] [v2:172.21.15.114:6818/2473986485,v1:172.21.15.114:6819/2473986485] exists,up 27e746ee-f67a-4a73-8976-86d012469657 2024-09-16T14:48:30.607 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6800/2290892207,v1:172.21.15.114:6801/2290892207] [v2:172.21.15.114:6802/2290892207,v1:172.21.15.114:6803/2290892207] exists,up 90975d4f-24b3-4c84-916e-3138bc52a076 2024-09-16T14:48:30.607 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6808/1795470140,v1:172.21.15.114:6809/1795470140] [v2:172.21.15.114:6810/1795470140,v1:172.21.15.114:6811/1795470140] exists,up 89c8157b-52ce-4c1c-9df7-e214df30a827 2024-09-16T14:48:30.607 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6823/1823608150,v1:172.21.15.114:6825/1823608150] [v2:172.21.15.114:6826/1823608150,v1:172.21.15.114:6827/1823608150] exists,up 0dfbd657-c9f0-42f9-9b43-323c45d89c1f 2024-09-16T14:48:30.617 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6800/725262,v1:172.21.15.40:6801/725262] [v2:172.21.15.40:6802/725262,v1:172.21.15.40:6803/725262] exists,up ffcfdeb7-eb75-4b3a-bc2a-f1834271654c', 'osd.1 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6819/501335559,v1:172.21.15.40:6821/501335559] [v2:172.21.15.40:6823/501335559,v1:172.21.15.40:6825/501335559] exists,up 6fb8c93f-5e3c-492f-bd0c-6d97764467f5', 'osd.2 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6808/1490162929,v1:172.21.15.40:6809/1490162929] [v2:172.21.15.40:6810/1490162929,v1:172.21.15.40:6811/1490162929] exists,up 500457a9-0d59-451c-b2ff-28aa363056bf', 'osd.3 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.40:6816/682463486,v1:172.21.15.40:6817/682463486] [v2:172.21.15.40:6818/682463486,v1:172.21.15.40:6820/682463486] exists,up f6e48d98-5d87-4cd5-8bd8-6e4c9dc56a39', 'osd.4 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6816/1550824417,v1:172.21.15.111:6818/1550824417] [v2:172.21.15.111:6819/1550824417,v1:172.21.15.111:6821/1550824417] exists,up 474dd40c-2b73-44ea-b6a9-48605b78125c', 'osd.5 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6800/3785556374,v1:172.21.15.111:6801/3785556374] [v2:172.21.15.111:6802/3785556374,v1:172.21.15.111:6803/3785556374] exists,up 479bfe9b-00f6-4763-9740-9c9fd11e2587', 'osd.6 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6817/370046418,v1:172.21.15.111:6820/370046418] [v2:172.21.15.111:6822/370046418,v1:172.21.15.111:6825/370046418] exists,up 6ac36041-045e-4813-ad35-ca4e224811fa', 'osd.7 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6808/3591930331,v1:172.21.15.111:6809/3591930331] [v2:172.21.15.111:6810/3591930331,v1:172.21.15.111:6811/3591930331] exists,up 129006cf-b82b-4e11-a007-63deb355d007', 'osd.8 up in weight 1 up_from 17 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6816/2473986485,v1:172.21.15.114:6817/2473986485] [v2:172.21.15.114:6818/2473986485,v1:172.21.15.114:6819/2473986485] exists,up 27e746ee-f67a-4a73-8976-86d012469657', 'osd.9 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6800/2290892207,v1:172.21.15.114:6801/2290892207] [v2:172.21.15.114:6802/2290892207,v1:172.21.15.114:6803/2290892207] exists,up 90975d4f-24b3-4c84-916e-3138bc52a076', 'osd.10 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6808/1795470140,v1:172.21.15.114:6809/1795470140] [v2:172.21.15.114:6810/1795470140,v1:172.21.15.114:6811/1795470140] exists,up 89c8157b-52ce-4c1c-9df7-e214df30a827', 'osd.11 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6823/1823608150,v1:172.21.15.114:6825/1823608150] [v2:172.21.15.114:6826/1823608150,v1:172.21.15.114:6827/1823608150] exists,up 0dfbd657-c9f0-42f9-9b43-323c45d89c1f'] 2024-09-16T14:48:30.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-16T14:48:30.719 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:48:30.719 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "mon.a", 2024-09-16T14:48:30.719 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster": "ceph", 2024-09-16T14:48:30.719 INFO:teuthology.orchestra.run.smithi040.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-16T14:48:30.719 INFO:teuthology.orchestra.run.smithi040.stdout: "admin_socket_mode": "", 2024-09-16T14:48:30.720 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-16T14:48:30.720 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_client_required": "cephx, none", 2024-09-16T14:48:30.720 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_cluster_required": "cephx", 2024-09-16T14:48:30.720 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_debug": "true", 2024-09-16T14:48:30.720 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-16T14:48:30.720 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-16T14:48:30.720 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_service_required": "cephx", 2024-09-16T14:48:30.720 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-16T14:48:30.720 INFO:teuthology.orchestra.run.smithi040.stdout: "auth_supported": "cephx", 2024-09-16T14:48:30.720 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_aio": "true", 2024-09-16T14:48:30.720 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-16T14:48:30.720 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_aio_poll_ms": "250", 2024-09-16T14:48:30.721 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_aio_reap_max": "16", 2024-09-16T14:48:30.721 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_async_discard": "false", 2024-09-16T14:48:30.721 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_block_size": "4096", 2024-09-16T14:48:30.721 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_debug_aio": "false", 2024-09-16T14:48:30.721 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-16T14:48:30.721 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-16T14:48:30.721 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_debug_inflight_ios": "false", 2024-09-16T14:48:30.721 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_enable_discard": "false", 2024-09-16T14:48:30.721 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_flock_retry": "3", 2024-09-16T14:48:30.721 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-16T14:48:30.721 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_inject_crash": "0", 2024-09-16T14:48:30.721 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-16T14:48:30.721 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_ioring": "false", 2024-09-16T14:48:30.721 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_ioring_hipri": "false", 2024-09-16T14:48:30.722 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-16T14:48:30.722 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-16T14:48:30.722 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-16T14:48:30.722 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-16T14:48:30.722 INFO:teuthology.orchestra.run.smithi040.stdout: "bdev_type": "", 2024-09-16T14:48:30.722 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_alloc_size": "1048576", 2024-09-16T14:48:30.722 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_allocator": "hybrid", 2024-09-16T14:48:30.722 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_buffered_io": "true", 2024-09-16T14:48:30.722 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_check_for_zeros": "false", 2024-09-16T14:48:30.722 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-16T14:48:30.722 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-16T14:48:30.722 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_compact_log_sync": "false", 2024-09-16T14:48:30.722 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-16T14:48:30.723 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-16T14:48:30.723 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-16T14:48:30.723 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-16T14:48:30.723 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_max_log_runway": "4194304", 2024-09-16T14:48:30.723 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_max_prefetch": "1048576", 2024-09-16T14:48:30.723 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_min_flush_size": "524288", 2024-09-16T14:48:30.723 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_min_log_runway": "1048576", 2024-09-16T14:48:30.723 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_replay_recovery": "false", 2024-09-16T14:48:30.723 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-16T14:48:30.723 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-16T14:48:30.723 INFO:teuthology.orchestra.run.smithi040.stdout: "bluefs_sync_write": "false", 2024-09-16T14:48:30.723 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-16T14:48:30.723 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-16T14:48:30.724 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-16T14:48:30.724 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_allocation_from_file": "true", 2024-09-16T14:48:30.724 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_allocator": "hybrid", 2024-09-16T14:48:30.724 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-16T14:48:30.724 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-16T14:48:30.724 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-16T14:48:30.724 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-16T14:48:30.724 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-16T14:48:30.724 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-16T14:48:30.724 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-16T14:48:30.724 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_block_create": "true", 2024-09-16T14:48:30.724 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_block_db_create": "false", 2024-09-16T14:48:30.724 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_block_db_path": "", 2024-09-16T14:48:30.725 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_block_db_size": "0", 2024-09-16T14:48:30.725 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_block_path": "", 2024-09-16T14:48:30.725 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_block_preallocate_file": "false", 2024-09-16T14:48:30.725 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_block_size": "107374182400", 2024-09-16T14:48:30.725 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_block_wal_create": "false", 2024-09-16T14:48:30.725 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_block_wal_path": "", 2024-09-16T14:48:30.725 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_block_wal_size": "100663296", 2024-09-16T14:48:30.725 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_bluefs": "true", 2024-09-16T14:48:30.726 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-16T14:48:30.726 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-16T14:48:30.726 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-16T14:48:30.726 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-16T14:48:30.726 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T14:48:30.726 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T14:48:30.726 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-16T14:48:30.726 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T14:48:30.726 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_autotune": "true", 2024-09-16T14:48:30.726 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-16T14:48:30.726 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-16T14:48:30.726 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-16T14:48:30.726 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-16T14:48:30.726 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_size": "0", 2024-09-16T14:48:30.727 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-16T14:48:30.727 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-16T14:48:30.727 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-16T14:48:30.727 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-16T14:48:30.727 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cache_type": "2q", 2024-09-16T14:48:30.727 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-16T14:48:30.727 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_clone_cow": "true", 2024-09-16T14:48:30.727 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-16T14:48:30.727 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-16T14:48:30.727 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-16T14:48:30.727 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-16T14:48:30.727 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-16T14:48:30.727 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-16T14:48:30.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-16T14:48:30.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_compression_mode": "none", 2024-09-16T14:48:30.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-16T14:48:30.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_csum_type": "crc32c", 2024-09-16T14:48:30.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-16T14:48:30.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_freelist": "false", 2024-09-16T14:48:30.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-16T14:48:30.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-16T14:48:30.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-16T14:48:30.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-16T14:48:30.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-16T14:48:30.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-16T14:48:30.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-16T14:48:30.728 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-16T14:48:30.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-16T14:48:30.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-16T14:48:30.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-16T14:48:30.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-16T14:48:30.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-16T14:48:30.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-16T14:48:30.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_small_allocations": "0", 2024-09-16T14:48:30.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-16T14:48:30.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_default_buffered_read": "true", 2024-09-16T14:48:30.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_default_buffered_write": "false", 2024-09-16T14:48:30.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-16T14:48:30.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-16T14:48:30.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-16T14:48:30.729 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-16T14:48:30.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-16T14:48:30.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-16T14:48:30.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-16T14:48:30.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-16T14:48:30.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fail_eio": "false", 2024-09-16T14:48:30.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-16T14:48:30.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-16T14:48:30.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-16T14:48:30.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-16T14:48:30.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-16T14:48:30.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-16T14:48:30.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_on_mount": "false", 2024-09-16T14:48:30.730 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-16T14:48:30.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_on_umount": "false", 2024-09-16T14:48:30.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-16T14:48:30.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-16T14:48:30.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-16T14:48:30.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-16T14:48:30.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-16T14:48:30.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-16T14:48:30.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-16T14:48:30.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-16T14:48:30.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_ignore_data_csum": "false", 2024-09-16T14:48:30.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-16T14:48:30.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-16T14:48:30.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-16T14:48:30.731 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-16T14:48:30.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_log_op_age": "5.000000", 2024-09-16T14:48:30.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_max_alloc_size": "0", 2024-09-16T14:48:30.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_max_blob_size": "0", 2024-09-16T14:48:30.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-16T14:48:30.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-16T14:48:30.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-16T14:48:30.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_max_deferred_txc": "32", 2024-09-16T14:48:30.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_min_alloc_size": "0", 2024-09-16T14:48:30.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-16T14:48:30.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-16T14:48:30.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_nid_prealloc": "1024", 2024-09-16T14:48:30.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-16T14:48:30.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-16T14:48:30.732 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-16T14:48:30.733 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-16T14:48:30.733 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_retry_disk_reads": "3", 2024-09-16T14:48:30.733 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_rocksdb_cf": "true", 2024-09-16T14:48:30.733 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-16T14:48:30.733 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-16T14:48:30.733 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-16T14:48:30.733 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-16T14:48:30.733 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-16T14:48:30.733 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-16T14:48:30.733 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_spdk_mem": "512", 2024-09-16T14:48:30.733 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-16T14:48:30.733 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-16T14:48:30.733 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-16T14:48:30.734 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-16T14:48:30.734 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-16T14:48:30.734 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-16T14:48:30.734 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-16T14:48:30.734 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_tracing": "false", 2024-09-16T14:48:30.734 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-16T14:48:30.734 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-16T14:48:30.734 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-16T14:48:30.734 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-16T14:48:30.734 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-16T14:48:30.734 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-16T14:48:30.734 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-16T14:48:30.734 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-16T14:48:30.735 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-16T14:48:30.735 INFO:teuthology.orchestra.run.smithi040.stdout: "bluestore_zero_block_detection": "false", 2024-09-16T14:48:30.735 INFO:teuthology.orchestra.run.smithi040.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-16T14:48:30.735 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-16T14:48:30.735 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-16T14:48:30.735 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-16T14:48:30.735 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-16T14:48:30.735 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-16T14:48:30.735 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-16T14:48:30.735 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-16T14:48:30.736 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-16T14:48:30.736 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-16T14:48:30.736 INFO:teuthology.orchestra.run.smithi040.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-16T14:48:30.736 INFO:teuthology.orchestra.run.smithi040.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-16T14:48:30.736 INFO:teuthology.orchestra.run.smithi040.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-16T14:48:30.736 INFO:teuthology.orchestra.run.smithi040.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-16T14:48:30.736 INFO:teuthology.orchestra.run.smithi040.stdout: "cephx_cluster_require_signatures": "false", 2024-09-16T14:48:30.736 INFO:teuthology.orchestra.run.smithi040.stdout: "cephx_cluster_require_version": "2", 2024-09-16T14:48:30.736 INFO:teuthology.orchestra.run.smithi040.stdout: "cephx_require_signatures": "false", 2024-09-16T14:48:30.736 INFO:teuthology.orchestra.run.smithi040.stdout: "cephx_require_version": "2", 2024-09-16T14:48:30.736 INFO:teuthology.orchestra.run.smithi040.stdout: "cephx_service_require_signatures": "false", 2024-09-16T14:48:30.736 INFO:teuthology.orchestra.run.smithi040.stdout: "cephx_service_require_version": "2", 2024-09-16T14:48:30.737 INFO:teuthology.orchestra.run.smithi040.stdout: "cephx_sign_messages": "true", 2024-09-16T14:48:30.737 INFO:teuthology.orchestra.run.smithi040.stdout: "chdir": "", 2024-09-16T14:48:30.737 INFO:teuthology.orchestra.run.smithi040.stdout: "client_acl_type": "", 2024-09-16T14:48:30.737 INFO:teuthology.orchestra.run.smithi040.stdout: "client_asio_thread_count": "2", 2024-09-16T14:48:30.737 INFO:teuthology.orchestra.run.smithi040.stdout: "client_cache_mid": "0.750000", 2024-09-16T14:48:30.737 INFO:teuthology.orchestra.run.smithi040.stdout: "client_cache_size": "16384", 2024-09-16T14:48:30.737 INFO:teuthology.orchestra.run.smithi040.stdout: "client_caps_release_delay": "5", 2024-09-16T14:48:30.737 INFO:teuthology.orchestra.run.smithi040.stdout: "client_check_pool_perm": "true", 2024-09-16T14:48:30.737 INFO:teuthology.orchestra.run.smithi040.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-16T14:48:30.737 INFO:teuthology.orchestra.run.smithi040.stdout: "client_debug_force_sync_read": "false", 2024-09-16T14:48:30.737 INFO:teuthology.orchestra.run.smithi040.stdout: "client_debug_getattr_caps": "false", 2024-09-16T14:48:30.737 INFO:teuthology.orchestra.run.smithi040.stdout: "client_debug_inject_features": "", 2024-09-16T14:48:30.737 INFO:teuthology.orchestra.run.smithi040.stdout: "client_debug_inject_tick_delay": "0", 2024-09-16T14:48:30.737 INFO:teuthology.orchestra.run.smithi040.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-16T14:48:30.738 INFO:teuthology.orchestra.run.smithi040.stdout: "client_die_on_failed_remount": "false", 2024-09-16T14:48:30.738 INFO:teuthology.orchestra.run.smithi040.stdout: "client_dirsize_rbytes": "true", 2024-09-16T14:48:30.738 INFO:teuthology.orchestra.run.smithi040.stdout: "client_force_lazyio": "false", 2024-09-16T14:48:30.738 INFO:teuthology.orchestra.run.smithi040.stdout: "client_fs": "", 2024-09-16T14:48:30.738 INFO:teuthology.orchestra.run.smithi040.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-16T14:48:30.738 INFO:teuthology.orchestra.run.smithi040.stdout: "client_inject_release_failure": "false", 2024-09-16T14:48:30.738 INFO:teuthology.orchestra.run.smithi040.stdout: "client_max_inline_size": "4096", 2024-09-16T14:48:30.738 INFO:teuthology.orchestra.run.smithi040.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-16T14:48:30.738 INFO:teuthology.orchestra.run.smithi040.stdout: "client_mds_namespace": "", 2024-09-16T14:48:30.738 INFO:teuthology.orchestra.run.smithi040.stdout: "client_metadata": "", 2024-09-16T14:48:30.738 INFO:teuthology.orchestra.run.smithi040.stdout: "client_mount_gid": "-1", 2024-09-16T14:48:30.738 INFO:teuthology.orchestra.run.smithi040.stdout: "client_mount_timeout": "300", 2024-09-16T14:48:30.738 INFO:teuthology.orchestra.run.smithi040.stdout: "client_mount_uid": "-1", 2024-09-16T14:48:30.739 INFO:teuthology.orchestra.run.smithi040.stdout: "client_mountpoint": "/", 2024-09-16T14:48:30.739 INFO:teuthology.orchestra.run.smithi040.stdout: "client_notify_timeout": "10", 2024-09-16T14:48:30.739 INFO:teuthology.orchestra.run.smithi040.stdout: "client_oc": "true", 2024-09-16T14:48:30.739 INFO:teuthology.orchestra.run.smithi040.stdout: "client_oc_max_dirty": "104857600", 2024-09-16T14:48:30.739 INFO:teuthology.orchestra.run.smithi040.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-16T14:48:30.739 INFO:teuthology.orchestra.run.smithi040.stdout: "client_oc_max_objects": "1000", 2024-09-16T14:48:30.739 INFO:teuthology.orchestra.run.smithi040.stdout: "client_oc_size": "209715200", 2024-09-16T14:48:30.739 INFO:teuthology.orchestra.run.smithi040.stdout: "client_oc_target_dirty": "8388608", 2024-09-16T14:48:30.739 INFO:teuthology.orchestra.run.smithi040.stdout: "client_permissions": "true", 2024-09-16T14:48:30.739 INFO:teuthology.orchestra.run.smithi040.stdout: "client_quota": "true", 2024-09-16T14:48:30.739 INFO:teuthology.orchestra.run.smithi040.stdout: "client_quota_df": "true", 2024-09-16T14:48:30.739 INFO:teuthology.orchestra.run.smithi040.stdout: "client_readahead_max_bytes": "0", 2024-09-16T14:48:30.739 INFO:teuthology.orchestra.run.smithi040.stdout: "client_readahead_max_periods": "4", 2024-09-16T14:48:30.739 INFO:teuthology.orchestra.run.smithi040.stdout: "client_readahead_min": "131072", 2024-09-16T14:48:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: "client_reconnect_stale": "false", 2024-09-16T14:48:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: "client_shutdown_timeout": "30", 2024-09-16T14:48:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: "client_snapdir": ".snap", 2024-09-16T14:48:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: "client_tick_interval": "1", 2024-09-16T14:48:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: "client_trace": "", 2024-09-16T14:48:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: "client_try_dentry_invalidate": "false", 2024-09-16T14:48:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: "client_use_faked_inos": "false", 2024-09-16T14:48:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: "client_use_random_mds": "false", 2024-09-16T14:48:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: "clog_to_graylog": "false", 2024-09-16T14:48:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-16T14:48:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: "clog_to_graylog_port": "12201", 2024-09-16T14:48:30.740 INFO:teuthology.orchestra.run.smithi040.stdout: "clog_to_monitors": "default=true", 2024-09-16T14:48:30.741 INFO:teuthology.orchestra.run.smithi040.stdout: "clog_to_syslog": "false", 2024-09-16T14:48:30.741 INFO:teuthology.orchestra.run.smithi040.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-16T14:48:30.741 INFO:teuthology.orchestra.run.smithi040.stdout: "clog_to_syslog_level": "info", 2024-09-16T14:48:30.741 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_addr": "-", 2024-09-16T14:48:30.741 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_network": "", 2024-09-16T14:48:30.741 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_network_interface": "", 2024-09-16T14:48:30.741 INFO:teuthology.orchestra.run.smithi040.stdout: "compressor_zlib_isal": "false", 2024-09-16T14:48:30.741 INFO:teuthology.orchestra.run.smithi040.stdout: "compressor_zlib_level": "5", 2024-09-16T14:48:30.741 INFO:teuthology.orchestra.run.smithi040.stdout: "compressor_zlib_winsize": "-15", 2024-09-16T14:48:30.741 INFO:teuthology.orchestra.run.smithi040.stdout: "compressor_zstd_level": "1", 2024-09-16T14:48:30.741 INFO:teuthology.orchestra.run.smithi040.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-16T14:48:30.741 INFO:teuthology.orchestra.run.smithi040.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-16T14:48:30.741 INFO:teuthology.orchestra.run.smithi040.stdout: "crimson_alien_op_num_threads": "6", 2024-09-16T14:48:30.742 INFO:teuthology.orchestra.run.smithi040.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-16T14:48:30.742 INFO:teuthology.orchestra.run.smithi040.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-16T14:48:30.742 INFO:teuthology.orchestra.run.smithi040.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-16T14:48:30.742 INFO:teuthology.orchestra.run.smithi040.stdout: "crimson_seastar_smp": "1", 2024-09-16T14:48:30.742 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "", 2024-09-16T14:48:30.742 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location_hook": "", 2024-09-16T14:48:30.742 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location_hook_timeout": "10", 2024-09-16T14:48:30.742 INFO:teuthology.orchestra.run.smithi040.stdout: "daemonize": "false", 2024-09-16T14:48:30.742 INFO:teuthology.orchestra.run.smithi040.stdout: "daos_pool": "tank", 2024-09-16T14:48:30.742 INFO:teuthology.orchestra.run.smithi040.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-16T14:48:30.742 INFO:teuthology.orchestra.run.smithi040.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-16T14:48:30.742 INFO:teuthology.orchestra.run.smithi040.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-16T14:48:30.742 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_alienstore": "0/5", 2024-09-16T14:48:30.742 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_allow_any_pool_priority": "false", 2024-09-16T14:48:30.743 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_asok": "1/5", 2024-09-16T14:48:30.743 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_asok_assert_abort": "false", 2024-09-16T14:48:30.743 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_asserts_on_shutdown": "true", 2024-09-16T14:48:30.743 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_auth": "20/20", 2024-09-16T14:48:30.743 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_bdev": "1/3", 2024-09-16T14:48:30.743 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_bluefs": "1/5", 2024-09-16T14:48:30.743 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_bluestore": "1/5", 2024-09-16T14:48:30.743 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_buffer": "0/1", 2024-09-16T14:48:30.743 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_ceph_exporter": "1/5", 2024-09-16T14:48:30.743 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_cephfs_mirror": "0/5", 2024-09-16T14:48:30.743 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_cephsqlite": "0/5", 2024-09-16T14:48:30.743 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_client": "0/5", 2024-09-16T14:48:30.743 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_compressor": "1/5", 2024-09-16T14:48:30.743 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_context": "0/1", 2024-09-16T14:48:30.744 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_crush": "1/1", 2024-09-16T14:48:30.744 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_crypto": "1/5", 2024-09-16T14:48:30.744 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_cyanstore": "0/5", 2024-09-16T14:48:30.744 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_deliberately_leak_memory": "false", 2024-09-16T14:48:30.744 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_disable_randomized_ping": "false", 2024-09-16T14:48:30.744 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_dpdk": "1/5", 2024-09-16T14:48:30.744 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_eventtrace": "1/5", 2024-09-16T14:48:30.744 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_filer": "0/1", 2024-09-16T14:48:30.744 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_filestore": "1/3", 2024-09-16T14:48:30.744 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_finisher": "1/1", 2024-09-16T14:48:30.744 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_fuse": "1/5", 2024-09-16T14:48:30.744 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_heartbeat_testing_span": "0", 2024-09-16T14:48:30.744 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_heartbeatmap": "1/5", 2024-09-16T14:48:30.744 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-16T14:48:30.745 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_javaclient": "1/5", 2024-09-16T14:48:30.745 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_journal": "1/3", 2024-09-16T14:48:30.745 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_journaler": "0/5", 2024-09-16T14:48:30.745 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_kstore": "1/5", 2024-09-16T14:48:30.745 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_leveldb": "4/5", 2024-09-16T14:48:30.745 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_lockdep": "0/1", 2024-09-16T14:48:30.745 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_mclock": "1/5", 2024-09-16T14:48:30.745 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_mds": "1/5", 2024-09-16T14:48:30.745 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_mds_balancer": "1/5", 2024-09-16T14:48:30.745 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_mds_locker": "1/5", 2024-09-16T14:48:30.745 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_mds_log": "1/5", 2024-09-16T14:48:30.745 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_mds_log_expire": "1/5", 2024-09-16T14:48:30.746 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_mds_migrator": "1/5", 2024-09-16T14:48:30.746 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_memstore": "1/5", 2024-09-16T14:48:30.746 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_mgr": "2/5", 2024-09-16T14:48:30.746 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_mgrc": "1/5", 2024-09-16T14:48:30.746 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_mon": "20/20", 2024-09-16T14:48:30.746 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_monc": "0/10", 2024-09-16T14:48:30.746 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_ms": "1/1", 2024-09-16T14:48:30.746 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_none": "0/5", 2024-09-16T14:48:30.746 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_objclass": "0/5", 2024-09-16T14:48:30.746 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_objectcacher": "0/5", 2024-09-16T14:48:30.746 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_objecter": "0/1", 2024-09-16T14:48:30.746 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_optracker": "0/5", 2024-09-16T14:48:30.746 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_osd": "1/5", 2024-09-16T14:48:30.747 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_paxos": "20/20", 2024-09-16T14:48:30.747 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_perfcounter": "1/5", 2024-09-16T14:48:30.747 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_prioritycache": "1/5", 2024-09-16T14:48:30.747 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_rados": "0/5", 2024-09-16T14:48:30.747 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_rbd": "0/5", 2024-09-16T14:48:30.747 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_rbd_mirror": "0/5", 2024-09-16T14:48:30.747 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_rbd_pwl": "0/5", 2024-09-16T14:48:30.747 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_rbd_replay": "0/5", 2024-09-16T14:48:30.747 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_refs": "0/0", 2024-09-16T14:48:30.747 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_reserver": "1/1", 2024-09-16T14:48:30.747 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_rgw": "1/5", 2024-09-16T14:48:30.747 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_rgw_access": "1/5", 2024-09-16T14:48:30.747 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_rgw_datacache": "1/5", 2024-09-16T14:48:30.747 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_rgw_dbstore": "1/5", 2024-09-16T14:48:30.748 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_rgw_flight": "1/5", 2024-09-16T14:48:30.748 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_rgw_sync": "1/5", 2024-09-16T14:48:30.748 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_rocksdb": "4/5", 2024-09-16T14:48:30.748 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_seastore": "0/5", 2024-09-16T14:48:30.748 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_seastore_backref": "0/5", 2024-09-16T14:48:30.748 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_seastore_cache": "0/5", 2024-09-16T14:48:30.748 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_seastore_cleaner": "0/5", 2024-09-16T14:48:30.748 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_seastore_device": "0/5", 2024-09-16T14:48:30.748 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_seastore_epm": "0/5", 2024-09-16T14:48:30.748 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-16T14:48:30.748 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_seastore_journal": "0/5", 2024-09-16T14:48:30.748 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_seastore_lba": "0/5", 2024-09-16T14:48:30.748 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_seastore_odata": "0/5", 2024-09-16T14:48:30.748 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_seastore_omap": "0/5", 2024-09-16T14:48:30.749 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_seastore_onode": "0/5", 2024-09-16T14:48:30.749 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_seastore_t": "0/5", 2024-09-16T14:48:30.749 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_seastore_tm": "0/5", 2024-09-16T14:48:30.749 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_striper": "0/1", 2024-09-16T14:48:30.749 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_test": "0/5", 2024-09-16T14:48:30.749 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_throttle": "1/1", 2024-09-16T14:48:30.749 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_timer": "0/1", 2024-09-16T14:48:30.749 INFO:teuthology.orchestra.run.smithi040.stdout: "debug_tp": "0/5", 2024-09-16T14:48:30.749 INFO:teuthology.orchestra.run.smithi040.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-16T14:48:30.749 INFO:teuthology.orchestra.run.smithi040.stdout: "device_failure_prediction_mode": "none", 2024-09-16T14:48:30.749 INFO:teuthology.orchestra.run.smithi040.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-16T14:48:30.749 INFO:teuthology.orchestra.run.smithi040.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-16T14:48:30.749 INFO:teuthology.orchestra.run.smithi040.stdout: "err_to_graylog": "false", 2024-09-16T14:48:30.750 INFO:teuthology.orchestra.run.smithi040.stdout: "err_to_journald": "false", 2024-09-16T14:48:30.750 INFO:teuthology.orchestra.run.smithi040.stdout: "err_to_stderr": "true", 2024-09-16T14:48:30.750 INFO:teuthology.orchestra.run.smithi040.stdout: "err_to_syslog": "false", 2024-09-16T14:48:30.750 INFO:teuthology.orchestra.run.smithi040.stdout: "event_tracing": "false", 2024-09-16T14:48:30.750 INFO:teuthology.orchestra.run.smithi040.stdout: "exporter_addr": "0.0.0.0", 2024-09-16T14:48:30.750 INFO:teuthology.orchestra.run.smithi040.stdout: "exporter_http_port": "9926", 2024-09-16T14:48:30.750 INFO:teuthology.orchestra.run.smithi040.stdout: "exporter_prio_limit": "5", 2024-09-16T14:48:30.750 INFO:teuthology.orchestra.run.smithi040.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-16T14:48:30.750 INFO:teuthology.orchestra.run.smithi040.stdout: "exporter_sort_metrics": "true", 2024-09-16T14:48:30.750 INFO:teuthology.orchestra.run.smithi040.stdout: "exporter_stats_period": "5", 2024-09-16T14:48:30.750 INFO:teuthology.orchestra.run.smithi040.stdout: "fake_statfs_for_testing": "0", 2024-09-16T14:48:30.750 INFO:teuthology.orchestra.run.smithi040.stdout: "fatal_signal_handlers": "true", 2024-09-16T14:48:30.750 INFO:teuthology.orchestra.run.smithi040.stdout: "filer_max_purge_ops": "10", 2024-09-16T14:48:30.751 INFO:teuthology.orchestra.run.smithi040.stdout: "filer_max_truncate_ops": "128", 2024-09-16T14:48:30.751 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_apply_finisher_threads": "1", 2024-09-16T14:48:30.751 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_blackhole": "false", 2024-09-16T14:48:30.751 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_btrfs_clone_range": "true", 2024-09-16T14:48:30.751 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_btrfs_snap": "true", 2024-09-16T14:48:30.751 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_caller_concurrency": "10", 2024-09-16T14:48:30.751 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_collect_device_partition_information": "true", 2024-09-16T14:48:30.751 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_commit_timeout": "600.000000", 2024-09-16T14:48:30.751 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_debug_inject_read_err": "false", 2024-09-16T14:48:30.751 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_debug_omap_check": "false", 2024-09-16T14:48:30.751 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-16T14:48:30.751 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_debug_verify_split": "false", 2024-09-16T14:48:30.751 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_dump_file": "", 2024-09-16T14:48:30.751 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-16T14:48:30.752 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-16T14:48:30.752 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_fadvise": "true", 2024-09-16T14:48:30.752 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_fail_eio": "true", 2024-09-16T14:48:30.752 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_fd_cache_shards": "16", 2024-09-16T14:48:30.752 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_fd_cache_size": "128", 2024-09-16T14:48:30.752 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_fiemap": "false", 2024-09-16T14:48:30.752 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_fiemap_threshold": "4096", 2024-09-16T14:48:30.752 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-16T14:48:30.752 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-16T14:48:30.752 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_inject_stall": "0", 2024-09-16T14:48:30.752 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_journal_parallel": "false", 2024-09-16T14:48:30.752 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_journal_trailing": "false", 2024-09-16T14:48:30.752 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_journal_writeahead": "false", 2024-09-16T14:48:30.752 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_kill_at": "0", 2024-09-16T14:48:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-16T14:48:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-16T14:48:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-16T14:48:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-16T14:48:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-16T14:48:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_max_inline_xattrs": "0", 2024-09-16T14:48:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-16T14:48:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-16T14:48:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-16T14:48:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-16T14:48:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_max_xattr_value_size": "0", 2024-09-16T14:48:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-16T14:48:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-16T14:48:30.753 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-16T14:48:30.754 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_merge_threshold": "-10", 2024-09-16T14:48:30.754 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-16T14:48:30.754 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_odsync_write": "false", 2024-09-16T14:48:30.754 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_omap_backend": "rocksdb", 2024-09-16T14:48:30.754 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_omap_backend_path": "", 2024-09-16T14:48:30.754 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-16T14:48:30.754 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-16T14:48:30.754 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-16T14:48:30.754 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_op_thread_timeout": "60", 2024-09-16T14:48:30.754 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_op_threads": "2", 2024-09-16T14:48:30.754 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_punch_hole": "false", 2024-09-16T14:48:30.755 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-16T14:48:30.755 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-16T14:48:30.755 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-16T14:48:30.755 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-16T14:48:30.755 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-16T14:48:30.755 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-16T14:48:30.755 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-16T14:48:30.755 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-16T14:48:30.755 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-16T14:48:30.755 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_queue_max_ops": "50", 2024-09-16T14:48:30.755 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-16T14:48:30.756 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_seek_data_hole": "false", 2024-09-16T14:48:30.756 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_sloppy_crc": "false", 2024-09-16T14:48:30.756 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-16T14:48:30.756 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_splice": "false", 2024-09-16T14:48:30.756 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_split_multiple": "2", 2024-09-16T14:48:30.756 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_split_rand_factor": "20", 2024-09-16T14:48:30.756 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_update_to": "1000", 2024-09-16T14:48:30.756 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-16T14:48:30.756 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-16T14:48:30.756 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-16T14:48:30.756 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-16T14:48:30.756 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-16T14:48:30.756 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-16T14:48:30.757 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_wbthrottle_enable": "true", 2024-09-16T14:48:30.757 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-16T14:48:30.757 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-16T14:48:30.757 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-16T14:48:30.757 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-16T14:48:30.757 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-16T14:48:30.757 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-16T14:48:30.757 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_xfs_extsize": "false", 2024-09-16T14:48:30.757 INFO:teuthology.orchestra.run.smithi040.stdout: "filestore_zfs_snap": "false", 2024-09-16T14:48:30.757 INFO:teuthology.orchestra.run.smithi040.stdout: "fio_dir": "/tmp/fio", 2024-09-16T14:48:30.757 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "aaaa1c0f-de8b-48c8-9412-815c8b70b7b8", 2024-09-16T14:48:30.757 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_allow_other": "true", 2024-09-16T14:48:30.757 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_atomic_o_trunc": "true", 2024-09-16T14:48:30.757 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_big_writes": "true", 2024-09-16T14:48:30.758 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_debug": "false", 2024-09-16T14:48:30.758 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_default_permissions": "false", 2024-09-16T14:48:30.758 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_disable_pagecache": "false", 2024-09-16T14:48:30.758 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_max_write": "0", 2024-09-16T14:48:30.758 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_multithreaded": "true", 2024-09-16T14:48:30.758 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_require_active_mds": "true", 2024-09-16T14:48:30.758 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_set_user_groups": "true", 2024-09-16T14:48:30.758 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_splice_move": "true", 2024-09-16T14:48:30.758 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_splice_read": "true", 2024-09-16T14:48:30.758 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_splice_write": "true", 2024-09-16T14:48:30.758 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-16T14:48:30.758 INFO:teuthology.orchestra.run.smithi040.stdout: "fuse_use_invalidate_cb": "true", 2024-09-16T14:48:30.758 INFO:teuthology.orchestra.run.smithi040.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-16T14:48:30.759 INFO:teuthology.orchestra.run.smithi040.stdout: "gss_target_name": "ceph", 2024-09-16T14:48:30.759 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_file": "", 2024-09-16T14:48:30.759 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_inject_failure": "0", 2024-09-16T14:48:30.759 INFO:teuthology.orchestra.run.smithi040.stdout: "heartbeat_interval": "5", 2024-09-16T14:48:30.759 INFO:teuthology.orchestra.run.smithi040.stdout: "host": "smithi040", 2024-09-16T14:48:30.759 INFO:teuthology.orchestra.run.smithi040.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-16T14:48:30.759 INFO:teuthology.orchestra.run.smithi040.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-16T14:48:30.759 INFO:teuthology.orchestra.run.smithi040.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-16T14:48:30.759 INFO:teuthology.orchestra.run.smithi040.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-16T14:48:30.759 INFO:teuthology.orchestra.run.smithi040.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-16T14:48:30.759 INFO:teuthology.orchestra.run.smithi040.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-16T14:48:30.759 INFO:teuthology.orchestra.run.smithi040.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-16T14:48:30.759 INFO:teuthology.orchestra.run.smithi040.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-16T14:48:30.760 INFO:teuthology.orchestra.run.smithi040.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-16T14:48:30.760 INFO:teuthology.orchestra.run.smithi040.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-16T14:48:30.760 INFO:teuthology.orchestra.run.smithi040.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-16T14:48:30.760 INFO:teuthology.orchestra.run.smithi040.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-16T14:48:30.760 INFO:teuthology.orchestra.run.smithi040.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-16T14:48:30.760 INFO:teuthology.orchestra.run.smithi040.stdout: "inject_early_sigterm": "false", 2024-09-16T14:48:30.760 INFO:teuthology.orchestra.run.smithi040.stdout: "jaeger_agent_port": "6799", 2024-09-16T14:48:30.760 INFO:teuthology.orchestra.run.smithi040.stdout: "jaeger_tracing_enable": "false", 2024-09-16T14:48:30.760 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_aio": "true", 2024-09-16T14:48:30.760 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_align_min_size": "65536", 2024-09-16T14:48:30.760 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_block_align": "true", 2024-09-16T14:48:30.760 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_block_size": "4096", 2024-09-16T14:48:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_dio": "true", 2024-09-16T14:48:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_discard": "false", 2024-09-16T14:48:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_force_aio": "false", 2024-09-16T14:48:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_ignore_corruption": "false", 2024-09-16T14:48:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_max_write_bytes": "10485760", 2024-09-16T14:48:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_max_write_entries": "100", 2024-09-16T14:48:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_replay_from": "0", 2024-09-16T14:48:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-16T14:48:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-16T14:48:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-16T14:48:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-16T14:48:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_write_header_frequency": "0", 2024-09-16T14:48:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: "journal_zero_on_create": "false", 2024-09-16T14:48:30.761 INFO:teuthology.orchestra.run.smithi040.stdout: "journaler_prefetch_periods": "10", 2024-09-16T14:48:30.762 INFO:teuthology.orchestra.run.smithi040.stdout: "journaler_prezero_periods": "5", 2024-09-16T14:48:30.762 INFO:teuthology.orchestra.run.smithi040.stdout: "journaler_write_head_interval": "15", 2024-09-16T14:48:30.762 INFO:teuthology.orchestra.run.smithi040.stdout: "key": "", 2024-09-16T14:48:30.762 INFO:teuthology.orchestra.run.smithi040.stdout: "keyfile": "", 2024-09-16T14:48:30.762 INFO:teuthology.orchestra.run.smithi040.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-16T14:48:30.762 INFO:teuthology.orchestra.run.smithi040.stdout: "kstore_backend": "rocksdb", 2024-09-16T14:48:30.762 INFO:teuthology.orchestra.run.smithi040.stdout: "kstore_default_stripe_size": "65536", 2024-09-16T14:48:30.762 INFO:teuthology.orchestra.run.smithi040.stdout: "kstore_fsck_on_mount": "false", 2024-09-16T14:48:30.762 INFO:teuthology.orchestra.run.smithi040.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-16T14:48:30.762 INFO:teuthology.orchestra.run.smithi040.stdout: "kstore_max_bytes": "67108864", 2024-09-16T14:48:30.762 INFO:teuthology.orchestra.run.smithi040.stdout: "kstore_max_ops": "512", 2024-09-16T14:48:30.762 INFO:teuthology.orchestra.run.smithi040.stdout: "kstore_nid_prealloc": "1024", 2024-09-16T14:48:30.762 INFO:teuthology.orchestra.run.smithi040.stdout: "kstore_onode_map_size": "1024", 2024-09-16T14:48:30.763 INFO:teuthology.orchestra.run.smithi040.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-16T14:48:30.763 INFO:teuthology.orchestra.run.smithi040.stdout: "kstore_sync_submit_transaction": "false", 2024-09-16T14:48:30.763 INFO:teuthology.orchestra.run.smithi040.stdout: "kstore_sync_transaction": "false", 2024-09-16T14:48:30.763 INFO:teuthology.orchestra.run.smithi040.stdout: "leveldb_block_size": "65536", 2024-09-16T14:48:30.763 INFO:teuthology.orchestra.run.smithi040.stdout: "leveldb_bloom_size": "0", 2024-09-16T14:48:30.763 INFO:teuthology.orchestra.run.smithi040.stdout: "leveldb_cache_size": "536870912", 2024-09-16T14:48:30.763 INFO:teuthology.orchestra.run.smithi040.stdout: "leveldb_compact_on_mount": "false", 2024-09-16T14:48:30.763 INFO:teuthology.orchestra.run.smithi040.stdout: "leveldb_compression": "false", 2024-09-16T14:48:30.763 INFO:teuthology.orchestra.run.smithi040.stdout: "leveldb_log": "", 2024-09-16T14:48:30.763 INFO:teuthology.orchestra.run.smithi040.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-16T14:48:30.763 INFO:teuthology.orchestra.run.smithi040.stdout: "leveldb_max_open_files": "0", 2024-09-16T14:48:30.763 INFO:teuthology.orchestra.run.smithi040.stdout: "leveldb_paranoid": "false", 2024-09-16T14:48:30.763 INFO:teuthology.orchestra.run.smithi040.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-16T14:48:30.763 INFO:teuthology.orchestra.run.smithi040.stdout: "librados_thread_count": "2", 2024-09-16T14:48:30.764 INFO:teuthology.orchestra.run.smithi040.stdout: "lockdep": "false", 2024-09-16T14:48:30.764 INFO:teuthology.orchestra.run.smithi040.stdout: "lockdep_force_backtrace": "false", 2024-09-16T14:48:30.764 INFO:teuthology.orchestra.run.smithi040.stdout: "log_coarse_timestamps": "true", 2024-09-16T14:48:30.764 INFO:teuthology.orchestra.run.smithi040.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-16T14:48:30.764 INFO:teuthology.orchestra.run.smithi040.stdout: "log_flush_on_exit": "false", 2024-09-16T14:48:30.764 INFO:teuthology.orchestra.run.smithi040.stdout: "log_graylog_host": "127.0.0.1", 2024-09-16T14:48:30.764 INFO:teuthology.orchestra.run.smithi040.stdout: "log_graylog_port": "12201", 2024-09-16T14:48:30.764 INFO:teuthology.orchestra.run.smithi040.stdout: "log_max_new": "1000", 2024-09-16T14:48:30.764 INFO:teuthology.orchestra.run.smithi040.stdout: "log_max_recent": "10000", 2024-09-16T14:48:30.764 INFO:teuthology.orchestra.run.smithi040.stdout: "log_stderr_prefix": "", 2024-09-16T14:48:30.764 INFO:teuthology.orchestra.run.smithi040.stdout: "log_stop_at_utilization": "0.970000", 2024-09-16T14:48:30.764 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_file": "true", 2024-09-16T14:48:30.764 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_graylog": "false", 2024-09-16T14:48:30.765 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_journald": "false", 2024-09-16T14:48:30.765 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_stderr": "false", 2024-09-16T14:48:30.765 INFO:teuthology.orchestra.run.smithi040.stdout: "log_to_syslog": "false", 2024-09-16T14:48:30.765 INFO:teuthology.orchestra.run.smithi040.stdout: "max_rotating_auth_attempts": "10", 2024-09-16T14:48:30.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-16T14:48:30.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_action_on_write_error": "1", 2024-09-16T14:48:30.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_alternate_name_max": "8192", 2024-09-16T14:48:30.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_asio_thread_count": "2", 2024-09-16T14:48:30.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_export_pin": "true", 2024-09-16T14:48:30.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_fragment_dirs": "true", 2024-09-16T14:48:30.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-16T14:48:30.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_fragment_interval": "5", 2024-09-16T14:48:30.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-16T14:48:30.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-16T14:48:30.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_interval": "10", 2024-09-16T14:48:30.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_max": "-1", 2024-09-16T14:48:30.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_max_until": "-1", 2024-09-16T14:48:30.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_merge_size": "50", 2024-09-16T14:48:30.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_midchunk": "0.300000", 2024-09-16T14:48:30.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-16T14:48:30.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_min_start": "0.200000", 2024-09-16T14:48:30.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_minchunk": "0.001000", 2024-09-16T14:48:30.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_mode": "0", 2024-09-16T14:48:30.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_need_max": "1.200000", 2024-09-16T14:48:30.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_need_min": "0.800000", 2024-09-16T14:48:30.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-16T14:48:30.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-16T14:48:30.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_split_bits": "3", 2024-09-16T14:48:30.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-16T14:48:30.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_split_size": "10000", 2024-09-16T14:48:30.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-16T14:48:30.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_target_decay": "10.000000", 2024-09-16T14:48:30.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-16T14:48:30.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_beacon_grace": "15.000000", 2024-09-16T14:48:30.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_beacon_interval": "4.000000", 2024-09-16T14:48:30.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-16T14:48:30.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-16T14:48:30.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache_mid": "0.700000", 2024-09-16T14:48:30.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache_release_free_interval": "10", 2024-09-16T14:48:30.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache_reservation": "0.050000", 2024-09-16T14:48:30.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-16T14:48:30.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache_trim_interval": "1", 2024-09-16T14:48:30.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cache_trim_threshold": "262144", 2024-09-16T14:48:30.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-16T14:48:30.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-16T14:48:30.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-16T14:48:30.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_client_prealloc_inos": "1000", 2024-09-16T14:48:30.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-16T14:48:30.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_connect_bootstrapping": "false", 2024-09-16T14:48:30.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_damage_table_max_entries": "10000", 2024-09-16T14:48:30.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-16T14:48:30.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_debug_auth_pins": "false", 2024-09-16T14:48:30.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_debug_frag": "false", 2024-09-16T14:48:30.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_debug_scatterstat": "false", 2024-09-16T14:48:30.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_debug_subtrees": "false", 2024-09-16T14:48:30.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_decay_halflife": "5.000000", 2024-09-16T14:48:30.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_default_dir_hash": "2", 2024-09-16T14:48:30.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_defer_session_stale": "true", 2024-09-16T14:48:30.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_deny_all_reconnect": "false", 2024-09-16T14:48:30.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_dir_keys_per_op": "16384", 2024-09-16T14:48:30.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_dir_max_commit_size": "10", 2024-09-16T14:48:30.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_dir_max_entries": "0", 2024-09-16T14:48:30.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_dir_prefetch": "true", 2024-09-16T14:48:30.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-16T14:48:30.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-16T14:48:30.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_dump_cache_on_map": "false", 2024-09-16T14:48:30.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-16T14:48:30.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-16T14:48:30.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_early_reply": "true", 2024-09-16T14:48:30.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_enable_op_tracker": "true", 2024-09-16T14:48:30.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_enforce_unique_name": "true", 2024-09-16T14:48:30.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-16T14:48:30.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-16T14:48:30.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_export_ephemeral_random": "true", 2024-09-16T14:48:30.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-16T14:48:30.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-16T14:48:30.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-16T14:48:30.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-16T14:48:30.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-16T14:48:30.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-16T14:48:30.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-16T14:48:30.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-16T14:48:30.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_health_summarize_threshold": "10", 2024-09-16T14:48:30.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-16T14:48:30.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-16T14:48:30.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_inject_health_dummy": "false", 2024-09-16T14:48:30.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-16T14:48:30.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_inject_migrator_session_race": "false", 2024-09-16T14:48:30.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-16T14:48:30.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-16T14:48:30.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-16T14:48:30.771 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_join_fs": "", 2024-09-16T14:48:30.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_journal_format": "1", 2024-09-16T14:48:30.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-16T14:48:30.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_kill_create_at": "0", 2024-09-16T14:48:30.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_kill_export_at": "0", 2024-09-16T14:48:30.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_kill_import_at": "0", 2024-09-16T14:48:30.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_kill_journal_at": "0", 2024-09-16T14:48:30.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_kill_journal_expire_at": "0", 2024-09-16T14:48:30.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_kill_journal_replay_at": "0", 2024-09-16T14:48:30.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_kill_link_at": "0", 2024-09-16T14:48:30.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_kill_mdstable_at": "0", 2024-09-16T14:48:30.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_kill_openc_at": "0", 2024-09-16T14:48:30.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_kill_rename_at": "0", 2024-09-16T14:48:30.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log_events_per_segment": "1024", 2024-09-16T14:48:30.772 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log_max_events": "-1", 2024-09-16T14:48:30.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log_max_segments": "128", 2024-09-16T14:48:30.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log_pause": "false", 2024-09-16T14:48:30.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log_segment_size": "0", 2024-09-16T14:48:30.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-16T14:48:30.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_log_warn_factor": "2.000000", 2024-09-16T14:48:30.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_max_caps_per_client": "1048576", 2024-09-16T14:48:30.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_max_completed_flushes": "100000", 2024-09-16T14:48:30.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_max_completed_requests": "100000", 2024-09-16T14:48:30.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_max_export_size": "20971520", 2024-09-16T14:48:30.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_max_file_recover": "32", 2024-09-16T14:48:30.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_max_purge_files": "64", 2024-09-16T14:48:30.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_max_purge_ops": "8192", 2024-09-16T14:48:30.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-16T14:48:30.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-16T14:48:30.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_max_snaps_per_dir": "100", 2024-09-16T14:48:30.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_metrics_update_interval": "2", 2024-09-16T14:48:30.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_min_caps_per_client": "100", 2024-09-16T14:48:30.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_min_caps_working_set": "10000", 2024-09-16T14:48:30.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-16T14:48:30.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_numa_node": "-1", 2024-09-16T14:48:30.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-16T14:48:30.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_op_complaint_time": "30.000000", 2024-09-16T14:48:30.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_op_history_duration": "600", 2024-09-16T14:48:30.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_op_history_size": "20", 2024-09-16T14:48:30.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_op_history_slow_op_size": "20", 2024-09-16T14:48:30.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-16T14:48:30.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_op_log_threshold": "5", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_ping_grace": "15", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_ping_interval": "5", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_recall_max_caps": "30000", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_recall_warning_threshold": "262144", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_replay_interval": "1.000000", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_root_ino_gid": "0", 2024-09-16T14:48:30.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_root_ino_uid": "0", 2024-09-16T14:48:30.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-16T14:48:30.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-16T14:48:30.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-16T14:48:30.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-16T14:48:30.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-16T14:48:30.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-16T14:48:30.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-16T14:48:30.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-16T14:48:30.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-16T14:48:30.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-16T14:48:30.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_shutdown_check": "0", 2024-09-16T14:48:30.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_skip_ino": "0", 2024-09-16T14:48:30.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-16T14:48:30.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_snap_max_uid": "4294967294", 2024-09-16T14:48:30.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_snap_min_uid": "0", 2024-09-16T14:48:30.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_snap_rstat": "false", 2024-09-16T14:48:30.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_standby_replay_damaged": "false", 2024-09-16T14:48:30.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_symlink_recovery": "true", 2024-09-16T14:48:30.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-16T14:48:30.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_thrash_exports": "0", 2024-09-16T14:48:30.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_thrash_fragments": "0", 2024-09-16T14:48:30.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_tick_interval": "5.000000", 2024-09-16T14:48:30.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_valgrind_exit": "false", 2024-09-16T14:48:30.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_verify_backtrace": "1", 2024-09-16T14:48:30.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_verify_scatter": "false", 2024-09-16T14:48:30.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-16T14:48:30.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_wipe_sessions": "false", 2024-09-16T14:48:30.778 INFO:teuthology.orchestra.run.smithi040.stdout: "mempool_debug": "false", 2024-09-16T14:48:30.778 INFO:teuthology.orchestra.run.smithi040.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-16T14:48:30.778 INFO:teuthology.orchestra.run.smithi040.stdout: "memstore_device_bytes": "1073741824", 2024-09-16T14:48:30.778 INFO:teuthology.orchestra.run.smithi040.stdout: "memstore_page_set": "false", 2024-09-16T14:48:30.778 INFO:teuthology.orchestra.run.smithi040.stdout: "memstore_page_size": "65536", 2024-09-16T14:48:30.778 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_client_bytes": "134217728", 2024-09-16T14:48:30.778 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_client_messages": "512", 2024-09-16T14:48:30.778 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-16T14:48:30.778 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-16T14:48:30.778 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-16T14:48:30.778 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-16T14:48:30.778 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_disabled_modules": "", 2024-09-16T14:48:30.778 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-16T14:48:30.778 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_max_pg_num_change": "128", 2024-09-16T14:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_mds_bytes": "134217728", 2024-09-16T14:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_mds_messages": "128", 2024-09-16T14:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-16T14:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_mon_bytes": "134217728", 2024-09-16T14:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_mon_messages": "128", 2024-09-16T14:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_osd_bytes": "536870912", 2024-09-16T14:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_osd_messages": "8192", 2024-09-16T14:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_pool": "true", 2024-09-16T14:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-16T14:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_standby_modules": "true", 2024-09-16T14:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_stats_period": "5", 2024-09-16T14:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_stats_threshold": "5", 2024-09-16T14:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_tick_period": "2", 2024-09-16T14:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-16T14:48:30.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-16T14:48:30.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_allow_pool_delete": "true", 2024-09-16T14:48:30.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_allow_pool_size_one": "true", 2024-09-16T14:48:30.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_auth_validate_all_caps": "true", 2024-09-16T14:48:30.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-16T14:48:30.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-16T14:48:30.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_client_bytes": "104857600", 2024-09-16T14:48:30.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_client_directed_command_retry": "5", 2024-09-16T14:48:30.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-16T14:48:30.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-16T14:48:30.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-16T14:48:30.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-16T14:48:30.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_client_hunt_parallel": "3", 2024-09-16T14:48:30.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_client_log_interval": "1.000000", 2024-09-16T14:48:30.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-16T14:48:30.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_client_ping_interval": "10.000000", 2024-09-16T14:48:30.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-16T14:48:30.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-16T14:48:30.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-16T14:48:30.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-16T14:48:30.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_cluster_log_file_level": "debug", 2024-09-16T14:48:30.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_cluster_log_to_file": "true", 2024-09-16T14:48:30.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-16T14:48:30.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-16T14:48:30.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-16T14:48:30.781 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_cluster_log_to_journald": "false", 2024-09-16T14:48:30.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-16T14:48:30.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-16T14:48:30.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-16T14:48:30.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-16T14:48:30.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_compact_on_bootstrap": "false", 2024-09-16T14:48:30.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_compact_on_start": "false", 2024-09-16T14:48:30.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_compact_on_trim": "true", 2024-09-16T14:48:30.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-16T14:48:30.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-16T14:48:30.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-16T14:48:30.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_cpu_threads": "4", 2024-09-16T14:48:30.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_crush_min_required_version": "hammer", 2024-09-16T14:48:30.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_daemon_bytes": "419430400", 2024-09-16T14:48:30.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-16T14:48:30.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_data_avail_crit": "5", 2024-09-16T14:48:30.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_data_avail_warn": "5", 2024-09-16T14:48:30.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_data_size_warn": "16106127360", 2024-09-16T14:48:30.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-16T14:48:30.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-16T14:48:30.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_debug_dump_json": "false", 2024-09-16T14:48:30.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-16T14:48:30.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_debug_dump_transactions": "false", 2024-09-16T14:48:30.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_debug_extra_checks": "false", 2024-09-16T14:48:30.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-16T14:48:30.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-16T14:48:30.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_debug_no_require_quincy": "false", 2024-09-16T14:48:30.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_debug_no_require_reef": "false", 2024-09-16T14:48:30.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-16T14:48:30.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-16T14:48:30.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-16T14:48:30.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_down_mkfs_grace": "120", 2024-09-16T14:48:30.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_election_default_strategy": "1", 2024-09-16T14:48:30.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_election_timeout": "5.000000", 2024-09-16T14:48:30.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-16T14:48:30.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_elector_ping_divisor": "2", 2024-09-16T14:48:30.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-16T14:48:30.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_enable_op_tracker": "true", 2024-09-16T14:48:30.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_fake_pool_delete": "false", 2024-09-16T14:48:30.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_force_quorum_join": "false", 2024-09-16T14:48:30.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_globalid_prealloc": "10000", 2024-09-16T14:48:30.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_health_detail_to_clog": "false", 2024-09-16T14:48:30.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_health_log_update_period": "5", 2024-09-16T14:48:30.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_health_max_detail": "50", 2024-09-16T14:48:30.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_health_to_clog": "true", 2024-09-16T14:48:30.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_health_to_clog_interval": "600", 2024-09-16T14:48:30.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-16T14:48:30.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_host": "172.21.15.40,172.21.15.111,172.21.15.114", 2024-09-16T14:48:30.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_host_override": "", 2024-09-16T14:48:30.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_initial_members": "", 2024-09-16T14:48:30.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-16T14:48:30.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-16T14:48:30.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-16T14:48:30.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-16T14:48:30.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-16T14:48:30.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_lease": "5.000000", 2024-09-16T14:48:30.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-16T14:48:30.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-16T14:48:30.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_log_full_interval": "50", 2024-09-16T14:48:30.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_log_max": "10000", 2024-09-16T14:48:30.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_log_max_summary": "50", 2024-09-16T14:48:30.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-16T14:48:30.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_max_log_epochs": "500", 2024-09-16T14:48:30.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-16T14:48:30.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-16T14:48:30.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_max_osd": "10000", 2024-09-16T14:48:30.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_max_pg_per_osd": "10000", 2024-09-16T14:48:30.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_max_pool_pg_num": "65536", 2024-09-16T14:48:30.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-16T14:48:30.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-16T14:48:30.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_mds_force_trim_to": "0", 2024-09-16T14:48:30.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_mds_skip_sanity": "false", 2024-09-16T14:48:30.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_memory_autotune": "true", 2024-09-16T14:48:30.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_memory_target": "2147483648", 2024-09-16T14:48:30.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_mgr_beacon_grace": "30", 2024-09-16T14:48:30.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-16T14:48:30.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_mgr_digest_period": "5", 2024-09-16T14:48:30.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_mgr_inactive_grace": "60", 2024-09-16T14:48:30.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-16T14:48:30.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-16T14:48:30.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_min_osdmap_epochs": "500", 2024-09-16T14:48:30.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_op_complaint_time": "30", 2024-09-16T14:48:30.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_op_history_duration": "600", 2024-09-16T14:48:30.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_op_history_size": "20", 2024-09-16T14:48:30.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_op_history_slow_op_size": "20", 2024-09-16T14:48:30.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-16T14:48:30.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_op_log_threshold": "5", 2024-09-16T14:48:30.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-16T14:48:30.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-16T14:48:30.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-16T14:48:30.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_auto_mark_in": "false", 2024-09-16T14:48:30.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-16T14:48:30.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-16T14:48:30.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-16T14:48:30.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_cache_size": "500", 2024-09-16T14:48:30.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-16T14:48:30.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-16T14:48:30.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-16T14:48:30.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_down_out_interval": "600", 2024-09-16T14:48:30.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-16T14:48:30.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-16T14:48:30.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_force_trim_to": "0", 2024-09-16T14:48:30.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-16T14:48:30.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-16T14:48:30.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-16T14:48:30.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-16T14:48:30.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-16T14:48:30.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-16T14:48:30.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-16T14:48:30.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-16T14:48:30.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_min_down_reporters": "2", 2024-09-16T14:48:30.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-16T14:48:30.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-16T14:48:30.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-16T14:48:30.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-16T14:48:30.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-16T14:48:30.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-16T14:48:30.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_report_timeout": "900", 2024-09-16T14:48:30.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-16T14:48:30.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-16T14:48:30.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-16T14:48:30.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-16T14:48:30.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-16T14:48:30.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-16T14:48:30.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-16T14:48:30.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-16T14:48:30.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-16T14:48:30.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_pg_stuck_threshold": "60", 2024-09-16T14:48:30.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-16T14:48:30.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-16T14:48:30.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-16T14:48:30.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-16T14:48:30.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-16T14:48:30.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-16T14:48:30.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_probe_timeout": "2.000000", 2024-09-16T14:48:30.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_reweight_max_change": "0.050000", 2024-09-16T14:48:30.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_reweight_max_osds": "4", 2024-09-16T14:48:30.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-16T14:48:30.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-16T14:48:30.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-16T14:48:30.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-16T14:48:30.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-16T14:48:30.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_scrub_interval": "300", 2024-09-16T14:48:30.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_scrub_max_keys": "100", 2024-09-16T14:48:30.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_scrub_timeout": "300", 2024-09-16T14:48:30.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_session_timeout": "300", 2024-09-16T14:48:30.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_smart_report_timeout": "5", 2024-09-16T14:48:30.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_stat_smooth_intervals": "6", 2024-09-16T14:48:30.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-16T14:48:30.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_stretch_pool_min_size": "2", 2024-09-16T14:48:30.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_stretch_pool_size": "4", 2024-09-16T14:48:30.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-16T14:48:30.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-16T14:48:30.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_sync_debug": "false", 2024-09-16T14:48:30.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-16T14:48:30.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-16T14:48:30.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_sync_provider_kill_at": "0", 2024-09-16T14:48:30.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_sync_requester_kill_at": "0", 2024-09-16T14:48:30.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_sync_timeout": "60.000000", 2024-09-16T14:48:30.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_target_pg_per_osd": "100", 2024-09-16T14:48:30.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_tick_interval": "5", 2024-09-16T14:48:30.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_timecheck_interval": "300.000000", 2024-09-16T14:48:30.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-16T14:48:30.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_use_min_delay_socket": "false", 2024-09-16T14:48:30.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_older_version_delay": "604800", 2024-09-16T14:48:30.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-16T14:48:30.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-16T14:48:30.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-16T14:48:30.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-16T14:48:30.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-16T14:48:30.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-16T14:48:30.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-16T14:48:30.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_misplaced": "false", 2024-09-16T14:48:30.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-16T14:48:30.794 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_older_version": "true", 2024-09-16T14:48:30.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-16T14:48:30.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-16T14:48:30.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-16T14:48:30.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-16T14:48:30.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-16T14:48:30.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-16T14:48:30.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-16T14:48:30.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-16T14:48:30.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-16T14:48:30.795 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": "", 2024-09-16T14:48:30.795 INFO:teuthology.orchestra.run.smithi040.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-16T14:48:30.795 INFO:teuthology.orchestra.run.smithi040.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-16T14:48:30.795 INFO:teuthology.orchestra.run.smithi040.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-16T14:48:30.795 INFO:teuthology.orchestra.run.smithi040.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-16T14:48:30.796 INFO:teuthology.orchestra.run.smithi040.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-16T14:48:30.796 INFO:teuthology.orchestra.run.smithi040.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-16T14:48:30.796 INFO:teuthology.orchestra.run.smithi040.stdout: "motr_tracing_enabled": "false", 2024-09-16T14:48:30.796 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_op_threads": "3", 2024-09-16T14:48:30.796 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-16T14:48:30.796 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_cm": "false", 2024-09-16T14:48:30.796 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_device_name": "", 2024-09-16T14:48:30.796 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_dscp": "96", 2024-09-16T14:48:30.796 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-16T14:48:30.796 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-16T14:48:30.796 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_local_gid": "", 2024-09-16T14:48:30.796 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-16T14:48:30.796 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_port_num": "1", 2024-09-16T14:48:30.797 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-16T14:48:30.797 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-16T14:48:30.797 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-16T14:48:30.797 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-16T14:48:30.797 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_sl": "3", 2024-09-16T14:48:30.797 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_support_srq": "true", 2024-09-16T14:48:30.797 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_rdma_type": "ib", 2024-09-16T14:48:30.797 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_async_reap_threshold": "5", 2024-09-16T14:48:30.797 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_bind_before_connect": "false", 2024-09-16T14:48:30.797 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_bind_ipv4": "true", 2024-09-16T14:48:30.797 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_bind_ipv6": "false", 2024-09-16T14:48:30.797 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_bind_msgr1": "true", 2024-09-16T14:48:30.797 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_bind_msgr2": "true", 2024-09-16T14:48:30.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_bind_port_max": "7568", 2024-09-16T14:48:30.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_bind_port_min": "6800", 2024-09-16T14:48:30.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-16T14:48:30.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_bind_retry_count": "3", 2024-09-16T14:48:30.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_bind_retry_delay": "5", 2024-09-16T14:48:30.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_blackhole_client": "false", 2024-09-16T14:48:30.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_blackhole_mds": "false", 2024-09-16T14:48:30.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_blackhole_mgr": "false", 2024-09-16T14:48:30.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_blackhole_mon": "false", 2024-09-16T14:48:30.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_blackhole_osd": "false", 2024-09-16T14:48:30.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_client_mode": "crc secure", 2024-09-16T14:48:30.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_cluster_mode": "crc secure", 2024-09-16T14:48:30.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_cluster_type": "", 2024-09-16T14:48:30.798 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_compress_secure": "false", 2024-09-16T14:48:30.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_connection_idle_timeout": "900", 2024-09-16T14:48:30.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_connection_ready_timeout": "10", 2024-09-16T14:48:30.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_crc_data": "true", 2024-09-16T14:48:30.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_crc_header": "true", 2024-09-16T14:48:30.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_die_on_bad_msg": "false", 2024-09-16T14:48:30.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_die_on_bug": "true", 2024-09-16T14:48:30.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_die_on_old_message": "true", 2024-09-16T14:48:30.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_die_on_skipped_message": "false", 2024-09-16T14:48:30.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-16T14:48:30.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-16T14:48:30.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_coremask": "0xF", 2024-09-16T14:48:30.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-16T14:48:30.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-16T14:48:30.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_enable_tso": "true", 2024-09-16T14:48:30.800 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-16T14:48:30.800 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-16T14:48:30.800 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_hugepages": "", 2024-09-16T14:48:30.800 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-16T14:48:30.800 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-16T14:48:30.800 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_lro": "true", 2024-09-16T14:48:30.800 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_memory_channel": "4", 2024-09-16T14:48:30.800 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-16T14:48:30.800 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_pmd": "", 2024-09-16T14:48:30.800 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_port_id": "0", 2024-09-16T14:48:30.800 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-16T14:48:30.800 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-16T14:48:30.800 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_dump_on_send": "false", 2024-09-16T14:48:30.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_initial_backoff": "0.200000", 2024-09-16T14:48:30.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_inject_delay_max": "1.000000", 2024-09-16T14:48:30.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-16T14:48:30.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_inject_delay_type": "", 2024-09-16T14:48:30.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-16T14:48:30.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_inject_network_congestion": "0", 2024-09-16T14:48:30.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_inject_socket_failures": "2500", 2024-09-16T14:48:30.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_learn_addr_from_peer": "true", 2024-09-16T14:48:30.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_max_accept_failures": "4", 2024-09-16T14:48:30.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_max_backoff": "15.000000", 2024-09-16T14:48:30.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_mon_client_mode": "secure crc", 2024-09-16T14:48:30.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-16T14:48:30.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_mon_service_mode": "secure crc", 2024-09-16T14:48:30.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_osd_compress_min_size": "1024", 2024-09-16T14:48:30.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_osd_compress_mode": "none", 2024-09-16T14:48:30.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-16T14:48:30.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-16T14:48:30.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_pq_min_cost": "65536", 2024-09-16T14:48:30.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_public_type": "", 2024-09-16T14:48:30.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_service_mode": "crc secure", 2024-09-16T14:48:30.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_tcp_listen_backlog": "512", 2024-09-16T14:48:30.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_tcp_nodelay": "true", 2024-09-16T14:48:30.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-16T14:48:30.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_tcp_rcvbuf": "0", 2024-09-16T14:48:30.802 INFO:teuthology.orchestra.run.smithi040.stdout: "ms_type": "async+posix", 2024-09-16T14:48:30.802 INFO:teuthology.orchestra.run.smithi040.stdout: "no_config_file": "false", 2024-09-16T14:48:30.802 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter_completion_locks_per_session": "32", 2024-09-16T14:48:30.802 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-16T14:48:30.803 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-16T14:48:30.803 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter_inflight_ops": "1024", 2024-09-16T14:48:30.803 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-16T14:48:30.803 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-16T14:48:30.803 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter_tick_interval": "5.000000", 2024-09-16T14:48:30.803 INFO:teuthology.orchestra.run.smithi040.stdout: "objecter_timeout": "10.000000", 2024-09-16T14:48:30.803 INFO:teuthology.orchestra.run.smithi040.stdout: "objectstore_blackhole": "false", 2024-09-16T14:48:30.803 INFO:teuthology.orchestra.run.smithi040.stdout: "openssl_engine_opts": "", 2024-09-16T14:48:30.803 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_agent_delay_time": "5.000000", 2024-09-16T14:48:30.803 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_agent_hist_halflife": "1000", 2024-09-16T14:48:30.803 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_agent_max_low_ops": "2", 2024-09-16T14:48:30.803 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_agent_max_ops": "4", 2024-09-16T14:48:30.803 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-16T14:48:30.803 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-16T14:48:30.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_agent_slop": "0.020000", 2024-09-16T14:48:30.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-16T14:48:30.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-16T14:48:30.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_asio_thread_count": "2", 2024-09-16T14:48:30.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_async_recovery_min_cost": "100", 2024-09-16T14:48:30.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-16T14:48:30.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-16T14:48:30.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_backfill_scan_max": "512", 2024-09-16T14:48:30.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_backfill_scan_min": "64", 2024-09-16T14:48:30.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_backoff_on_degraded": "false", 2024-09-16T14:48:30.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_backoff_on_peering": "false", 2024-09-16T14:48:30.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_backoff_on_unfound": "true", 2024-09-16T14:48:30.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_beacon_report_interval": "300", 2024-09-16T14:48:30.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bench_duration": "30", 2024-09-16T14:48:30.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-16T14:48:30.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bench_max_block_size": "67108864", 2024-09-16T14:48:30.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-16T14:48:30.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_blkin_trace_all": "false", 2024-09-16T14:48:30.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-16T14:48:30.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-16T14:48:30.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-16T14:48:30.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-16T14:48:30.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_check_for_log_corruption": "false", 2024-09-16T14:48:30.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-16T14:48:30.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T14:48:30.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-16T14:48:30.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T14:48:30.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_class_update_on_start": "true", 2024-09-16T14:48:30.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_client_message_cap": "256", 2024-09-16T14:48:30.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_client_message_size_cap": "524288000", 2024-09-16T14:48:30.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_client_op_priority": "63", 2024-09-16T14:48:30.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_client_watch_timeout": "30", 2024-09-16T14:48:30.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_command_max_records": "256", 2024-09-16T14:48:30.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_compact_on_start": "false", 2024-09-16T14:48:30.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-16T14:48:30.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-16T14:48:30.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-16T14:48:30.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_crush_update_on_start": "true", 2024-09-16T14:48:30.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_crush_update_weight_set": "true", 2024-09-16T14:48:30.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-16T14:48:30.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-16T14:48:30.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-16T14:48:30.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-16T14:48:30.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-16T14:48:30.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_feed_pullee": "-1", 2024-09-16T14:48:30.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-16T14:48:30.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-16T14:48:30.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-16T14:48:30.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_misdirected_ops": "false", 2024-09-16T14:48:30.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_no_acting_change": "false", 2024-09-16T14:48:30.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_no_purge_strays": "false", 2024-09-16T14:48:30.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_op_order": "false", 2024-09-16T14:48:30.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-16T14:48:30.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-16T14:48:30.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-16T14:48:30.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-16T14:48:30.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_shutdown": "false", 2024-09-16T14:48:30.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-16T14:48:30.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-16T14:48:30.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-16T14:48:30.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-16T14:48:30.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_verify_snaps": "false", 2024-09-16T14:48:30.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-16T14:48:30.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-16T14:48:30.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_deep_scrub_keys": "1024", 2024-09-16T14:48:30.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-16T14:48:30.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-16T14:48:30.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-16T14:48:30.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_deep_scrub_stride": "524288", 2024-09-16T14:48:30.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-16T14:48:30.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-16T14:48:30.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_default_notify_timeout": "30", 2024-09-16T14:48:30.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_delete_sleep": "0.000000", 2024-09-16T14:48:30.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-16T14:48:30.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-16T14:48:30.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-16T14:48:30.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_discard_disconnected_ops": "true", 2024-09-16T14:48:30.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_enable_op_tracker": "true", 2024-09-16T14:48:30.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-16T14:48:30.809 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-16T14:48:30.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-16T14:48:30.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-16T14:48:30.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_fast_info": "true", 2024-09-16T14:48:30.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_fast_shutdown": "true", 2024-09-16T14:48:30.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-16T14:48:30.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-16T14:48:30.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-16T14:48:30.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-16T14:48:30.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-16T14:48:30.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_function_tracing": "false", 2024-09-16T14:48:30.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_heartbeat_grace": "20", 2024-09-16T14:48:30.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_heartbeat_interval": "6", 2024-09-16T14:48:30.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-16T14:48:30.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_heartbeat_min_peers": "10", 2024-09-16T14:48:30.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_heartbeat_min_size": "2000", 2024-09-16T14:48:30.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_heartbeat_stale": "600", 2024-09-16T14:48:30.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-16T14:48:30.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_hit_set_max_size": "100000", 2024-09-16T14:48:30.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_hit_set_min_size": "1000", 2024-09-16T14:48:30.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-16T14:48:30.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-16T14:48:30.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-16T14:48:30.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-16T14:48:30.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-16T14:48:30.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-16T14:48:30.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_journal_size": "5120", 2024-09-16T14:48:30.812 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_kill_backfill_at": "0", 2024-09-16T14:48:30.812 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-16T14:48:30.812 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_map_cache_size": "50", 2024-09-16T14:48:30.812 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_map_dedup": "true", 2024-09-16T14:48:30.812 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_map_message_max": "40", 2024-09-16T14:48:30.812 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-16T14:48:30.812 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_map_share_max_epochs": "40", 2024-09-16T14:48:30.812 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_attr_name_len": "100", 2024-09-16T14:48:30.812 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_attr_size": "0", 2024-09-16T14:48:30.812 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_backfills": "1", 2024-09-16T14:48:30.812 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_markdown_count": "5", 2024-09-16T14:48:30.812 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_markdown_period": "600", 2024-09-16T14:48:30.812 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_object_name_len": "2048", 2024-09-16T14:48:30.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_object_namespace_len": "256", 2024-09-16T14:48:30.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_object_size": "134217728", 2024-09-16T14:48:30.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-16T14:48:30.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-16T14:48:30.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_pg_blocked_by": "16", 2024-09-16T14:48:30.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_pg_log_entries": "10000", 2024-09-16T14:48:30.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-16T14:48:30.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_pgls": "1024", 2024-09-16T14:48:30.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_push_cost": "8388608", 2024-09-16T14:48:30.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_push_objects": "10", 2024-09-16T14:48:30.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_scrubs": "3", 2024-09-16T14:48:30.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-16T14:48:30.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_trimming_pgs": "2", 2024-09-16T14:48:30.813 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_write_op_reply_len": "64", 2024-09-16T14:48:30.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_max_write_size": "90", 2024-09-16T14:48:30.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-16T14:48:30.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-16T14:48:30.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-16T14:48:30.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-16T14:48:30.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-16T14:48:30.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-16T14:48:30.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-16T14:48:30.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-16T14:48:30.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_profile": "balanced", 2024-09-16T14:48:30.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-16T14:48:30.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-16T14:48:30.814 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-16T14:48:30.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-16T14:48:30.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-16T14:48:30.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-16T14:48:30.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-16T14:48:30.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-16T14:48:30.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-16T14:48:30.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-16T14:48:30.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-16T14:48:30.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_memory_base": "805306368", 2024-09-16T14:48:30.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_memory_cache_min": "134217728", 2024-09-16T14:48:30.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-16T14:48:30.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-16T14:48:30.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_memory_target": "4294967296", 2024-09-16T14:48:30.815 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_memory_target_autotune": "false", 2024-09-16T14:48:30.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-16T14:48:30.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_min_pg_log_entries": "250", 2024-09-16T14:48:30.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_min_recovery_priority": "0", 2024-09-16T14:48:30.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-16T14:48:30.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-16T14:48:30.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mon_report_interval": "5", 2024-09-16T14:48:30.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-16T14:48:30.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-16T14:48:30.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_num_cache_shards": "32", 2024-09-16T14:48:30.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_num_op_tracker_shard": "32", 2024-09-16T14:48:30.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_numa_auto_affinity": "true", 2024-09-16T14:48:30.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_numa_node": "-1", 2024-09-16T14:48:30.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_numa_prefer_iface": "true", 2024-09-16T14:48:30.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_object_clean_region_max_num_intervals": "1000", 2024-09-16T14:48:30.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_objecter_finishers": "1", 2024-09-16T14:48:30.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_objectstore": "bluestore", 2024-09-16T14:48:30.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_objectstore_fuse": "false", 2024-09-16T14:48:30.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_objectstore_tracing": "false", 2024-09-16T14:48:30.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_complaint_time": "30.000000", 2024-09-16T14:48:30.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_history_duration": "600", 2024-09-16T14:48:30.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_history_size": "20", 2024-09-16T14:48:30.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_history_slow_op_size": "20", 2024-09-16T14:48:30.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-16T14:48:30.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_log_threshold": "5", 2024-09-16T14:48:30.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_num_shards": "0", 2024-09-16T14:48:30.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_num_shards_hdd": "5", 2024-09-16T14:48:30.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_num_shards_ssd": "8", 2024-09-16T14:48:30.818 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-16T14:48:30.818 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-16T14:48:30.818 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-16T14:48:30.818 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-16T14:48:30.818 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_pq_min_cost": "65536", 2024-09-16T14:48:30.818 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-16T14:48:30.818 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_queue_cut_off": "high", 2024-09-16T14:48:30.818 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-16T14:48:30.818 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_op_thread_timeout": "15", 2024-09-16T14:48:30.818 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_open_classes_on_start": "true", 2024-09-16T14:48:30.818 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_os_flags": "0", 2024-09-16T14:48:30.818 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_peering_op_priority": "255", 2024-09-16T14:48:30.818 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pg_delete_cost": "1048576", 2024-09-16T14:48:30.818 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pg_delete_priority": "5", 2024-09-16T14:48:30.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-16T14:48:30.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-16T14:48:30.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-16T14:48:30.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pg_log_trim_max": "10000", 2024-09-16T14:48:30.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pg_log_trim_min": "100", 2024-09-16T14:48:30.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-16T14:48:30.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-16T14:48:30.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-16T14:48:30.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-16T14:48:30.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-16T14:48:30.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-16T14:48:30.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-16T14:48:30.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-16T14:48:30.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-16T14:48:30.820 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_crimson": "false", 2024-09-16T14:48:30.820 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-16T14:48:30.820 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-16T14:48:30.820 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-16T14:48:30.820 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-16T14:48:30.820 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-16T14:48:30.820 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-16T14:48:30.820 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-16T14:48:30.820 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-16T14:48:30.820 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_flags": "0", 2024-09-16T14:48:30.820 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-16T14:48:30.820 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_min_size": "0", 2024-09-16T14:48:30.820 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-16T14:48:30.821 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_pg_num": "32", 2024-09-16T14:48:30.821 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_pgp_num": "0", 2024-09-16T14:48:30.821 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-16T14:48:30.821 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_size": "2", 2024-09-16T14:48:30.821 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_default_type": "replicated", 2024-09-16T14:48:30.821 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-16T14:48:30.821 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-16T14:48:30.821 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_push_per_object_cost": "1000", 2024-09-16T14:48:30.821 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-16T14:48:30.821 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recover_clone_overlap": "true", 2024-09-16T14:48:30.821 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-16T14:48:30.821 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_cost": "20971520", 2024-09-16T14:48:30.821 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-16T14:48:30.821 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_max_active": "0", 2024-09-16T14:48:30.822 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-16T14:48:30.822 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-16T14:48:30.822 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-16T14:48:30.822 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-16T14:48:30.822 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_max_single_start": "1", 2024-09-16T14:48:30.822 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_op_priority": "3", 2024-09-16T14:48:30.822 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-16T14:48:30.822 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_priority": "5", 2024-09-16T14:48:30.824 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-16T14:48:30.824 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_sleep": "0.000000", 2024-09-16T14:48:30.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-16T14:48:30.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-16T14:48:30.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-16T14:48:30.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_repair_during_recovery": "false", 2024-09-16T14:48:30.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_requested_scrub_priority": "120", 2024-09-16T14:48:30.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-16T14:48:30.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-16T14:48:30.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_auto_repair": "false", 2024-09-16T14:48:30.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-16T14:48:30.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-16T14:48:30.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_begin_hour": "0", 2024-09-16T14:48:30.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_begin_week_day": "0", 2024-09-16T14:48:30.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_chunk_max": "25", 2024-09-16T14:48:30.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_chunk_min": "5", 2024-09-16T14:48:30.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_cost": "52428800", 2024-09-16T14:48:30.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_during_recovery": "false", 2024-09-16T14:48:30.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_end_hour": "0", 2024-09-16T14:48:30.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_end_week_day": "0", 2024-09-16T14:48:30.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_event_cost": "4096", 2024-09-16T14:48:30.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-16T14:48:30.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-16T14:48:30.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_invalid_stats": "true", 2024-09-16T14:48:30.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-16T14:48:30.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-16T14:48:30.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_max_preemptions": "5", 2024-09-16T14:48:30.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-16T14:48:30.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_priority": "5", 2024-09-16T14:48:30.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-16T14:48:30.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_sleep": "0.000000", 2024-09-16T14:48:30.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-16T14:48:30.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-16T14:48:30.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-16T14:48:30.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-16T14:48:30.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_skip_data_digest": "false", 2024-09-16T14:48:30.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_smart_report_timeout": "5", 2024-09-16T14:48:30.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_snap_trim_cost": "1048576", 2024-09-16T14:48:30.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_snap_trim_priority": "5", 2024-09-16T14:48:30.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-16T14:48:30.828 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-16T14:48:30.828 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-16T14:48:30.828 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-16T14:48:30.828 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-16T14:48:30.828 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-16T14:48:30.828 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-16T14:48:30.828 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_target_transaction_size": "30", 2024-09-16T14:48:30.828 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-16T14:48:30.828 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-16T14:48:30.828 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-16T14:48:30.828 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-16T14:48:30.828 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-16T14:48:30.829 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-16T14:48:30.829 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-16T14:48:30.829 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-16T14:48:30.829 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-16T14:48:30.829 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-16T14:48:30.829 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_tracing": "false", 2024-09-16T14:48:30.829 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_use_stale_snap": "false", 2024-09-16T14:48:30.829 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-16T14:48:30.829 INFO:teuthology.orchestra.run.smithi040.stdout: "osdc_blkin_trace_all": "false", 2024-09-16T14:48:30.829 INFO:teuthology.orchestra.run.smithi040.stdout: "paxos_kill_at": "0", 2024-09-16T14:48:30.829 INFO:teuthology.orchestra.run.smithi040.stdout: "paxos_max_join_drift": "10", 2024-09-16T14:48:30.829 INFO:teuthology.orchestra.run.smithi040.stdout: "paxos_min": "500", 2024-09-16T14:48:30.829 INFO:teuthology.orchestra.run.smithi040.stdout: "paxos_min_wait": "0.050000", 2024-09-16T14:48:30.830 INFO:teuthology.orchestra.run.smithi040.stdout: "paxos_propose_interval": "1.000000", 2024-09-16T14:48:30.830 INFO:teuthology.orchestra.run.smithi040.stdout: "paxos_service_trim_max": "500", 2024-09-16T14:48:30.830 INFO:teuthology.orchestra.run.smithi040.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-16T14:48:30.830 INFO:teuthology.orchestra.run.smithi040.stdout: "paxos_service_trim_min": "250", 2024-09-16T14:48:30.830 INFO:teuthology.orchestra.run.smithi040.stdout: "paxos_stash_full_interval": "25", 2024-09-16T14:48:30.830 INFO:teuthology.orchestra.run.smithi040.stdout: "paxos_trim_max": "500", 2024-09-16T14:48:30.830 INFO:teuthology.orchestra.run.smithi040.stdout: "paxos_trim_min": "250", 2024-09-16T14:48:30.830 INFO:teuthology.orchestra.run.smithi040.stdout: "perf": "true", 2024-09-16T14:48:30.830 INFO:teuthology.orchestra.run.smithi040.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-16T14:48:30.830 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-16T14:48:30.830 INFO:teuthology.orchestra.run.smithi040.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-16T14:48:30.830 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "-", 2024-09-16T14:48:30.830 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrv": "", 2024-09-16T14:48:30.831 INFO:teuthology.orchestra.run.smithi040.stdout: "public_bind_addr": "-", 2024-09-16T14:48:30.831 INFO:teuthology.orchestra.run.smithi040.stdout: "public_network": "", 2024-09-16T14:48:30.831 INFO:teuthology.orchestra.run.smithi040.stdout: "public_network_interface": "", 2024-09-16T14:48:30.831 INFO:teuthology.orchestra.run.smithi040.stdout: "qat_compressor_enabled": "false", 2024-09-16T14:48:30.831 INFO:teuthology.orchestra.run.smithi040.stdout: "qat_compressor_session_max_number": "256", 2024-09-16T14:48:30.831 INFO:teuthology.orchestra.run.smithi040.stdout: "rados_mon_op_timeout": "0", 2024-09-16T14:48:30.831 INFO:teuthology.orchestra.run.smithi040.stdout: "rados_osd_op_timeout": "0", 2024-09-16T14:48:30.831 INFO:teuthology.orchestra.run.smithi040.stdout: "rados_tracing": "false", 2024-09-16T14:48:30.831 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_atime_update_interval": "60", 2024-09-16T14:48:30.831 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-16T14:48:30.831 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_balance_parent_reads": "false", 2024-09-16T14:48:30.831 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_balance_snap_reads": "false", 2024-09-16T14:48:30.831 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_blkin_trace_all": "false", 2024-09-16T14:48:30.831 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-16T14:48:30.832 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-16T14:48:30.832 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_cache": "true", 2024-09-16T14:48:30.832 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-16T14:48:30.832 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-16T14:48:30.832 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-16T14:48:30.832 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-16T14:48:30.832 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_cache_policy": "writearound", 2024-09-16T14:48:30.832 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_cache_size": "33554432", 2024-09-16T14:48:30.832 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-16T14:48:30.832 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-16T14:48:30.832 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_clone_copy_on_read": "false", 2024-09-16T14:48:30.832 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_compression_hint": "none", 2024-09-16T14:48:30.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_concurrent_management_ops": "10", 2024-09-16T14:48:30.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-16T14:48:30.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_default_clone_format": "auto", 2024-09-16T14:48:30.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_default_data_pool": "", 2024-09-16T14:48:30.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_default_features": "61", 2024-09-16T14:48:30.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_default_format": "2", 2024-09-16T14:48:30.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_default_map_options": "", 2024-09-16T14:48:30.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_default_order": "22", 2024-09-16T14:48:30.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_default_pool": "rbd", 2024-09-16T14:48:30.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-16T14:48:30.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_default_stripe_count": "0", 2024-09-16T14:48:30.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_default_stripe_unit": "0", 2024-09-16T14:48:30.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-16T14:48:30.834 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-16T14:48:30.834 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-16T14:48:30.834 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_enable_alloc_hint": "true", 2024-09-16T14:48:30.834 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-16T14:48:30.834 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_io_scheduler": "simple", 2024-09-16T14:48:30.834 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-16T14:48:30.834 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-16T14:48:30.834 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-16T14:48:30.834 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-16T14:48:30.834 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-16T14:48:30.834 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-16T14:48:30.834 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-16T14:48:30.834 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-16T14:48:30.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-16T14:48:30.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_journal_order": "24", 2024-09-16T14:48:30.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_journal_pool": "", 2024-09-16T14:48:30.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_journal_splay_width": "4", 2024-09-16T14:48:30.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_localize_parent_reads": "false", 2024-09-16T14:48:30.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_localize_snap_reads": "false", 2024-09-16T14:48:30.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-16T14:48:30.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-16T14:48:30.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-16T14:48:30.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-16T14:48:30.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-16T14:48:30.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-16T14:48:30.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-16T14:48:30.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-16T14:48:30.836 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-16T14:48:30.836 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-16T14:48:30.836 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-16T14:48:30.836 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-16T14:48:30.836 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-16T14:48:30.836 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-16T14:48:30.836 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-16T14:48:30.836 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-16T14:48:30.836 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-16T14:48:30.836 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-16T14:48:30.836 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-16T14:48:30.836 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-16T14:48:30.836 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-16T14:48:30.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-16T14:48:30.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-16T14:48:30.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-16T14:48:30.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-16T14:48:30.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-16T14:48:30.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-16T14:48:30.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-16T14:48:30.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-16T14:48:30.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-16T14:48:30.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-16T14:48:30.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_mtime_update_interval": "60", 2024-09-16T14:48:30.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_non_blocking_aio": "true", 2024-09-16T14:48:30.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_op_thread_timeout": "60", 2024-09-16T14:48:30.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_op_threads": "1", 2024-09-16T14:48:30.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_parent_cache_enabled": "false", 2024-09-16T14:48:30.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-16T14:48:30.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-16T14:48:30.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-16T14:48:30.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_plugins": "", 2024-09-16T14:48:30.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_bps_burst": "0", 2024-09-16T14:48:30.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-16T14:48:30.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_bps_limit": "0", 2024-09-16T14:48:30.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_exclude_ops": "0", 2024-09-16T14:48:30.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_iops_burst": "0", 2024-09-16T14:48:30.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-16T14:48:30.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_iops_limit": "0", 2024-09-16T14:48:30.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-16T14:48:30.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-16T14:48:30.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-16T14:48:30.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-16T14:48:30.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-16T14:48:30.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-16T14:48:30.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-16T14:48:30.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-16T14:48:30.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-16T14:48:30.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-16T14:48:30.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-16T14:48:30.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-16T14:48:30.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-16T14:48:30.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-16T14:48:30.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_read_from_replica_policy": "default", 2024-09-16T14:48:30.840 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-16T14:48:30.840 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-16T14:48:30.840 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-16T14:48:30.840 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-16T14:48:30.840 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_skip_partial_discard": "true", 2024-09-16T14:48:30.840 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-16T14:48:30.840 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_tracing": "false", 2024-09-16T14:48:30.840 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_validate_names": "true", 2024-09-16T14:48:30.840 INFO:teuthology.orchestra.run.smithi040.stdout: "rbd_validate_pool": "true", 2024-09-16T14:48:30.840 INFO:teuthology.orchestra.run.smithi040.stdout: "restapi_base_url": "", 2024-09-16T14:48:30.840 INFO:teuthology.orchestra.run.smithi040.stdout: "restapi_log_level": "", 2024-09-16T14:48:30.840 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_acl_grants_max_num": "100", 2024-09-16T14:48:30.840 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_admin_entry": "admin", 2024-09-16T14:48:30.841 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-16T14:48:30.841 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_backend_store": "rados", 2024-09-16T14:48:30.841 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_barbican_url": "", 2024-09-16T14:48:30.841 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_beast_enable_async": "true", 2024-09-16T14:48:30.841 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-16T14:48:30.841 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-16T14:48:30.841 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-16T14:48:30.841 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-16T14:48:30.841 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-16T14:48:30.841 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-16T14:48:30.841 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-16T14:48:30.841 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_cache_enabled": "true", 2024-09-16T14:48:30.841 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_cache_expiry_interval": "900", 2024-09-16T14:48:30.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_cache_lru_size": "10000", 2024-09-16T14:48:30.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_config_store": "rados", 2024-09-16T14:48:30.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_content_length_compat": "false", 2024-09-16T14:48:30.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_copy_obj_progress": "true", 2024-09-16T14:48:30.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-16T14:48:30.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_cors_rules_max_num": "100", 2024-09-16T14:48:30.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_cross_domain_policy": "", 2024-09-16T14:48:30.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-16T14:48:30.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_kmip_addr": "", 2024-09-16T14:48:30.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-16T14:48:30.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-16T14:48:30.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-16T14:48:30.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-16T14:48:30.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_kmip_password": "", 2024-09-16T14:48:30.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-16T14:48:30.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_kmip_username": "", 2024-09-16T14:48:30.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_require_ssl": "true", 2024-09-16T14:48:30.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-16T14:48:30.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-16T14:48:30.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-16T14:48:30.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-16T14:48:30.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-16T14:48:30.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-16T14:48:30.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-16T14:48:30.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-16T14:48:30.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-16T14:48:30.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-16T14:48:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-16T14:48:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-16T14:48:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-16T14:48:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-16T14:48:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-16T14:48:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_vault_addr": "", 2024-09-16T14:48:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_vault_auth": "token", 2024-09-16T14:48:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_vault_namespace": "", 2024-09-16T14:48:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_vault_prefix": "", 2024-09-16T14:48:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-16T14:48:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-16T14:48:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-16T14:48:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-16T14:48:30.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_vault_token_file": "", 2024-09-16T14:48:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-16T14:48:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_curl_buffersize": "524288", 2024-09-16T14:48:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-16T14:48:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_curl_low_speed_time": "300", 2024-09-16T14:48:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-16T14:48:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-16T14:48:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-16T14:48:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-16T14:48:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-16T14:48:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-16T14:48:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-16T14:48:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-16T14:48:30.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-16T14:48:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-16T14:48:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-16T14:48:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_data_log_changes_size": "1000", 2024-09-16T14:48:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_data_log_num_shards": "128", 2024-09-16T14:48:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-16T14:48:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_data_log_window": "30", 2024-09-16T14:48:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-16T14:48:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-16T14:48:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-16T14:48:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-16T14:48:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-16T14:48:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-16T14:48:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-16T14:48:30.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-16T14:48:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-16T14:48:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-16T14:48:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-16T14:48:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-16T14:48:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-16T14:48:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-16T14:48:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-16T14:48:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-16T14:48:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-16T14:48:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-16T14:48:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-16T14:48:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-16T14:48:30.847 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-16T14:48:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-16T14:48:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-16T14:48:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-16T14:48:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dns_name": "", 2024-09-16T14:48:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dns_s3website_name": "", 2024-09-16T14:48:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_dynamic_resharding": "true", 2024-09-16T14:48:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-16T14:48:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_enable_gc_threads": "true", 2024-09-16T14:48:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_enable_lc_threads": "true", 2024-09-16T14:48:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_enable_ops_log": "false", 2024-09-16T14:48:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_enable_quota_threads": "true", 2024-09-16T14:48:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_enable_static_website": "false", 2024-09-16T14:48:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_enable_usage_log": "false", 2024-09-16T14:48:30.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_enforce_swift_acls": "true", 2024-09-16T14:48:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_exit_timeout_secs": "120", 2024-09-16T14:48:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_expose_bucket": "false", 2024-09-16T14:48:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_extended_http_attrs": "", 2024-09-16T14:48:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_filter": "none", 2024-09-16T14:48:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-16T14:48:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_frontends": "beast port=7480", 2024-09-16T14:48:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-16T14:48:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_gc_max_deferred": "50", 2024-09-16T14:48:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-16T14:48:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_gc_max_objs": "32", 2024-09-16T14:48:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-16T14:48:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-16T14:48:30.849 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_gc_processor_period": "3600", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_init_timeout": "300", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_admin_domain": "", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_admin_password": "", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_admin_password_path": "", 2024-09-16T14:48:30.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_admin_project": "", 2024-09-16T14:48:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_admin_tenant": "", 2024-09-16T14:48:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_admin_token": "", 2024-09-16T14:48:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_admin_token_path": "", 2024-09-16T14:48:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_admin_user": "", 2024-09-16T14:48:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_api_version": "2", 2024-09-16T14:48:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_barbican_domain": "", 2024-09-16T14:48:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_barbican_password": "", 2024-09-16T14:48:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_barbican_project": "", 2024-09-16T14:48:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-16T14:48:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_barbican_user": "", 2024-09-16T14:48:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-16T14:48:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-16T14:48:30.851 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-16T14:48:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-16T14:48:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-16T14:48:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_url": "", 2024-09-16T14:48:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-16T14:48:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_lc_debug_interval": "-1", 2024-09-16T14:48:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_lc_lock_max_time": "90", 2024-09-16T14:48:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_lc_max_objs": "32", 2024-09-16T14:48:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_lc_max_rules": "1000", 2024-09-16T14:48:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_lc_max_worker": "3", 2024-09-16T14:48:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-16T14:48:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_lc_thread_delay": "0", 2024-09-16T14:48:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-16T14:48:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_ldap_dnattr": "uid", 2024-09-16T14:48:30.852 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-16T14:48:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_ldap_searchfilter": "", 2024-09-16T14:48:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-16T14:48:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-16T14:48:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-16T14:48:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-16T14:48:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-16T14:48:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_log_http_headers": "", 2024-09-16T14:48:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-16T14:48:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-16T14:48:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_log_object_name_utc": "false", 2024-09-16T14:48:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-16T14:48:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_max_attr_name_len": "0", 2024-09-16T14:48:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_max_attr_size": "0", 2024-09-16T14:48:30.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-16T14:48:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_max_chunk_size": "4194304", 2024-09-16T14:48:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-16T14:48:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-16T14:48:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-16T14:48:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_max_listing_results": "1000", 2024-09-16T14:48:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_max_notify_retries": "10", 2024-09-16T14:48:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-16T14:48:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_max_put_param_size": "1048576", 2024-09-16T14:48:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_max_put_size": "5368709120", 2024-09-16T14:48:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_max_slo_entries": "1000", 2024-09-16T14:48:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_md_log_max_shards": "64", 2024-09-16T14:48:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-16T14:48:30.854 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-16T14:48:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-16T14:48:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-16T14:48:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_mp_lock_max_time": "600", 2024-09-16T14:48:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-16T14:48:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-16T14:48:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-16T14:48:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-16T14:48:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-16T14:48:30.855 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-16T14:48:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-16T14:48:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-16T14:48:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_nfs_max_gc": "300", 2024-09-16T14:48:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-16T14:48:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-16T14:48:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-16T14:48:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-16T14:48:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-16T14:48:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-16T14:48:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-16T14:48:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_num_async_rados_threads": "32", 2024-09-16T14:48:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_num_control_oids": "8", 2024-09-16T14:48:30.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_numa_node": "-1", 2024-09-16T14:48:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-16T14:48:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-16T14:48:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_objexp_chunk_size": "100", 2024-09-16T14:48:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_objexp_gc_interval": "600", 2024-09-16T14:48:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-16T14:48:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-16T14:48:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-16T14:48:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_op_thread_timeout": "600", 2024-09-16T14:48:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_op_tracing": "false", 2024-09-16T14:48:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_opa_token": "", 2024-09-16T14:48:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_opa_url": "", 2024-09-16T14:48:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_opa_verify_ssl": "true", 2024-09-16T14:48:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-16T14:48:30.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-16T14:48:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_ops_log_rados": "false", 2024-09-16T14:48:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_ops_log_socket_path": "", 2024-09-16T14:48:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-16T14:48:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-16T14:48:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-16T14:48:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_period_push_interval": "2.000000", 2024-09-16T14:48:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-16T14:48:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-16T14:48:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-16T14:48:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_print_continue": "true", 2024-09-16T14:48:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-16T14:48:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-16T14:48:30.858 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-16T14:48:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-16T14:48:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-16T14:48:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_rados_tracing": "false", 2024-09-16T14:48:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_realm": "", 2024-09-16T14:48:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_realm_id": "", 2024-09-16T14:48:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-16T14:48:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_region": "", 2024-09-16T14:48:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-16T14:48:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-16T14:48:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-16T14:48:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-16T14:48:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_request_uri": "", 2024-09-16T14:48:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_reshard_batch_size": "64", 2024-09-16T14:48:30.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-16T14:48:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_reshard_max_aio": "128", 2024-09-16T14:48:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_reshard_num_logs": "16", 2024-09-16T14:48:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_reshard_thread_interval": "600", 2024-09-16T14:48:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_resolve_cname": "false", 2024-09-16T14:48:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-16T14:48:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_run_sync_thread": "true", 2024-09-16T14:48:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-16T14:48:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-16T14:48:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-16T14:48:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-16T14:48:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-16T14:48:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-16T14:48:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-16T14:48:30.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-16T14:48:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_scheduler_type": "throttler", 2024-09-16T14:48:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_script_uri": "", 2024-09-16T14:48:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_service_provider_name": "", 2024-09-16T14:48:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-16T14:48:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sts_client_id": "", 2024-09-16T14:48:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sts_client_secret": "", 2024-09-16T14:48:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sts_entry": "sts", 2024-09-16T14:48:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sts_key": "sts", 2024-09-16T14:48:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-16T14:48:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sts_min_session_duration": "900", 2024-09-16T14:48:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sts_token_introspection_url": "", 2024-09-16T14:48:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_swift_account_in_url": "false", 2024-09-16T14:48:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_swift_auth_entry": "auth", 2024-09-16T14:48:30.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_swift_auth_url": "", 2024-09-16T14:48:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_swift_custom_header": "", 2024-09-16T14:48:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-16T14:48:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_swift_need_stats": "true", 2024-09-16T14:48:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_swift_tenant_name": "", 2024-09-16T14:48:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_swift_token_expiration": "86400", 2024-09-16T14:48:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_swift_url": "", 2024-09-16T14:48:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_swift_url_prefix": "swift", 2024-09-16T14:48:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-16T14:48:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-16T14:48:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-16T14:48:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sync_lease_period": "120", 2024-09-16T14:48:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-16T14:48:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-16T14:48:30.862 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-16T14:48:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-16T14:48:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-16T14:48:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-16T14:48:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-16T14:48:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-16T14:48:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-16T14:48:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_thread_pool_size": "512", 2024-09-16T14:48:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_torrent_comment": "", 2024-09-16T14:48:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_torrent_createby": "", 2024-09-16T14:48:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_torrent_encoding": "", 2024-09-16T14:48:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_torrent_flag": "false", 2024-09-16T14:48:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_torrent_origin": "", 2024-09-16T14:48:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-16T14:48:30.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_torrent_tracker": "", 2024-09-16T14:48:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_trust_forwarded_https": "false", 2024-09-16T14:48:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-16T14:48:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-16T14:48:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_usage_max_shards": "32", 2024-09-16T14:48:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_usage_max_user_shards": "1", 2024-09-16T14:48:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_use_opa_authz": "false", 2024-09-16T14:48:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-16T14:48:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-16T14:48:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_user_max_buckets": "1000", 2024-09-16T14:48:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_user_policies_max_num": "100", 2024-09-16T14:48:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-16T14:48:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-16T14:48:30.864 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-16T14:48:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-16T14:48:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_user_unique_email": "true", 2024-09-16T14:48:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_verify_ssl": "true", 2024-09-16T14:48:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-16T14:48:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_zone": "", 2024-09-16T14:48:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_zone_id": "", 2024-09-16T14:48:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-16T14:48:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_zonegroup": "", 2024-09-16T14:48:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_zonegroup_id": "", 2024-09-16T14:48:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-16T14:48:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rgwlc_auto_session_clear": "true", 2024-09-16T14:48:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-16T14:48:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_block_size": "4096", 2024-09-16T14:48:30.865 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-16T14:48:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-16T14:48:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-16T14:48:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-16T14:48:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-16T14:48:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_cache_size": "536870912", 2024-09-16T14:48:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-16T14:48:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-16T14:48:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-16T14:48:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-16T14:48:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-16T14:48:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-16T14:48:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-16T14:48:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-16T14:48:30.866 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_index_type": "binary_search", 2024-09-16T14:48:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-16T14:48:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-16T14:48:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_partition_filters": "false", 2024-09-16T14:48:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_perf": "false", 2024-09-16T14:48:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-16T14:48:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-16T14:48:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-16T14:48:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "run_dir": "/var/run/ceph", 2024-09-16T14:48:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_block_create": "true", 2024-09-16T14:48:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_cache_lru_size": "67108864", 2024-09-16T14:48:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-16T14:48:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_default_max_object_size": "16777216", 2024-09-16T14:48:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-16T14:48:30.867 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_device_size": "53687091200", 2024-09-16T14:48:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_journal_batch_capacity": "16", 2024-09-16T14:48:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-16T14:48:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-16T14:48:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-16T14:48:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_main_device_type": "SSD", 2024-09-16T14:48:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-16T14:48:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-16T14:48:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-16T14:48:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-16T14:48:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-16T14:48:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "seastore_segment_size": "67108864", 2024-09-16T14:48:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "set_keepcaps": "false", 2024-09-16T14:48:30.868 INFO:teuthology.orchestra.run.smithi040.stdout: "setgroup": "", 2024-09-16T14:48:30.869 INFO:teuthology.orchestra.run.smithi040.stdout: "setuser": "", 2024-09-16T14:48:30.869 INFO:teuthology.orchestra.run.smithi040.stdout: "setuser_match_path": "", 2024-09-16T14:48:30.869 INFO:teuthology.orchestra.run.smithi040.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-16T14:48:30.869 INFO:teuthology.orchestra.run.smithi040.stdout: "thp": "false", 2024-09-16T14:48:30.869 INFO:teuthology.orchestra.run.smithi040.stdout: "threadpool_default_timeout": "60", 2024-09-16T14:48:30.869 INFO:teuthology.orchestra.run.smithi040.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-16T14:48:30.869 INFO:teuthology.orchestra.run.smithi040.stdout: "throttler_perf_counter": "true" 2024-09-16T14:48:30.869 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:48:30.871 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-16T14:48:30.985 INFO:teuthology.orchestra.run.smithi040.stdout:mon.a: {} 2024-09-16T14:48:30.986 INFO:teuthology.orchestra.run.smithi040.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T14:48:30.996 INFO:teuthology.orchestra.run.smithi040.stdout:mon.b: {} 2024-09-16T14:48:30.996 INFO:teuthology.orchestra.run.smithi040.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T14:48:31.006 INFO:teuthology.orchestra.run.smithi040.stdout:mon.c: {} 2024-09-16T14:48:31.006 INFO:teuthology.orchestra.run.smithi040.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T14:48:31.016 DEBUG:teuthology.orchestra.run.smithi040:> type ceph-objectstore-tool 2024-09-16T14:48:31.031 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T14:48:31.031 DEBUG:teuthology.orchestra.run.smithi114:> type ceph-objectstore-tool 2024-09-16T14:48:31.050 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T14:48:31.050 DEBUG:teuthology.orchestra.run.smithi111:> type ceph-objectstore-tool 2024-09-16T14:48:31.069 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T14:48:31.069 INFO:teuthology.run_tasks:Running task rados... 2024-09-16T14:48:31.083 INFO:tasks.rados:Beginning rados... 2024-09-16T14:48:31.083 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 4, 'l': 3, 'm': 2, 'name': 'lrcprofile', 'plugin': 'lrc'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False} 2024-09-16T14:48:31.083 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 2500, 'ms tcp read timeout': 5, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 6, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'ae7f37330c1a72493684eb1fcaed6f2a01195bb6'}} 2024-09-16T14:48:31.083 INFO:tasks.rados:config is {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 4, 'l': 3, 'm': 2, 'name': 'lrcprofile', 'plugin': 'lrc'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False} 2024-09-16T14:48:31.084 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-09-16T14:48:31.125 INFO:tasks.rados:joining rados 2024-09-16T14:48:31.125 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-16T14:48:31.125 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-16T14:48:31.125 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:48:31.125 INFO:tasks.thrashosds.thrasher:test_bluestore_reshard started 2024-09-16T14:48:31.126 INFO:tasks.thrashosds.thrasher:Killing osd 6, live_osds are [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-16T14:48:31.126 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-16T14:48:31.126 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:48:31.126 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-16T14:48:31.126 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-16T14:48:31.127 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:48:31.127 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-16T14:48:31.127 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:48:31.128 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-16T14:48:31.128 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:48:31.128 INFO:tasks.rados:clients are ['client.0'] 2024-09-16T14:48:31.128 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set lrcprofile crush-failure-domain=osd k=4 l=3 m=2 name=lrcprofile plugin=lrc 2024-09-16T14:48:31.129 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:31.126+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:31.156 INFO:tasks.ceph.osd.6:Stopped 2024-09-16T14:48:31.156 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 6 2024-09-16T14:48:31.228 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:31.227+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:31.265 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:48:31.265 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:48:31.274 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:31.274 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:31.287 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:48:31.296 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:48:31.296 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:48:31.329 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:31.327+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:31.329 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:48:31.330 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:48:31.363 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:48:31.363 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:48:31.395 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:48:31.395 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:48:31.401 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:31.401 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:31.411 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:48:31.430 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:48:31.430 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:48:31.430 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:31.429+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:31.523 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-16T14:48:31.522+0000 7f544d3bb640 -1 mon.a@0(leader).osd e24 definitely_dead 0 2024-09-16T14:48:31.530 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:31.529+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:31.536 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:31.536 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:31.545 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:48:31.631 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:31.630+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:31.652 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:31.652 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:31.661 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:48:31.733 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:31.732+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:31.768 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:31.768 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:31.777 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:48:31.848 INFO:teuthology.orchestra.run.smithi040.stderr:marked down osd.6. 2024-09-16T14:48:31.848 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:31.847+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:31.851 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is set 2024-09-16T14:48:31.863 INFO:tasks.rados:starting run 0 out of 1 2024-09-16T14:48:31.863 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-16T14:48:31.863 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 16 erasure lrcprofile 2024-09-16T14:48:31.868 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] 2024-09-16T14:48:31.868 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2024-09-16T14:48:31.929 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:31.929 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:31.935 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:31.933+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:31.939 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:48:32.035 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:32.033+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:32.091 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:32.091 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:32.100 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:48:32.144 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:32.143+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:32.211 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:32.211 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:32.217 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-16T14:48:32.216+0000 7f544d3bb640 -1 mon.a@0(leader).osd e25 definitely_dead 0 2024-09-16T14:48:32.220 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:48:32.237 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:32.236+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:32.338 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:32.337+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:32.367 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:32.375 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:48:32.439 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:32.438+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:32.482 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:32.490 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:48:32.540 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:32.539+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:32.597 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:32.597 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:32.606 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:48:32.641 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:32.640+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:32.712 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:32.712 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:32.720 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:48:32.742 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:32.741+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:32.835 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:32.836 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:32.843 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:32.842+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:32.844 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:48:32.885 INFO:teuthology.orchestra.run.smithi040.stderr:marked out osd.6. 2024-09-16T14:48:32.898 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.6 2024-09-16T14:48:32.898 INFO:tasks.thrashosds.thrasher:remote for osd 6 is ubuntu@smithi111.front.sepia.ceph.com 2024-09-16T14:48:32.898 DEBUG:teuthology.orchestra.run.smithi111:> type ceph-objectstore-tool 2024-09-16T14:48:32.912 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T14:48:32.913 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op list-pgs 2024-09-16T14:48:32.944 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:32.943+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:32.949 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:32.949 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:32.957 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:48:33.045 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:33.044+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:33.100 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:33.101 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:33.109 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:48:33.153 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:33.151+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:33.213 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:33.213 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:33.221 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:48:33.263 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:33.262+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:33.349 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:33.348+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:33.366 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:33.367 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:33.375 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:48:33.449 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:33.448+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:33.476 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:33.477 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:33.485 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:48:33.550 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:33.548+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:33.588 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:33.588 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:48:33.588 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:48:33.588 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:33.597 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:48:33.650 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:33.649+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:33.738 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:33.738 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:48:33.739 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:48:33.739 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:48:33.739 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:33.747 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:48:33.792 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:33.790+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:33.852 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:33.853 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:48:33.853 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:48:33.853 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:48:33.853 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.853 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:33.853 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:48:33.853 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.709641119, 2024-09-16T14:48:33.853 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.223181782, 2024-09-16T14:48:33.853 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.853 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.853 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.853 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.853 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 1 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.854 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139495+0000", 2024-09-16T14:48:33.855 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:33.855 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.855 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.855 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.855 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139504+0000", 2024-09-16T14:48:33.855 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6119999999999999e-06 2024-09-16T14:48:33.855 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139514+0000", 2024-09-16T14:48:33.856 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:48:33.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139533+0000", 2024-09-16T14:48:33.856 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9573e-05 2024-09-16T14:48:33.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.857 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139585+0000", 2024-09-16T14:48:33.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2170999999999999e-05 2024-09-16T14:48:33.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.857 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:48:33.857 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139594+0000", 2024-09-16T14:48:33.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.884e-06 2024-09-16T14:48:33.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.857 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.857 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359233+0000", 2024-09-16T14:48:33.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.21963890899999999 2024-09-16T14:48:33.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359437+0000", 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00020374999999999999 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359611+0000", 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017445899999999999 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.360506+0000", 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00089459299999999995 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.858 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.859 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.859 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362594+0000", 2024-09-16T14:48:33.859 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002087776 2024-09-16T14:48:33.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.859 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.859 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.859 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362636+0000", 2024-09-16T14:48:33.859 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1933e-05 2024-09-16T14:48:33.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.859 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.859 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.859 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362679+0000", 2024-09-16T14:48:33.860 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2877999999999998e-05 2024-09-16T14:48:33.860 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.860 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.860 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.860 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.860 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.860 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:33.860 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:48:33.860 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.460929712, 2024-09-16T14:48:33.860 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077691490000000004, 2024-09-16T14:48:33.860 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.860 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.861 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.861 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.861 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.861 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 21 2024-09-16T14:48:33.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.861 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.861 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.861 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.861 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:48:33.861 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.861 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.861 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:48:33.862 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:48:33.862 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.863 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388278+0000", 2024-09-16T14:48:33.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0809000000000006e-05 2024-09-16T14:48:33.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.863 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.863 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.863 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388286+0000", 2024-09-16T14:48:33.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.204e-06 2024-09-16T14:48:33.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.863 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.863 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.863 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388291+0000", 2024-09-16T14:48:33.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9349999999999999e-06 2024-09-16T14:48:33.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.863 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.864 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.864 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388713+0000", 2024-09-16T14:48:33.864 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00042189899999999999 2024-09-16T14:48:33.864 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.864 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.864 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.864 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388766+0000", 2024-09-16T14:48:33.864 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2750999999999997e-05 2024-09-16T14:48:33.864 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.864 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.864 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.864 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388853+0000", 2024-09-16T14:48:33.864 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6635e-05 2024-09-16T14:48:33.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.390635+0000", 2024-09-16T14:48:33.865 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0017817950000000001 2024-09-16T14:48:33.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.395898+0000", 2024-09-16T14:48:33.865 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0052630510000000004 2024-09-16T14:48:33.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.395941+0000", 2024-09-16T14:48:33.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3248999999999999e-05 2024-09-16T14:48:33.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.866 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.395977+0000", 2024-09-16T14:48:33.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6501000000000002e-05 2024-09-16T14:48:33.866 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.866 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.866 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.866 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.866 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:33.866 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:48:33.867 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.460838704, 2024-09-16T14:48:33.867 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077998870000000001, 2024-09-16T14:48:33.867 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.867 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.867 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.867 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.867 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.867 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 22 2024-09-16T14:48:33.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.867 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.867 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.867 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:48:33.867 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.868 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.868 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.868 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:48:33.868 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.868 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.868 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.868 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:48:33.868 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.868 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.868 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388311+0000", 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2853e-05 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388318+0000", 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7549999999999997e-06 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388322+0000", 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5889999999999999e-06 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.869 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.389160+0000", 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000837877 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.389230+0000", 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0551999999999998e-05 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.389371+0000", 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014102099999999999 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.870 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.390881+0000", 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00151004 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.396022+0000", 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005140716 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.396074+0000", 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2089999999999998e-05 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.871 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.872 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.396099+0000", 2024-09-16T14:48:33.872 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5038e-05 2024-09-16T14:48:33.872 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.872 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.872 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.872 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.872 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:33.872 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.406548+0000", 2024-09-16T14:48:33.872 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.442589654000001, 2024-09-16T14:48:33.872 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0070184770000000004, 2024-09-16T14:48:33.872 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.872 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.872 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.873 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.873 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.873 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 32 2024-09-16T14:48:33.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.873 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.873 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.873 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.873 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406548+0000", 2024-09-16T14:48:33.873 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.873 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.873 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.873 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406548+0000", 2024-09-16T14:48:33.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.874 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.874 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406548+0000", 2024-09-16T14:48:33.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.874 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.874 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406599+0000", 2024-09-16T14:48:33.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.1523000000000001e-05 2024-09-16T14:48:33.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.875 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.875 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406606+0000", 2024-09-16T14:48:33.875 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6059999999999999e-06 2024-09-16T14:48:33.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.875 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.875 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.875 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406610+0000", 2024-09-16T14:48:33.875 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1420000000000003e-06 2024-09-16T14:48:33.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.875 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.875 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.875 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.407344+0000", 2024-09-16T14:48:33.875 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00073374699999999998 2024-09-16T14:48:33.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.876 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.407433+0000", 2024-09-16T14:48:33.876 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9081999999999999e-05 2024-09-16T14:48:33.876 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.876 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.407568+0000", 2024-09-16T14:48:33.876 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00013519000000000001 2024-09-16T14:48:33.876 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.876 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.410018+0000", 2024-09-16T14:48:33.877 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002450149 2024-09-16T14:48:33.877 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.877 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.877 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.877 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413478+0000", 2024-09-16T14:48:33.877 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034598440000000001 2024-09-16T14:48:33.877 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.877 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.877 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.877 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413532+0000", 2024-09-16T14:48:33.877 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.4215999999999997e-05 2024-09-16T14:48:33.877 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.877 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.878 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.878 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413567+0000", 2024-09-16T14:48:33.878 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.4612e-05 2024-09-16T14:48:33.878 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.878 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.878 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.878 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.878 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.878 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:33.878 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.406616+0000", 2024-09-16T14:48:33.878 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.442522136999999, 2024-09-16T14:48:33.878 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0070467129999999996, 2024-09-16T14:48:33.878 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.879 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.879 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.879 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.879 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.879 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 33 2024-09-16T14:48:33.879 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.879 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.879 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.879 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406616+0000", 2024-09-16T14:48:33.879 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.879 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.879 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.880 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406616+0000", 2024-09-16T14:48:33.880 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.880 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.880 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.880 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.880 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406615+0000", 2024-09-16T14:48:33.880 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.880 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.880 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.880 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.880 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406618+0000", 2024-09-16T14:48:33.880 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5339999999999998e-06 2024-09-16T14:48:33.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.881 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.881 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.881 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406621+0000", 2024-09-16T14:48:33.881 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.4340000000000001e-06 2024-09-16T14:48:33.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.881 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.881 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.881 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406625+0000", 2024-09-16T14:48:33.881 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6320000000000001e-06 2024-09-16T14:48:33.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.881 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.881 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.882 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.407856+0000", 2024-09-16T14:48:33.882 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001230769 2024-09-16T14:48:33.882 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.882 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.882 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.882 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.407904+0000", 2024-09-16T14:48:33.882 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.799e-05 2024-09-16T14:48:33.882 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.882 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.882 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.882 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.407987+0000", 2024-09-16T14:48:33.882 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3510999999999994e-05 2024-09-16T14:48:33.883 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.883 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.410021+0000", 2024-09-16T14:48:33.883 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0020337440000000001 2024-09-16T14:48:33.883 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.883 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413605+0000", 2024-09-16T14:48:33.883 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0035842650000000001 2024-09-16T14:48:33.883 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.884 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413640+0000", 2024-09-16T14:48:33.884 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5064000000000002e-05 2024-09-16T14:48:33.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.884 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.884 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.884 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413662+0000", 2024-09-16T14:48:33.884 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2218999999999999e-05 2024-09-16T14:48:33.884 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.884 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.884 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.884 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.884 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:33.884 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.406628+0000", 2024-09-16T14:48:33.885 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.442510027999999, 2024-09-16T14:48:33.885 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0071159200000000004, 2024-09-16T14:48:33.885 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.885 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.885 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.885 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.885 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.885 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 34 2024-09-16T14:48:33.885 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.885 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.885 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.885 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.885 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406628+0000", 2024-09-16T14:48:33.886 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.888 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.889 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406628+0000", 2024-09-16T14:48:33.889 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.889 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.889 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406628+0000", 2024-09-16T14:48:33.889 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:48:33.889 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.889 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406630+0000", 2024-09-16T14:48:33.889 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.266e-06 2024-09-16T14:48:33.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.890 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.890 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.890 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406636+0000", 2024-09-16T14:48:33.890 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.7810000000000002e-06 2024-09-16T14:48:33.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.890 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.890 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.890 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406639+0000", 2024-09-16T14:48:33.890 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8449999999999999e-06 2024-09-16T14:48:33.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.890 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.890 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.891 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.408361+0000", 2024-09-16T14:48:33.891 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001722847 2024-09-16T14:48:33.891 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.891 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.891 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.891 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.408406+0000", 2024-09-16T14:48:33.891 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4984999999999997e-05 2024-09-16T14:48:33.891 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.891 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.891 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.891 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.408486+0000", 2024-09-16T14:48:33.891 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0028e-05 2024-09-16T14:48:33.891 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.891 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.892 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.410024+0000", 2024-09-16T14:48:33.892 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015372750000000001 2024-09-16T14:48:33.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.892 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.892 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413696+0000", 2024-09-16T14:48:33.892 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036724959999999999 2024-09-16T14:48:33.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.892 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.892 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413724+0000", 2024-09-16T14:48:33.892 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7563e-05 2024-09-16T14:48:33.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.893 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.893 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.893 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413744+0000", 2024-09-16T14:48:33.893 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0071000000000001e-05 2024-09-16T14:48:33.893 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.893 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.893 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.893 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.893 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.893 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:33.893 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.406644+0000", 2024-09-16T14:48:33.893 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.442493836000001, 2024-09-16T14:48:33.893 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0071766360000000001, 2024-09-16T14:48:33.894 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.894 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.894 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.894 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.894 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.894 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 35 2024-09-16T14:48:33.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.894 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.894 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.894 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.894 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406644+0000", 2024-09-16T14:48:33.894 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406644+0000", 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406644+0000", 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406674+0000", 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0513000000000001e-05 2024-09-16T14:48:33.895 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406679+0000", 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6550000000000003e-06 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406683+0000", 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.7500000000000001e-06 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.408720+0000", 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0020375860000000001 2024-09-16T14:48:33.896 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.897 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.897 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.897 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.408800+0000", 2024-09-16T14:48:33.897 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9350000000000004e-05 2024-09-16T14:48:33.897 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.897 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.897 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.897 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.408938+0000", 2024-09-16T14:48:33.897 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00013873999999999999 2024-09-16T14:48:33.897 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.897 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.897 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.897 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.410027+0000", 2024-09-16T14:48:33.897 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001088969 2024-09-16T14:48:33.898 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.898 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.898 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.898 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413779+0000", 2024-09-16T14:48:33.898 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003751561 2024-09-16T14:48:33.898 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.898 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.898 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.898 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413803+0000", 2024-09-16T14:48:33.898 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3907e-05 2024-09-16T14:48:33.898 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.898 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.899 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.899 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413821+0000", 2024-09-16T14:48:33.899 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7944999999999998e-05 2024-09-16T14:48:33.899 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.899 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.899 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.899 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.899 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.899 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:33.899 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:48:33.899 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.442451827999999, 2024-09-16T14:48:33.899 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0072029349999999997, 2024-09-16T14:48:33.899 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.899 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.900 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.900 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.900 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.900 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 36 2024-09-16T14:48:33.900 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.900 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.900 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.900 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.900 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:48:33.900 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.900 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.900 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.900 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.901 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:48:33.901 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.901 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.901 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.901 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.901 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:48:33.901 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.901 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.901 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.901 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.901 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406695+0000", 2024-09-16T14:48:33.901 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1530000000000003e-06 2024-09-16T14:48:33.901 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.902 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.902 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.902 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406700+0000", 2024-09-16T14:48:33.902 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7079999999999996e-06 2024-09-16T14:48:33.902 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.902 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.902 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.902 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406702+0000", 2024-09-16T14:48:33.902 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7410000000000001e-06 2024-09-16T14:48:33.902 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.902 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.902 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.902 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409398+0000", 2024-09-16T14:48:33.902 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0026957420000000001 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409441+0000", 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3071999999999997e-05 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409512+0000", 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0871999999999995e-05 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.410156+0000", 2024-09-16T14:48:33.903 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00064395800000000003 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413850+0000", 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036938959999999999 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413872+0000", 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2195e-05 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413889+0000", 2024-09-16T14:48:33.904 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6948000000000001e-05 2024-09-16T14:48:33.905 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.905 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.905 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.905 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.905 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.905 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:33.905 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.406706+0000", 2024-09-16T14:48:33.905 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.44243225, 2024-09-16T14:48:33.905 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0072520049999999997, 2024-09-16T14:48:33.905 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.905 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.905 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.905 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.905 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.906 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 37 2024-09-16T14:48:33.906 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.906 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.906 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.906 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.906 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406706+0000", 2024-09-16T14:48:33.906 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.906 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.906 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.906 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.906 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406706+0000", 2024-09-16T14:48:33.906 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.906 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.906 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406705+0000", 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406708+0000", 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.57e-06 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406712+0000", 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.425e-06 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.907 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.908 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.908 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406715+0000", 2024-09-16T14:48:33.908 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7700000000000002e-06 2024-09-16T14:48:33.908 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.908 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.908 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.908 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409721+0000", 2024-09-16T14:48:33.908 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003006327 2024-09-16T14:48:33.908 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.908 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.908 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.908 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409770+0000", 2024-09-16T14:48:33.908 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8022000000000002e-05 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409853+0000", 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3604999999999997e-05 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.410391+0000", 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00053785400000000004 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413911+0000", 2024-09-16T14:48:33.909 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0035205240000000001 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413939+0000", 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7032000000000001e-05 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413958+0000", 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9147000000000002e-05 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.910 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.911 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:33.911 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.620547+0000", 2024-09-16T14:48:33.911 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.228590619, 2024-09-16T14:48:33.911 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0071149270000000001, 2024-09-16T14:48:33.911 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.911 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.911 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.911 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.911 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.911 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 115 2024-09-16T14:48:33.911 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.911 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.911 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.911 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620547+0000", 2024-09-16T14:48:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620547+0000", 2024-09-16T14:48:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620547+0000", 2024-09-16T14:48:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.912 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.913 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.913 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620616+0000", 2024-09-16T14:48:33.913 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9244000000000003e-05 2024-09-16T14:48:33.913 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.913 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.913 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.913 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620624+0000", 2024-09-16T14:48:33.913 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0979999999999994e-06 2024-09-16T14:48:33.913 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.913 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.913 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.913 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620631+0000", 2024-09-16T14:48:33.913 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.81e-06 2024-09-16T14:48:33.914 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.914 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.914 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.914 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625202+0000", 2024-09-16T14:48:33.914 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045710350000000002 2024-09-16T14:48:33.914 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.914 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.914 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.914 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625255+0000", 2024-09-16T14:48:33.914 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3303000000000001e-05 2024-09-16T14:48:33.914 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.914 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.914 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625330+0000", 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4486000000000005e-05 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627422+0000", 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0020923259999999998 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627596+0000", 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017343400000000001 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.915 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.916 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627626+0000", 2024-09-16T14:48:33.916 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0094999999999999e-05 2024-09-16T14:48:33.916 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.916 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.916 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.916 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627662+0000", 2024-09-16T14:48:33.916 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6603000000000002e-05 2024-09-16T14:48:33.916 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.916 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.916 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.916 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.916 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.916 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:33.916 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.620638+0000", 2024-09-16T14:48:33.917 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.228500147, 2024-09-16T14:48:33.917 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0078265469999999997, 2024-09-16T14:48:33.917 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.917 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.917 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.917 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.917 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.917 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 116 2024-09-16T14:48:33.917 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.917 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.917 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.917 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.917 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620638+0000", 2024-09-16T14:48:33.917 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620638+0000", 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620637+0000", 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620641+0000", 2024-09-16T14:48:33.918 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9360000000000001e-06 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620647+0000", 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3339999999999998e-06 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620652+0000", 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0950000000000003e-06 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625724+0000", 2024-09-16T14:48:33.919 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050719370000000003 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625768+0000", 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4039000000000003e-05 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625836+0000", 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7181999999999995e-05 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627424+0000", 2024-09-16T14:48:33.920 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015887119999999999 2024-09-16T14:48:33.921 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.921 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.921 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.921 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628421+0000", 2024-09-16T14:48:33.921 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00099648699999999989 2024-09-16T14:48:33.921 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.921 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.921 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.921 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628446+0000", 2024-09-16T14:48:33.921 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5307999999999999e-05 2024-09-16T14:48:33.921 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.921 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.921 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.921 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628464+0000", 2024-09-16T14:48:33.922 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8170999999999999e-05 2024-09-16T14:48:33.922 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.922 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.922 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.922 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.922 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.922 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:33.922 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:48:33.922 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.228480423000001, 2024-09-16T14:48:33.922 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0079223479999999992, 2024-09-16T14:48:33.922 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.922 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.922 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.922 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.923 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.923 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 117 2024-09-16T14:48:33.923 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.923 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.923 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.923 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.923 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:48:33.923 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.923 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.923 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.923 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.923 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:48:33.923 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.923 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.924 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.924 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.924 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:48:33.924 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.924 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.924 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.924 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.924 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620662+0000", 2024-09-16T14:48:33.924 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7609999999999998e-06 2024-09-16T14:48:33.924 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.924 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.924 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.924 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620668+0000", 2024-09-16T14:48:33.925 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.84e-06 2024-09-16T14:48:33.925 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.925 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.925 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.925 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620673+0000", 2024-09-16T14:48:33.925 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.468e-06 2024-09-16T14:48:33.925 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.925 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.925 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.925 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.626022+0000", 2024-09-16T14:48:33.925 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053484680000000003 2024-09-16T14:48:33.925 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.925 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.926 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.926 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.626071+0000", 2024-09-16T14:48:33.926 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9854000000000001e-05 2024-09-16T14:48:33.926 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.926 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.926 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.926 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.626145+0000", 2024-09-16T14:48:33.926 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.3159999999999997e-05 2024-09-16T14:48:33.926 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.926 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.926 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.926 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627426+0000", 2024-09-16T14:48:33.926 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001281847 2024-09-16T14:48:33.926 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.927 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.927 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.927 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628501+0000", 2024-09-16T14:48:33.927 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0010748019999999999 2024-09-16T14:48:33.927 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.927 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.927 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.927 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628541+0000", 2024-09-16T14:48:33.927 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9598999999999998e-05 2024-09-16T14:48:33.927 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.927 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.927 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.927 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628580+0000", 2024-09-16T14:48:33.928 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8977999999999998e-05 2024-09-16T14:48:33.928 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.928 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.928 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.928 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.928 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.928 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:33.928 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:48:33.928 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.457213530000001, 2024-09-16T14:48:33.928 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0081240189999999997, 2024-09-16T14:48:33.928 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.928 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.929 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.929 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.929 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.929 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 158 2024-09-16T14:48:33.929 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.929 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.929 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.929 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.929 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:48:33.929 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.930 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.930 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.930 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.930 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:48:33.930 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.930 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.930 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.930 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.930 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:48:33.930 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.930 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.930 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.930 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.930 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392007+0000", 2024-09-16T14:48:33.931 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3108000000000005e-05 2024-09-16T14:48:33.931 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.931 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.931 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.931 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392014+0000", 2024-09-16T14:48:33.931 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6170000000000003e-06 2024-09-16T14:48:33.931 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.931 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.931 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.931 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392017+0000", 2024-09-16T14:48:33.931 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3589999999999999e-06 2024-09-16T14:48:33.931 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.931 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394165+0000", 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021481270000000001 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394238+0000", 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2366999999999993e-05 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394347+0000", 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000109907 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.932 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.933 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.933 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396066+0000", 2024-09-16T14:48:33.933 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001718731 2024-09-16T14:48:33.933 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.933 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.933 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.934 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.399984+0000", 2024-09-16T14:48:33.934 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0039181850000000002 2024-09-16T14:48:33.934 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.934 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.934 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.934 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400027+0000", 2024-09-16T14:48:33.934 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2184000000000001e-05 2024-09-16T14:48:33.934 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.934 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.934 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.934 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400048+0000", 2024-09-16T14:48:33.935 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1897999999999999e-05 2024-09-16T14:48:33.935 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.935 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.935 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.935 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.935 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.935 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:33.935 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:48:33.935 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.457116736, 2024-09-16T14:48:33.935 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0081462830000000007, 2024-09-16T14:48:33.935 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.935 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.935 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.935 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.936 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.936 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 159 2024-09-16T14:48:33.936 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.936 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.936 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.936 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.936 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:48:33.936 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.936 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.936 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.936 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.936 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:48:33.936 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.936 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.937 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.937 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.937 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:48:33.937 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.937 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.937 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.937 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.937 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392023+0000", 2024-09-16T14:48:33.937 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:48:33.937 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.937 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.937 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.937 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392035+0000", 2024-09-16T14:48:33.937 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2207e-05 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392040+0000", 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7450000000000003e-06 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394756+0000", 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0027154010000000001 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394799+0000", 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3921999999999997e-05 2024-09-16T14:48:33.938 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394867+0000", 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7770999999999999e-05 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396922+0000", 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0020549819999999999 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400094+0000", 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031720979999999999 2024-09-16T14:48:33.939 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.940 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.940 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.940 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400134+0000", 2024-09-16T14:48:33.940 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9449999999999997e-05 2024-09-16T14:48:33.940 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.940 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.940 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.940 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400167+0000", 2024-09-16T14:48:33.940 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3711000000000003e-05 2024-09-16T14:48:33.940 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.940 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.940 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.940 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.940 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:160 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.457094503, 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082045659999999999, 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 160 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:48:33.941 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392045+0000", 2024-09-16T14:48:33.942 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.283e-06 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392050+0000", 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1099999999999996e-06 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392052+0000", 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4660000000000002e-06 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395234+0000", 2024-09-16T14:48:33.943 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031823260000000001 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395283+0000", 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8347000000000003e-05 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395364+0000", 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0853999999999995e-05 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396928+0000", 2024-09-16T14:48:33.944 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015642239999999999 2024-09-16T14:48:33.945 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.945 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.945 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.945 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400202+0000", 2024-09-16T14:48:33.945 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032747359999999999 2024-09-16T14:48:33.945 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.945 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.945 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.945 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400227+0000", 2024-09-16T14:48:33.945 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4079000000000001e-05 2024-09-16T14:48:33.945 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.945 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.945 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.945 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400248+0000", 2024-09-16T14:48:33.946 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1444e-05 2024-09-16T14:48:33.946 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.946 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.946 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.946 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.946 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.946 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:161 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:33.946 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:48:33.946 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.457083067999999, 2024-09-16T14:48:33.946 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082675720000000008, 2024-09-16T14:48:33.946 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.946 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.946 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.946 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 161 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392122+0000", 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7698000000000001e-05 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392136+0000", 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3464e-05 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.948 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.949 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.949 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392140+0000", 2024-09-16T14:48:33.949 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0790000000000002e-06 2024-09-16T14:48:33.949 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.949 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.949 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.949 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395579+0000", 2024-09-16T14:48:33.949 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034393219999999999 2024-09-16T14:48:33.949 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.949 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.949 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.949 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395624+0000", 2024-09-16T14:48:33.949 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4946e-05 2024-09-16T14:48:33.950 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.950 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.950 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.950 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395692+0000", 2024-09-16T14:48:33.950 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7843999999999996e-05 2024-09-16T14:48:33.950 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.950 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.950 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.950 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397834+0000", 2024-09-16T14:48:33.950 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021421529999999999 2024-09-16T14:48:33.950 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.950 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.950 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.950 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400281+0000", 2024-09-16T14:48:33.951 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002447238 2024-09-16T14:48:33.951 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.951 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.951 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.951 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400305+0000", 2024-09-16T14:48:33.951 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3428999999999999e-05 2024-09-16T14:48:33.951 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.951 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.951 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.951 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400322+0000", 2024-09-16T14:48:33.951 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7632e-05 2024-09-16T14:48:33.951 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.951 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.951 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.952 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.952 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.952 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:33.952 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:48:33.952 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.456993789, 2024-09-16T14:48:33.952 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.008260814, 2024-09-16T14:48:33.952 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.952 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.952 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.952 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.952 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.952 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 162 2024-09-16T14:48:33.952 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.952 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.953 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.953 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.953 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:48:33.953 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.953 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.953 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.953 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.953 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:48:33.953 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.953 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.953 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.953 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.953 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:48:33.954 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.954 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.954 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.954 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.954 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392146+0000", 2024-09-16T14:48:33.954 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.526e-06 2024-09-16T14:48:33.954 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.954 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.954 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.954 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392151+0000", 2024-09-16T14:48:33.954 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2980000000000002e-06 2024-09-16T14:48:33.954 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.954 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.955 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.955 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392153+0000", 2024-09-16T14:48:33.955 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5409999999999999e-06 2024-09-16T14:48:33.955 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.955 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.955 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.955 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396103+0000", 2024-09-16T14:48:33.955 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0039497559999999996 2024-09-16T14:48:33.955 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.955 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.955 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.955 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396188+0000", 2024-09-16T14:48:33.955 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5038999999999997e-05 2024-09-16T14:48:33.956 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.956 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.956 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.956 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396315+0000", 2024-09-16T14:48:33.956 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000126895 2024-09-16T14:48:33.956 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.956 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.956 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.956 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397839+0000", 2024-09-16T14:48:33.956 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015246859999999999 2024-09-16T14:48:33.956 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.956 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.956 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.956 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400356+0000", 2024-09-16T14:48:33.957 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002516524 2024-09-16T14:48:33.957 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.957 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.957 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.957 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400379+0000", 2024-09-16T14:48:33.957 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2793e-05 2024-09-16T14:48:33.957 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.957 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.957 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.957 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400405+0000", 2024-09-16T14:48:33.957 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6140999999999999e-05 2024-09-16T14:48:33.957 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.957 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.957 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.456982397999999, 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083183530000000006, 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 163 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.958 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392155+0000", 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.959 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.960 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.960 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392159+0000", 2024-09-16T14:48:33.960 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0459999999999999e-06 2024-09-16T14:48:33.960 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.960 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.960 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.960 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392162+0000", 2024-09-16T14:48:33.960 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:48:33.960 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.960 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.960 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.960 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392165+0000", 2024-09-16T14:48:33.960 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.424e-06 2024-09-16T14:48:33.960 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.961 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.961 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.961 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396578+0000", 2024-09-16T14:48:33.961 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044136139999999997 2024-09-16T14:48:33.961 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.961 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.961 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.961 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396625+0000", 2024-09-16T14:48:33.961 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6486000000000002e-05 2024-09-16T14:48:33.961 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.961 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.961 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.961 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396694+0000", 2024-09-16T14:48:33.961 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9049999999999998e-05 2024-09-16T14:48:33.962 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.962 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.962 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.962 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397843+0000", 2024-09-16T14:48:33.962 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011485740000000001 2024-09-16T14:48:33.962 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.962 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.962 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.962 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400438+0000", 2024-09-16T14:48:33.962 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025950909999999999 2024-09-16T14:48:33.963 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.963 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.963 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.963 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400458+0000", 2024-09-16T14:48:33.963 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0248e-05 2024-09-16T14:48:33.963 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.963 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.963 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.963 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400474+0000", 2024-09-16T14:48:33.963 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5999999999999999e-05 2024-09-16T14:48:33.963 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.963 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.963 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.964 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.964 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.964 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:33.964 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:48:33.964 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.456970801000001, 2024-09-16T14:48:33.964 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083881519999999994, 2024-09-16T14:48:33.964 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.964 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.964 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.964 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.964 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.964 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 164 2024-09-16T14:48:33.964 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.965 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.965 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.965 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.965 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:48:33.965 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.965 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.965 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.965 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.965 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:48:33.965 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.965 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.965 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.965 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.965 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:48:33.966 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:48:33.966 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.966 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.966 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.966 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392212+0000", 2024-09-16T14:48:33.966 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5108999999999998e-05 2024-09-16T14:48:33.966 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.966 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.966 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.966 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392216+0000", 2024-09-16T14:48:33.966 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2549999999999999e-06 2024-09-16T14:48:33.966 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.966 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.966 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.967 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392220+0000", 2024-09-16T14:48:33.967 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3850000000000002e-06 2024-09-16T14:48:33.967 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.967 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.967 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.967 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396961+0000", 2024-09-16T14:48:33.967 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.004740976 2024-09-16T14:48:33.967 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.967 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.967 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.967 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397029+0000", 2024-09-16T14:48:33.967 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7889e-05 2024-09-16T14:48:33.967 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.967 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.968 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.968 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397131+0000", 2024-09-16T14:48:33.968 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000102315 2024-09-16T14:48:33.968 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.968 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.968 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.968 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400259+0000", 2024-09-16T14:48:33.968 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031285229999999998 2024-09-16T14:48:33.968 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.968 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.968 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.968 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400501+0000", 2024-09-16T14:48:33.968 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00024211999999999999 2024-09-16T14:48:33.969 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.969 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.969 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.969 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400530+0000", 2024-09-16T14:48:33.969 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8935999999999999e-05 2024-09-16T14:48:33.969 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.969 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.969 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.969 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400555+0000", 2024-09-16T14:48:33.969 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4861000000000001e-05 2024-09-16T14:48:33.969 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.969 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.969 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.969 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.970 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.970 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:33.970 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:48:33.970 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.456915036, 2024-09-16T14:48:33.970 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010593673, 2024-09-16T14:48:33.970 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:33.970 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:33.970 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:33.970 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:33.970 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:33.970 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 165 2024-09-16T14:48:33.970 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.970 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:33.970 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.971 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:33.971 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:48:33.971 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.971 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.971 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.971 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:33.971 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:48:33.971 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:33.971 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.971 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.971 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:33.971 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:48:33.971 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:33.971 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.972 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.972 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:33.972 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392226+0000", 2024-09-16T14:48:33.972 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2789999999999999e-06 2024-09-16T14:48:33.972 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.972 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.972 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:33.972 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392234+0000", 2024-09-16T14:48:33.972 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2369999999999992e-06 2024-09-16T14:48:33.972 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.972 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.972 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:33.972 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392237+0000", 2024-09-16T14:48:33.973 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8619999999999999e-06 2024-09-16T14:48:33.973 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.973 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.973 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:33.973 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397503+0000", 2024-09-16T14:48:33.973 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0052656220000000002 2024-09-16T14:48:33.973 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.973 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.973 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:33.973 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397558+0000", 2024-09-16T14:48:33.973 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5386999999999997e-05 2024-09-16T14:48:33.973 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.973 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.973 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:33.974 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397628+0000", 2024-09-16T14:48:33.974 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0119999999999999e-05 2024-09-16T14:48:33.974 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.974 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.974 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:33.974 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402533+0000", 2024-09-16T14:48:33.974 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049046200000000002 2024-09-16T14:48:33.974 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.974 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.974 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:33.974 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402778+0000", 2024-09-16T14:48:33.974 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000245348 2024-09-16T14:48:33.974 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.974 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.975 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:33.975 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402800+0000", 2024-09-16T14:48:33.975 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2277e-05 2024-09-16T14:48:33.975 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:33.975 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:33.975 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:33.975 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402817+0000", 2024-09-16T14:48:33.975 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6265000000000001e-05 2024-09-16T14:48:33.975 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.975 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.975 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.975 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:33.975 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:33.975 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:33.976 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:48:33.977 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:33.850+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:33.978 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:33.976+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:33.978 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'unique_pool_0' created 2024-09-16T14:48:33.979 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:48:33.979 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-16T14:48:34.077 INFO:teuthology.orchestra.run.smithi111.stdout:2.7 2024-09-16T14:48:34.078 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:34.076+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:34.094 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:34.095 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:34.095 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:34.095 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:34.103 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:48:34.187 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:34.186+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:34.245 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:34.246 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:34.246 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:34.254 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:48:34.279 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:34.278+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:34.381 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:34.379+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:34.398 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:34.398 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:34.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:34.398 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [] 2024-09-16T14:48:34.398 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:34.407 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:48:34.491 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:34.490+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:34.549 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:34.550 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:34.550 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:34.550 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:34.558 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:48:34.599 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:34.598+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:34.601 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op export --pgid 2.7 --file /var/log/ceph/exp.2.7.6 2024-09-16T14:48:34.684 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:34.682+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:34.700 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:34.700 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:34.700 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:34.700 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:34.700 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:34.709 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:48:34.785 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:34.784+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:34.810 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:34.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:34.811 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [] 2024-09-16T14:48:34.811 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:34.819 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:48:34.867 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is set 2024-09-16T14:48:34.867 INFO:teuthology.orchestra.run.smithi040.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-16T14:48:34.886 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:34.885+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:34.929 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:34.929 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:34.930 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:34.930 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:34.938 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:48:34.987 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:34.986+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:35.041 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:35.041 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:35.042 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:35.042 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:35.042 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:35.050 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:48:35.088 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:35.087+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:35.189 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:35.188+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:35.192 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:35.192 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:35.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:35.193 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [] 2024-09-16T14:48:35.193 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:35.202 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:48:35.290 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:35.289+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:35.304 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:35.304 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:35.305 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:35.313 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:48:35.391 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:35.390+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:35.414 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:35.414 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:35.414 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:35.422 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:48:35.492 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:35.491+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:35.527 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:4 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'2, mlcod=0'0)", 2024-09-16T14:48:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.364271+0000", 2024-09-16T14:48:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.159822295, 2024-09-16T14:48:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036960159999999999, 2024-09-16T14:48:35.527 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364271+0000", 2024-09-16T14:48:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364271+0000", 2024-09-16T14:48:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364270+0000", 2024-09-16T14:48:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364277+0000", 2024-09-16T14:48:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7519999999999999e-06 2024-09-16T14:48:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364290+0000", 2024-09-16T14:48:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3159999999999999e-05 2024-09-16T14:48:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364297+0000", 2024-09-16T14:48:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.776e-06 2024-09-16T14:48:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364342+0000", 2024-09-16T14:48:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4601999999999997e-05 2024-09-16T14:48:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364373+0000", 2024-09-16T14:48:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1878000000000002e-05 2024-09-16T14:48:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.367937+0000", 2024-09-16T14:48:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003563325 2024-09-16T14:48:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.367967+0000", 2024-09-16T14:48:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.046e-05 2024-09-16T14:48:35.531 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:18 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'8, mlcod=19'5)", 2024-09-16T14:48:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.380607+0000", 2024-09-16T14:48:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.143486951, 2024-09-16T14:48:35.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042947610000000002, 2024-09-16T14:48:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380607+0000", 2024-09-16T14:48:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380607+0000", 2024-09-16T14:48:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380606+0000", 2024-09-16T14:48:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380611+0000", 2024-09-16T14:48:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:48:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380620+0000", 2024-09-16T14:48:35.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5709999999999998e-06 2024-09-16T14:48:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380625+0000", 2024-09-16T14:48:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1580000000000004e-06 2024-09-16T14:48:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380659+0000", 2024-09-16T14:48:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3587000000000002e-05 2024-09-16T14:48:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380679+0000", 2024-09-16T14:48:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0242999999999999e-05 2024-09-16T14:48:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.384855+0000", 2024-09-16T14:48:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041758089999999999 2024-09-16T14:48:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.384901+0000", 2024-09-16T14:48:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6634000000000001e-05 2024-09-16T14:48:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2024-09-16T14:48:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.389034+0000", 2024-09-16T14:48:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.135059371000001, 2024-09-16T14:48:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066701460000000001, 2024-09-16T14:48:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.537 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389034+0000", 2024-09-16T14:48:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389034+0000", 2024-09-16T14:48:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389033+0000", 2024-09-16T14:48:35.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389110+0000", 2024-09-16T14:48:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6965000000000006e-05 2024-09-16T14:48:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389122+0000", 2024-09-16T14:48:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.166e-05 2024-09-16T14:48:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389129+0000", 2024-09-16T14:48:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8120000000000002e-06 2024-09-16T14:48:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389166+0000", 2024-09-16T14:48:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6986999999999997e-05 2024-09-16T14:48:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389186+0000", 2024-09-16T14:48:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0043000000000001e-05 2024-09-16T14:48:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395613+0000", 2024-09-16T14:48:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064270109999999998 2024-09-16T14:48:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395704+0000", 2024-09-16T14:48:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1595000000000006e-05 2024-09-16T14:48:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:22 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'12, mlcod=19'8)", 2024-09-16T14:48:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.389513+0000", 2024-09-16T14:48:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.134580980000001, 2024-09-16T14:48:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062632759999999999, 2024-09-16T14:48:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389513+0000", 2024-09-16T14:48:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389513+0000", 2024-09-16T14:48:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389512+0000", 2024-09-16T14:48:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389520+0000", 2024-09-16T14:48:35.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2609999999999993e-06 2024-09-16T14:48:35.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389533+0000", 2024-09-16T14:48:35.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2860999999999999e-05 2024-09-16T14:48:35.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389539+0000", 2024-09-16T14:48:35.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2369999999999997e-06 2024-09-16T14:48:35.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389568+0000", 2024-09-16T14:48:35.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9411999999999999e-05 2024-09-16T14:48:35.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389584+0000", 2024-09-16T14:48:35.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5891000000000001e-05 2024-09-16T14:48:35.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395712+0000", 2024-09-16T14:48:35.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061272360000000003 2024-09-16T14:48:35.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395776+0000", 2024-09-16T14:48:35.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4290999999999998e-05 2024-09-16T14:48:35.546 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.546 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.546 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.546 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:23 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'13, mlcod=19'11)", 2024-09-16T14:48:35.546 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.396595+0000", 2024-09-16T14:48:35.546 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.127498099, 2024-09-16T14:48:35.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003499107, 2024-09-16T14:48:35.547 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.547 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.547 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396595+0000", 2024-09-16T14:48:35.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396595+0000", 2024-09-16T14:48:35.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396595+0000", 2024-09-16T14:48:35.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:35.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396601+0000", 2024-09-16T14:48:35.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4899999999999997e-06 2024-09-16T14:48:35.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396613+0000", 2024-09-16T14:48:35.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1905e-05 2024-09-16T14:48:35.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396618+0000", 2024-09-16T14:48:35.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1039999999999998e-06 2024-09-16T14:48:35.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396656+0000", 2024-09-16T14:48:35.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8408000000000002e-05 2024-09-16T14:48:35.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396669+0000", 2024-09-16T14:48:35.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2362e-05 2024-09-16T14:48:35.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.400046+0000", 2024-09-16T14:48:35.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0033771500000000002 2024-09-16T14:48:35.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.400095+0000", 2024-09-16T14:48:35.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8581999999999997e-05 2024-09-16T14:48:35.551 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.551 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.551 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.551 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:31 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'17, mlcod=19'14)", 2024-09-16T14:48:35.551 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.407210+0000", 2024-09-16T14:48:35.551 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.116883647, 2024-09-16T14:48:35.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031163440000000001, 2024-09-16T14:48:35.551 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.551 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.551 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407210+0000", 2024-09-16T14:48:35.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407210+0000", 2024-09-16T14:48:35.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407209+0000", 2024-09-16T14:48:35.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:35.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407216+0000", 2024-09-16T14:48:35.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8340000000000001e-06 2024-09-16T14:48:35.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407229+0000", 2024-09-16T14:48:35.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2828e-05 2024-09-16T14:48:35.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407235+0000", 2024-09-16T14:48:35.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3520000000000003e-06 2024-09-16T14:48:35.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407276+0000", 2024-09-16T14:48:35.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1366999999999999e-05 2024-09-16T14:48:35.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407301+0000", 2024-09-16T14:48:35.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5034000000000001e-05 2024-09-16T14:48:35.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.410294+0000", 2024-09-16T14:48:35.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002992303 2024-09-16T14:48:35.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.410326+0000", 2024-09-16T14:48:35.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2549000000000003e-05 2024-09-16T14:48:35.555 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.555 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.555 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.556 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:32 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'18, mlcod=19'14)", 2024-09-16T14:48:35.556 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.407675+0000", 2024-09-16T14:48:35.556 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.11641814, 2024-09-16T14:48:35.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0056403440000000003, 2024-09-16T14:48:35.556 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.556 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.556 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407675+0000", 2024-09-16T14:48:35.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407675+0000", 2024-09-16T14:48:35.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407675+0000", 2024-09-16T14:48:35.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:35.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407750+0000", 2024-09-16T14:48:35.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4728999999999996e-05 2024-09-16T14:48:35.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407759+0000", 2024-09-16T14:48:35.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3519999999999996e-06 2024-09-16T14:48:35.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407764+0000", 2024-09-16T14:48:35.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.994e-06 2024-09-16T14:48:35.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407797+0000", 2024-09-16T14:48:35.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2641999999999997e-05 2024-09-16T14:48:35.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407816+0000", 2024-09-16T14:48:35.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9219e-05 2024-09-16T14:48:35.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413272+0000", 2024-09-16T14:48:35.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054566129999999999 2024-09-16T14:48:35.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413316+0000", 2024-09-16T14:48:35.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3350999999999999e-05 2024-09-16T14:48:35.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.560 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.560 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2024-09-16T14:48:35.560 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.408061+0000", 2024-09-16T14:48:35.560 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.116032081, 2024-09-16T14:48:35.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052755520000000002, 2024-09-16T14:48:35.561 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.561 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.561 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408061+0000", 2024-09-16T14:48:35.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408061+0000", 2024-09-16T14:48:35.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408061+0000", 2024-09-16T14:48:35.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:35.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408066+0000", 2024-09-16T14:48:35.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6210000000000001e-06 2024-09-16T14:48:35.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408077+0000", 2024-09-16T14:48:35.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0906e-05 2024-09-16T14:48:35.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408081+0000", 2024-09-16T14:48:35.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8689999999999997e-06 2024-09-16T14:48:35.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408114+0000", 2024-09-16T14:48:35.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2795e-05 2024-09-16T14:48:35.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408133+0000", 2024-09-16T14:48:35.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9017000000000001e-05 2024-09-16T14:48:35.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413318+0000", 2024-09-16T14:48:35.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051848040000000003 2024-09-16T14:48:35.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413337+0000", 2024-09-16T14:48:35.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9173000000000001e-05 2024-09-16T14:48:35.564 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.564 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.565 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.565 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:34 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'20, mlcod=19'15)", 2024-09-16T14:48:35.565 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.408607+0000", 2024-09-16T14:48:35.565 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.115486502, 2024-09-16T14:48:35.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047534309999999998, 2024-09-16T14:48:35.565 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.565 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.565 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408607+0000", 2024-09-16T14:48:35.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408607+0000", 2024-09-16T14:48:35.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408606+0000", 2024-09-16T14:48:35.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:35.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408613+0000", 2024-09-16T14:48:35.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7510000000000002e-06 2024-09-16T14:48:35.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408626+0000", 2024-09-16T14:48:35.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2819e-05 2024-09-16T14:48:35.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408631+0000", 2024-09-16T14:48:35.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1599999999999997e-06 2024-09-16T14:48:35.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408673+0000", 2024-09-16T14:48:35.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2274999999999998e-05 2024-09-16T14:48:35.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408692+0000", 2024-09-16T14:48:35.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9219e-05 2024-09-16T14:48:35.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413340+0000", 2024-09-16T14:48:35.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046472359999999999 2024-09-16T14:48:35.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413360+0000", 2024-09-16T14:48:35.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0882000000000001e-05 2024-09-16T14:48:35.569 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.569 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.569 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.569 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'15)", 2024-09-16T14:48:35.569 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.409042+0000", 2024-09-16T14:48:35.569 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.115051113, 2024-09-16T14:48:35.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.004339893, 2024-09-16T14:48:35.570 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.570 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.570 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409042+0000", 2024-09-16T14:48:35.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409042+0000", 2024-09-16T14:48:35.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409042+0000", 2024-09-16T14:48:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409112+0000", 2024-09-16T14:48:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0382000000000005e-05 2024-09-16T14:48:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409121+0000", 2024-09-16T14:48:35.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9560000000000003e-06 2024-09-16T14:48:35.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409126+0000", 2024-09-16T14:48:35.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3460000000000001e-06 2024-09-16T14:48:35.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409159+0000", 2024-09-16T14:48:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2351999999999998e-05 2024-09-16T14:48:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409178+0000", 2024-09-16T14:48:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9510000000000001e-05 2024-09-16T14:48:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413362+0000", 2024-09-16T14:48:35.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041835220000000003 2024-09-16T14:48:35.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413382+0000", 2024-09-16T14:48:35.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0463999999999999e-05 2024-09-16T14:48:35.574 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.574 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.574 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.574 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:36 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'22, mlcod=19'15)", 2024-09-16T14:48:35.574 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.409634+0000", 2024-09-16T14:48:35.574 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.114459683, 2024-09-16T14:48:35.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0037732019999999998, 2024-09-16T14:48:35.575 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.575 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.575 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409634+0000", 2024-09-16T14:48:35.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409634+0000", 2024-09-16T14:48:35.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409633+0000", 2024-09-16T14:48:35.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:35.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409641+0000", 2024-09-16T14:48:35.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8220000000000007e-06 2024-09-16T14:48:35.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409654+0000", 2024-09-16T14:48:35.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3427999999999999e-05 2024-09-16T14:48:35.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409669+0000", 2024-09-16T14:48:35.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4572999999999999e-05 2024-09-16T14:48:35.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409709+0000", 2024-09-16T14:48:35.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0392000000000002e-05 2024-09-16T14:48:35.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409728+0000", 2024-09-16T14:48:35.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8904000000000001e-05 2024-09-16T14:48:35.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413385+0000", 2024-09-16T14:48:35.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036566789999999999 2024-09-16T14:48:35.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413407+0000", 2024-09-16T14:48:35.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2357999999999998e-05 2024-09-16T14:48:35.578 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.579 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.579 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.579 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:37 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'23, mlcod=19'15)", 2024-09-16T14:48:35.579 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.409927+0000", 2024-09-16T14:48:35.579 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.114166802, 2024-09-16T14:48:35.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003502027, 2024-09-16T14:48:35.579 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.579 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.579 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409927+0000", 2024-09-16T14:48:35.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409927+0000", 2024-09-16T14:48:35.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409926+0000", 2024-09-16T14:48:35.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:35.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409939+0000", 2024-09-16T14:48:35.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3161e-05 2024-09-16T14:48:35.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409948+0000", 2024-09-16T14:48:35.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:48:35.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409952+0000", 2024-09-16T14:48:35.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9820000000000002e-06 2024-09-16T14:48:35.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.410009+0000", 2024-09-16T14:48:35.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7037000000000003e-05 2024-09-16T14:48:35.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.410023+0000", 2024-09-16T14:48:35.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4718e-05 2024-09-16T14:48:35.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413408+0000", 2024-09-16T14:48:35.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0033848889999999999 2024-09-16T14:48:35.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413429+0000", 2024-09-16T14:48:35.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0472999999999999e-05 2024-09-16T14:48:35.583 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.583 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.583 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.583 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:42 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'28, mlcod=19'26)", 2024-09-16T14:48:35.584 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.424909+0000", 2024-09-16T14:48:35.584 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.099184665999999, 2024-09-16T14:48:35.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003424524, 2024-09-16T14:48:35.584 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.584 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.584 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.424909+0000", 2024-09-16T14:48:35.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.424909+0000", 2024-09-16T14:48:35.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.424907+0000", 2024-09-16T14:48:35.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:48:35.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.424920+0000", 2024-09-16T14:48:35.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2713e-05 2024-09-16T14:48:35.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.424946+0000", 2024-09-16T14:48:35.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6193000000000001e-05 2024-09-16T14:48:35.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.424958+0000", 2024-09-16T14:48:35.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2242000000000001e-05 2024-09-16T14:48:35.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.425069+0000", 2024-09-16T14:48:35.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111192 2024-09-16T14:48:35.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.425104+0000", 2024-09-16T14:48:35.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4471999999999998e-05 2024-09-16T14:48:35.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.428288+0000", 2024-09-16T14:48:35.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003184259 2024-09-16T14:48:35.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.428333+0000", 2024-09-16T14:48:35.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5266000000000003e-05 2024-09-16T14:48:35.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.588 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.588 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:158 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'112, mlcod=20'107)", 2024-09-16T14:48:35.588 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:23.394419+0000", 2024-09-16T14:48:35.588 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.129674285, 2024-09-16T14:48:35.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054479120000000001, 2024-09-16T14:48:35.589 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.589 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.589 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394419+0000", 2024-09-16T14:48:35.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394419+0000", 2024-09-16T14:48:35.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394419+0000", 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394481+0000", 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2515999999999996e-05 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394489+0000", 2024-09-16T14:48:35.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394495+0000", 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3070000000000002e-06 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394518+0000", 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3818000000000001e-05 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394530+0000", 2024-09-16T14:48:35.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1158999999999999e-05 2024-09-16T14:48:35.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399835+0000", 2024-09-16T14:48:35.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053054829999999997 2024-09-16T14:48:35.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399867+0000", 2024-09-16T14:48:35.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2141000000000003e-05 2024-09-16T14:48:35.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.592 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.593 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:159 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'113, mlcod=20'107)", 2024-09-16T14:48:35.593 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:23.394933+0000", 2024-09-16T14:48:35.593 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.129160064000001, 2024-09-16T14:48:35.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049514190000000003, 2024-09-16T14:48:35.593 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.593 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.593 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394933+0000", 2024-09-16T14:48:35.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394933+0000", 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394933+0000", 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394938+0000", 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5050000000000001e-06 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394944+0000", 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6699999999999997e-06 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394949+0000", 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7439999999999998e-06 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394976+0000", 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6682e-05 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394997+0000", 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1291e-05 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399870+0000", 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00487339 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399885+0000", 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4619e-05 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.596 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.597 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.597 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:160 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'114, mlcod=20'109)", 2024-09-16T14:48:35.597 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:23.395425+0000", 2024-09-16T14:48:35.597 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.128668653, 2024-09-16T14:48:35.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044722859999999998, 2024-09-16T14:48:35.597 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.597 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.597 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395425+0000", 2024-09-16T14:48:35.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395425+0000", 2024-09-16T14:48:35.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395424+0000", 2024-09-16T14:48:35.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:35.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395429+0000", 2024-09-16T14:48:35.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5519999999999998e-06 2024-09-16T14:48:35.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395436+0000", 2024-09-16T14:48:35.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7859999999999999e-06 2024-09-16T14:48:35.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395441+0000", 2024-09-16T14:48:35.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9899999999999997e-06 2024-09-16T14:48:35.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395466+0000", 2024-09-16T14:48:35.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4739000000000001e-05 2024-09-16T14:48:35.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395478+0000", 2024-09-16T14:48:35.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2604e-05 2024-09-16T14:48:35.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399886+0000", 2024-09-16T14:48:35.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044076769999999996 2024-09-16T14:48:35.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399897+0000", 2024-09-16T14:48:35.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1396999999999999e-05 2024-09-16T14:48:35.601 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.601 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.601 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.601 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:161 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'115, mlcod=20'109)", 2024-09-16T14:48:35.602 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:23.395800+0000", 2024-09-16T14:48:35.602 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.128293530000001, 2024-09-16T14:48:35.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041068930000000003, 2024-09-16T14:48:35.602 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.602 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.602 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395800+0000", 2024-09-16T14:48:35.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395800+0000", 2024-09-16T14:48:35.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395800+0000", 2024-09-16T14:48:35.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:35.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395833+0000", 2024-09-16T14:48:35.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3416e-05 2024-09-16T14:48:35.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395841+0000", 2024-09-16T14:48:35.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5039999999999996e-06 2024-09-16T14:48:35.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395845+0000", 2024-09-16T14:48:35.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9599999999999999e-06 2024-09-16T14:48:35.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395868+0000", 2024-09-16T14:48:35.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2484e-05 2024-09-16T14:48:35.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395879+0000", 2024-09-16T14:48:35.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1222e-05 2024-09-16T14:48:35.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399898+0000", 2024-09-16T14:48:35.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.004018817 2024-09-16T14:48:35.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399907+0000", 2024-09-16T14:48:35.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9199999999999993e-06 2024-09-16T14:48:35.606 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.606 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.606 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.606 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:162 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'116, mlcod=20'109)", 2024-09-16T14:48:35.606 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:23.396386+0000", 2024-09-16T14:48:35.606 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.127707805, 2024-09-16T14:48:35.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00367265, 2024-09-16T14:48:35.606 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.606 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.607 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396386+0000", 2024-09-16T14:48:35.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396386+0000", 2024-09-16T14:48:35.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396385+0000", 2024-09-16T14:48:35.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:35.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396390+0000", 2024-09-16T14:48:35.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2140000000000004e-06 2024-09-16T14:48:35.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396398+0000", 2024-09-16T14:48:35.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0570000000000004e-06 2024-09-16T14:48:35.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396402+0000", 2024-09-16T14:48:35.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7709999999999997e-06 2024-09-16T14:48:35.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396802+0000", 2024-09-16T14:48:35.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000399579 2024-09-16T14:48:35.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396825+0000", 2024-09-16T14:48:35.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2840000000000002e-05 2024-09-16T14:48:35.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.400026+0000", 2024-09-16T14:48:35.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032011639999999998 2024-09-16T14:48:35.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.400058+0000", 2024-09-16T14:48:35.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2534999999999999e-05 2024-09-16T14:48:35.610 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.610 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.610 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.610 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:163 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'117, mlcod=20'109)", 2024-09-16T14:48:35.611 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:23.396761+0000", 2024-09-16T14:48:35.611 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.127332743, 2024-09-16T14:48:35.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0033157460000000001, 2024-09-16T14:48:35.611 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.611 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.611 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396761+0000", 2024-09-16T14:48:35.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396761+0000", 2024-09-16T14:48:35.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396760+0000", 2024-09-16T14:48:35.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:35.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396765+0000", 2024-09-16T14:48:35.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0130000000000001e-06 2024-09-16T14:48:35.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396771+0000", 2024-09-16T14:48:35.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0589999999999999e-06 2024-09-16T14:48:35.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396779+0000", 2024-09-16T14:48:35.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8089999999999993e-06 2024-09-16T14:48:35.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.397080+0000", 2024-09-16T14:48:35.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00030036800000000002 2024-09-16T14:48:35.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.397100+0000", 2024-09-16T14:48:35.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0038999999999999e-05 2024-09-16T14:48:35.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.400061+0000", 2024-09-16T14:48:35.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029615549999999998 2024-09-16T14:48:35.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.400077+0000", 2024-09-16T14:48:35.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5383000000000002e-05 2024-09-16T14:48:35.615 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.615 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.615 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.615 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:165 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'119, mlcod=20'109)", 2024-09-16T14:48:35.615 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:23.397690+0000", 2024-09-16T14:48:35.615 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.126403111, 2024-09-16T14:48:35.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.004996749, 2024-09-16T14:48:35.615 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:35.615 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:35.615 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:35.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:35.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.397690+0000", 2024-09-16T14:48:35.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:35.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.397690+0000", 2024-09-16T14:48:35.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:35.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:35.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.397690+0000", 2024-09-16T14:48:35.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:35.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:35.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.397695+0000", 2024-09-16T14:48:35.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7840000000000003e-06 2024-09-16T14:48:35.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:35.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.397701+0000", 2024-09-16T14:48:35.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6810000000000001e-06 2024-09-16T14:48:35.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:35.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.397706+0000", 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399587+0000", 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0018804920000000001 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399605+0000", 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8156e-05 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:35.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.402664+0000", 2024-09-16T14:48:35.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030596909999999998 2024-09-16T14:48:35.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:35.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:35.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:35.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.402687+0000", 2024-09-16T14:48:35.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2796e-05 2024-09-16T14:48:35.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.619 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:35.619 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:35.619 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:35.621 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:35.619+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:35.621 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:48:35.690 INFO:teuthology.orchestra.run.smithi111.stderr:Exporting 2.7 info 2.7( empty local-lis/les=20/21 n=0 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) 2024-09-16T14:48:35.692 INFO:teuthology.orchestra.run.smithi111.stderr:Export successful 2024-09-16T14:48:35.721 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:35.720+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:35.741 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:35.741 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:35.751 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:48:35.822 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:35.821+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:35.858 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:35.858 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:35.868 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:48:35.896 DEBUG:teuthology.orchestra.run.smithi040:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2024-09-16T14:48:35.924 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:35.922+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:35.938 INFO:tasks.rados.rados.0.smithi040.stdout:adding op weight read -> 100 2024-09-16T14:48:35.938 INFO:tasks.rados.rados.0.smithi040.stdout:adding op weight delete -> 50 2024-09-16T14:48:35.939 INFO:tasks.rados.rados.0.smithi040.stdout:adding op weight snap_create -> 50 2024-09-16T14:48:35.939 INFO:tasks.rados.rados.0.smithi040.stdout:adding op weight snap_remove -> 50 2024-09-16T14:48:35.939 INFO:tasks.rados.rados.0.smithi040.stdout:adding op weight rollback -> 50 2024-09-16T14:48:35.939 INFO:tasks.rados.rados.0.smithi040.stdout:adding op weight setattr -> 25 2024-09-16T14:48:35.939 INFO:tasks.rados.rados.0.smithi040.stdout:adding op weight rmattr -> 25 2024-09-16T14:48:35.939 INFO:tasks.rados.rados.0.smithi040.stdout:adding op weight copy_from -> 50 2024-09-16T14:48:35.939 INFO:tasks.rados.rados.0.smithi040.stdout:adding op weight append -> 100 2024-09-16T14:48:35.939 INFO:tasks.rados.rados.0.smithi040.stdout:ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable) 2024-09-16T14:48:35.939 INFO:tasks.rados.rados.0.smithi040.stdout:Configuration: 2024-09-16T14:48:35.939 INFO:tasks.rados.rados.0.smithi040.stdout: Number of operations: 400 2024-09-16T14:48:35.939 INFO:tasks.rados.rados.0.smithi040.stdout: Number of objects: 50 2024-09-16T14:48:35.939 INFO:tasks.rados.rados.0.smithi040.stdout: Max in flight operations: 16 2024-09-16T14:48:35.939 INFO:tasks.rados.rados.0.smithi040.stdout: Object size (in bytes): 4000000 2024-09-16T14:48:35.939 INFO:tasks.rados.rados.0.smithi040.stdout: Write stride min: 400000 2024-09-16T14:48:35.940 INFO:tasks.rados.rados.0.smithi040.stdout: Write stride max: 800000 2024-09-16T14:48:35.975 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:35.975 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:35.984 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:48:36.024 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:36.023+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:36.125 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:36.124+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:36.131 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:36.131 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:36.140 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:48:36.206 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --force --op remove --pgid 2.7 2024-09-16T14:48:36.243 INFO:teuthology.orchestra.run.smithi040.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T14:48:36.248 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:36.246+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:36.276 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:48:36.276 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:48:36.308 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:48:36.308 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:48:36.314 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:36.314 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:36.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:48:36.326 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:36.325+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:36.330 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:48:36.330 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:48:36.348 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:48:36.348 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:48:36.365 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:48:36.365 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:48:36.368 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:48:36.368 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:48:36.427 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:36.425+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:36.433 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:36.433 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:36.442 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:48:36.528 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:36.527+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:36.549 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:36.549 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:36.557 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:48:36.629 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:36.628+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:36.663 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:36.663 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:36.672 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:48:36.730 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:36.729+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:36.779 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:36.779 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:36.788 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:48:36.830 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:36.829+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:36.873 INFO:tasks.rados.rados.0.smithi040.stdout:1: write initial oid 1 2024-09-16T14:48:36.873 INFO:tasks.rados.rados.0.smithi040.stdout:1: seq_num 0 ranges {0=32768} 2024-09-16T14:48:36.873 INFO:tasks.rados.rados.0.smithi040.stdout:1: writing smithi04040151-1 from 0 to 32768 tid 1 2024-09-16T14:48:36.873 INFO:tasks.rados.rados.0.smithi040.stdout:2: write initial oid 2 2024-09-16T14:48:36.873 INFO:tasks.rados.rados.0.smithi040.stdout:2: seq_num 1 ranges {0=32768} 2024-09-16T14:48:36.874 INFO:tasks.rados.rados.0.smithi040.stdout:2: writing smithi04040151-2 from 0 to 32768 tid 1 2024-09-16T14:48:36.874 INFO:tasks.rados.rados.0.smithi040.stdout:3: write initial oid 3 2024-09-16T14:48:36.874 INFO:tasks.rados.rados.0.smithi040.stdout:3: seq_num 2 ranges {0=49152} 2024-09-16T14:48:36.874 INFO:tasks.rados.rados.0.smithi040.stdout:3: writing smithi04040151-3 from 0 to 49152 tid 1 2024-09-16T14:48:36.874 INFO:tasks.rados.rados.0.smithi040.stdout:4: write initial oid 4 2024-09-16T14:48:36.874 INFO:tasks.rados.rados.0.smithi040.stdout:4: seq_num 3 ranges {0=65536} 2024-09-16T14:48:36.875 INFO:tasks.rados.rados.0.smithi040.stdout:4: writing smithi04040151-4 from 0 to 65536 tid 1 2024-09-16T14:48:36.875 INFO:tasks.rados.rados.0.smithi040.stdout:5: write initial oid 5 2024-09-16T14:48:36.875 INFO:tasks.rados.rados.0.smithi040.stdout:5: seq_num 4 ranges {0=81920} 2024-09-16T14:48:36.875 INFO:tasks.rados.rados.0.smithi040.stdout:5: writing smithi04040151-5 from 0 to 81920 tid 1 2024-09-16T14:48:36.875 INFO:tasks.rados.rados.0.smithi040.stdout:6: write initial oid 6 2024-09-16T14:48:36.875 INFO:tasks.rados.rados.0.smithi040.stdout:6: seq_num 5 ranges {0=98304} 2024-09-16T14:48:36.875 INFO:tasks.rados.rados.0.smithi040.stdout:6: writing smithi04040151-6 from 0 to 98304 tid 1 2024-09-16T14:48:36.875 INFO:tasks.rados.rados.0.smithi040.stdout:7: write initial oid 7 2024-09-16T14:48:36.875 INFO:tasks.rados.rados.0.smithi040.stdout:7: seq_num 6 ranges {0=114688} 2024-09-16T14:48:36.879 INFO:tasks.rados.rados.0.smithi040.stdout:7: writing smithi04040151-7 from 0 to 114688 tid 1 2024-09-16T14:48:36.879 INFO:tasks.rados.rados.0.smithi040.stdout:8: write initial oid 8 2024-09-16T14:48:36.879 INFO:tasks.rados.rados.0.smithi040.stdout:8: seq_num 7 ranges {0=131072} 2024-09-16T14:48:36.879 INFO:tasks.rados.rados.0.smithi040.stdout:8: writing smithi04040151-8 from 0 to 131072 tid 1 2024-09-16T14:48:36.879 INFO:tasks.rados.rados.0.smithi040.stdout:9: write initial oid 9 2024-09-16T14:48:36.879 INFO:tasks.rados.rados.0.smithi040.stdout:9: seq_num 8 ranges {0=147456} 2024-09-16T14:48:36.879 INFO:tasks.rados.rados.0.smithi040.stdout:9: writing smithi04040151-9 from 0 to 147456 tid 1 2024-09-16T14:48:36.880 INFO:tasks.rados.rados.0.smithi040.stdout:10: write initial oid 10 2024-09-16T14:48:36.880 INFO:tasks.rados.rados.0.smithi040.stdout:10: seq_num 9 ranges {0=163840} 2024-09-16T14:48:36.880 INFO:tasks.rados.rados.0.smithi040.stdout:10: writing smithi04040151-10 from 0 to 163840 tid 1 2024-09-16T14:48:36.881 INFO:tasks.rados.rados.0.smithi040.stdout:11: write initial oid 11 2024-09-16T14:48:36.881 INFO:tasks.rados.rados.0.smithi040.stdout:11: seq_num 10 ranges {0=180224} 2024-09-16T14:48:36.885 INFO:tasks.rados.rados.0.smithi040.stdout:11: writing smithi04040151-11 from 0 to 180224 tid 1 2024-09-16T14:48:36.885 INFO:tasks.rados.rados.0.smithi040.stdout:1: finishing write tid 1 to smithi04040151-1 2024-09-16T14:48:36.885 INFO:tasks.rados.rados.0.smithi040.stdout:1: finishing write tid 2 to smithi04040151-1 2024-09-16T14:48:36.886 INFO:tasks.rados.rados.0.smithi040.stdout:5: finishing write tid 1 to smithi04040151-5 2024-09-16T14:48:36.886 INFO:tasks.rados.rados.0.smithi040.stdout:12: write initial oid 12 2024-09-16T14:48:36.886 INFO:tasks.rados.rados.0.smithi040.stdout:12: seq_num 11 ranges {0=196608} 2024-09-16T14:48:36.887 INFO:tasks.rados.rados.0.smithi040.stdout:12: writing smithi04040151-12 from 0 to 196608 tid 1 2024-09-16T14:48:36.887 INFO:tasks.rados.rados.0.smithi040.stdout:5: finishing write tid 2 to smithi04040151-5 2024-09-16T14:48:36.887 INFO:tasks.rados.rados.0.smithi040.stdout:3: finishing write tid 1 to smithi04040151-3 2024-09-16T14:48:36.888 INFO:tasks.rados.rados.0.smithi040.stdout:13: write initial oid 13 2024-09-16T14:48:36.888 INFO:tasks.rados.rados.0.smithi040.stdout:13: seq_num 12 ranges {0=212992} 2024-09-16T14:48:36.889 INFO:tasks.rados.rados.0.smithi040.stdout:13: writing smithi04040151-13 from 0 to 212992 tid 1 2024-09-16T14:48:36.889 INFO:tasks.rados.rados.0.smithi040.stdout:14: write initial oid 14 2024-09-16T14:48:36.889 INFO:tasks.rados.rados.0.smithi040.stdout:14: seq_num 13 ranges {0=229376} 2024-09-16T14:48:36.891 INFO:tasks.rados.rados.0.smithi040.stdout:14: writing smithi04040151-14 from 0 to 229376 tid 1 2024-09-16T14:48:36.891 INFO:tasks.rados.rados.0.smithi040.stdout:15: write initial oid 15 2024-09-16T14:48:36.892 INFO:tasks.rados.rados.0.smithi040.stdout:15: seq_num 14 ranges {0=245760} 2024-09-16T14:48:36.894 INFO:tasks.rados.rados.0.smithi040.stdout:15: writing smithi04040151-15 from 0 to 245760 tid 1 2024-09-16T14:48:36.894 INFO:tasks.rados.rados.0.smithi040.stdout:1: finishing write tid 3 to smithi04040151-1 2024-09-16T14:48:36.894 INFO:tasks.rados.rados.0.smithi040.stdout:1: oid 1 updating version 0 to 1 2024-09-16T14:48:36.894 INFO:tasks.rados.rados.0.smithi040.stdout:1: oid 1 updating version 1 to 2 2024-09-16T14:48:36.895 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 1 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-16T14:48:36.895 INFO:tasks.rados.rados.0.smithi040.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-09-16T14:48:36.895 INFO:tasks.rados.rados.0.smithi040.stdout:5: finishing write tid 3 to smithi04040151-5 2024-09-16T14:48:36.895 INFO:tasks.rados.rados.0.smithi040.stdout:5: oid 5 updating version 0 to 1 2024-09-16T14:48:36.895 INFO:tasks.rados.rados.0.smithi040.stdout:5: oid 5 updating version 1 to 2 2024-09-16T14:48:36.895 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 5 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-16T14:48:36.895 INFO:tasks.rados.rados.0.smithi040.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-09-16T14:48:36.895 INFO:tasks.rados.rados.0.smithi040.stdout:3: finishing write tid 2 to smithi04040151-3 2024-09-16T14:48:36.895 INFO:tasks.rados.rados.0.smithi040.stdout:6: finishing write tid 1 to smithi04040151-6 2024-09-16T14:48:36.895 INFO:tasks.rados.rados.0.smithi040.stdout:2: finishing write tid 1 to smithi04040151-2 2024-09-16T14:48:36.895 INFO:tasks.rados.rados.0.smithi040.stdout:4: finishing write tid 1 to smithi04040151-4 2024-09-16T14:48:36.895 INFO:tasks.rados.rados.0.smithi040.stdout:4: finishing write tid 2 to smithi04040151-4 2024-09-16T14:48:36.895 INFO:tasks.rados.rados.0.smithi040.stdout:2: finishing write tid 2 to smithi04040151-2 2024-09-16T14:48:36.896 INFO:tasks.rados.rados.0.smithi040.stdout:3: finishing write tid 3 to smithi04040151-3 2024-09-16T14:48:36.896 INFO:tasks.rados.rados.0.smithi040.stdout:3: oid 3 updating version 0 to 1 2024-09-16T14:48:36.896 INFO:tasks.rados.rados.0.smithi040.stdout:3: oid 3 updating version 1 to 2 2024-09-16T14:48:36.896 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 3 v 2 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-16T14:48:36.896 INFO:tasks.rados.rados.0.smithi040.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-09-16T14:48:36.896 INFO:tasks.rados.rados.0.smithi040.stdout:8: finishing write tid 1 to smithi04040151-8 2024-09-16T14:48:36.896 INFO:tasks.rados.rados.0.smithi040.stdout:9: finishing write tid 1 to smithi04040151-9 2024-09-16T14:48:36.896 INFO:tasks.rados.rados.0.smithi040.stdout:8: finishing write tid 2 to smithi04040151-8 2024-09-16T14:48:36.896 INFO:tasks.rados.rados.0.smithi040.stdout:4: finishing write tid 3 to smithi04040151-4 2024-09-16T14:48:36.896 INFO:tasks.rados.rados.0.smithi040.stdout:4: oid 4 updating version 0 to 1 2024-09-16T14:48:36.896 INFO:tasks.rados.rados.0.smithi040.stdout:4: oid 4 updating version 1 to 2 2024-09-16T14:48:36.896 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-16T14:48:36.896 INFO:tasks.rados.rados.0.smithi040.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-09-16T14:48:36.897 INFO:tasks.rados.rados.0.smithi040.stdout:9: finishing write tid 2 to smithi04040151-9 2024-09-16T14:48:36.897 INFO:tasks.rados.rados.0.smithi040.stdout:1: done (14 left) 2024-09-16T14:48:36.897 INFO:tasks.rados.rados.0.smithi040.stdout:3: done (13 left) 2024-09-16T14:48:36.897 INFO:tasks.rados.rados.0.smithi040.stdout:4: done (12 left) 2024-09-16T14:48:36.897 INFO:tasks.rados.rados.0.smithi040.stdout:5: done (11 left) 2024-09-16T14:48:36.897 INFO:tasks.rados.rados.0.smithi040.stdout:16: write initial oid 16 2024-09-16T14:48:36.897 INFO:tasks.rados.rados.0.smithi040.stdout:16: seq_num 15 ranges {0=262144} 2024-09-16T14:48:36.897 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:48:36.898 INFO:tasks.rados.rados.0.smithi040.stdout:16: writing smithi04040151-16 from 0 to 262144 tid 1 2024-09-16T14:48:36.898 INFO:tasks.rados.rados.0.smithi040.stdout:17: write initial oid 17 2024-09-16T14:48:36.898 INFO:tasks.rados.rados.0.smithi040.stdout:17: seq_num 16 ranges {0=278528} 2024-09-16T14:48:36.899 INFO:tasks.rados.rados.0.smithi040.stdout:17: writing smithi04040151-17 from 0 to 278528 tid 1 2024-09-16T14:48:36.899 INFO:tasks.rados.rados.0.smithi040.stdout:18: write initial oid 18 2024-09-16T14:48:36.899 INFO:tasks.rados.rados.0.smithi040.stdout:18: seq_num 17 ranges {0=294912} 2024-09-16T14:48:36.901 INFO:tasks.rados.rados.0.smithi040.stdout:18: writing smithi04040151-18 from 0 to 294912 tid 1 2024-09-16T14:48:36.902 INFO:tasks.rados.rados.0.smithi040.stdout:7: finishing write tid 1 to smithi04040151-7 2024-09-16T14:48:36.902 INFO:tasks.rados.rados.0.smithi040.stdout:2: finishing write tid 3 to smithi04040151-2 2024-09-16T14:48:36.902 INFO:tasks.rados.rados.0.smithi040.stdout:2: oid 2 updating version 0 to 1 2024-09-16T14:48:36.902 INFO:tasks.rados.rados.0.smithi040.stdout:2: oid 2 updating version 1 to 2 2024-09-16T14:48:36.902 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 2 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T14:48:36.902 INFO:tasks.rados.rados.0.smithi040.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-09-16T14:48:36.902 INFO:tasks.rados.rados.0.smithi040.stdout:7: finishing write tid 2 to smithi04040151-7 2024-09-16T14:48:36.902 INFO:tasks.rados.rados.0.smithi040.stdout:6: finishing write tid 2 to smithi04040151-6 2024-09-16T14:48:36.902 INFO:tasks.rados.rados.0.smithi040.stdout:10: finishing write tid 1 to smithi04040151-10 2024-09-16T14:48:36.902 INFO:tasks.rados.rados.0.smithi040.stdout:10: finishing write tid 2 to smithi04040151-10 2024-09-16T14:48:36.903 INFO:tasks.rados.rados.0.smithi040.stdout:11: finishing write tid 1 to smithi04040151-11 2024-09-16T14:48:36.903 INFO:tasks.rados.rados.0.smithi040.stdout:11: finishing write tid 2 to smithi04040151-11 2024-09-16T14:48:36.903 INFO:tasks.rados.rados.0.smithi040.stdout:8: finishing write tid 3 to smithi04040151-8 2024-09-16T14:48:36.903 INFO:tasks.rados.rados.0.smithi040.stdout:8: oid 8 updating version 0 to 1 2024-09-16T14:48:36.903 INFO:tasks.rados.rados.0.smithi040.stdout:8: oid 8 updating version 1 to 2 2024-09-16T14:48:36.903 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 8 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-16T14:48:36.903 INFO:tasks.rados.rados.0.smithi040.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-09-16T14:48:36.903 INFO:tasks.rados.rados.0.smithi040.stdout:9: finishing write tid 3 to smithi04040151-9 2024-09-16T14:48:36.903 INFO:tasks.rados.rados.0.smithi040.stdout:9: oid 9 updating version 0 to 1 2024-09-16T14:48:36.903 INFO:tasks.rados.rados.0.smithi040.stdout:9: oid 9 updating version 1 to 2 2024-09-16T14:48:36.904 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 9 v 2 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-16T14:48:36.904 INFO:tasks.rados.rados.0.smithi040.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-09-16T14:48:36.904 INFO:tasks.rados.rados.0.smithi040.stdout:11: finishing write tid 3 to smithi04040151-11 2024-09-16T14:48:36.904 INFO:tasks.rados.rados.0.smithi040.stdout:11: oid 11 updating version 0 to 2 2024-09-16T14:48:36.904 INFO:tasks.rados.rados.0.smithi040.stdout:11: oid 11 version 2 is already newer than 1 2024-09-16T14:48:36.904 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 11 v 2 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-16T14:48:36.904 INFO:tasks.rados.rados.0.smithi040.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-09-16T14:48:36.904 INFO:tasks.rados.rados.0.smithi040.stdout:13: finishing write tid 1 to smithi04040151-13 2024-09-16T14:48:36.904 INFO:tasks.rados.rados.0.smithi040.stdout:13: finishing write tid 2 to smithi04040151-13 2024-09-16T14:48:36.904 INFO:tasks.rados.rados.0.smithi040.stdout:2: done (13 left) 2024-09-16T14:48:36.904 INFO:tasks.rados.rados.0.smithi040.stdout:8: done (12 left) 2024-09-16T14:48:36.904 INFO:tasks.rados.rados.0.smithi040.stdout:9: done (11 left) 2024-09-16T14:48:36.905 INFO:tasks.rados.rados.0.smithi040.stdout:11: done (10 left) 2024-09-16T14:48:36.905 INFO:tasks.rados.rados.0.smithi040.stdout:19: write initial oid 19 2024-09-16T14:48:36.905 INFO:tasks.rados.rados.0.smithi040.stdout:19: seq_num 18 ranges {0=311296} 2024-09-16T14:48:36.905 INFO:tasks.rados.rados.0.smithi040.stdout:19: writing smithi04040151-19 from 0 to 311296 tid 1 2024-09-16T14:48:36.905 INFO:tasks.rados.rados.0.smithi040.stdout:20: write initial oid 20 2024-09-16T14:48:36.905 INFO:tasks.rados.rados.0.smithi040.stdout:20: seq_num 19 ranges {0=327680} 2024-09-16T14:48:36.906 INFO:tasks.rados.rados.0.smithi040.stdout:20: writing smithi04040151-20 from 0 to 327680 tid 1 2024-09-16T14:48:36.907 INFO:tasks.rados.rados.0.smithi040.stdout:21: write initial oid 21 2024-09-16T14:48:36.907 INFO:tasks.rados.rados.0.smithi040.stdout:21: seq_num 20 ranges {0=344064} 2024-09-16T14:48:36.909 INFO:tasks.rados.rados.0.smithi040.stdout:21: writing smithi04040151-21 from 0 to 344064 tid 1 2024-09-16T14:48:36.910 INFO:tasks.rados.rados.0.smithi040.stdout:13: finishing write tid 3 to smithi04040151-13 2024-09-16T14:48:36.910 INFO:tasks.rados.rados.0.smithi040.stdout:13: oid 13 updating version 0 to 3 2024-09-16T14:48:36.910 INFO:tasks.rados.rados.0.smithi040.stdout:13: oid 13 updating version 3 to 4 2024-09-16T14:48:36.910 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 13 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-16T14:48:36.910 INFO:tasks.rados.rados.0.smithi040.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-09-16T14:48:36.910 INFO:tasks.rados.rados.0.smithi040.stdout:10: finishing write tid 3 to smithi04040151-10 2024-09-16T14:48:36.910 INFO:tasks.rados.rados.0.smithi040.stdout:10: oid 10 updating version 0 to 3 2024-09-16T14:48:36.910 INFO:tasks.rados.rados.0.smithi040.stdout:10: oid 10 updating version 3 to 4 2024-09-16T14:48:36.910 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 10 v 4 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-16T14:48:36.911 INFO:tasks.rados.rados.0.smithi040.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-09-16T14:48:36.911 INFO:tasks.rados.rados.0.smithi040.stdout:7: finishing write tid 3 to smithi04040151-7 2024-09-16T14:48:36.911 INFO:tasks.rados.rados.0.smithi040.stdout:7: oid 7 updating version 0 to 2 2024-09-16T14:48:36.911 INFO:tasks.rados.rados.0.smithi040.stdout:7: oid 7 version 2 is already newer than 1 2024-09-16T14:48:36.911 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 7 v 2 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-16T14:48:36.911 INFO:tasks.rados.rados.0.smithi040.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-09-16T14:48:36.911 INFO:tasks.rados.rados.0.smithi040.stdout:7: done (12 left) 2024-09-16T14:48:36.911 INFO:tasks.rados.rados.0.smithi040.stdout:10: done (11 left) 2024-09-16T14:48:36.911 INFO:tasks.rados.rados.0.smithi040.stdout:13: done (10 left) 2024-09-16T14:48:36.911 INFO:tasks.rados.rados.0.smithi040.stdout:22: write initial oid 22 2024-09-16T14:48:36.911 INFO:tasks.rados.rados.0.smithi040.stdout:22: seq_num 21 ranges {0=360448} 2024-09-16T14:48:36.912 INFO:tasks.rados.rados.0.smithi040.stdout:22: writing smithi04040151-22 from 0 to 360448 tid 1 2024-09-16T14:48:36.913 INFO:tasks.rados.rados.0.smithi040.stdout:23: write initial oid 23 2024-09-16T14:48:36.913 INFO:tasks.rados.rados.0.smithi040.stdout:23: seq_num 22 ranges {0=376832} 2024-09-16T14:48:36.915 INFO:tasks.rados.rados.0.smithi040.stdout:23: writing smithi04040151-23 from 0 to 376832 tid 1 2024-09-16T14:48:36.915 INFO:tasks.rados.rados.0.smithi040.stdout:24: write initial oid 24 2024-09-16T14:48:36.916 INFO:tasks.rados.rados.0.smithi040.stdout:24: seq_num 23 ranges {0=393216} 2024-09-16T14:48:36.919 INFO:tasks.rados.rados.0.smithi040.stdout:24: writing smithi04040151-24 from 0 to 393216 tid 1 2024-09-16T14:48:36.919 INFO:tasks.rados.rados.0.smithi040.stdout:25: write initial oid 25 2024-09-16T14:48:36.919 INFO:tasks.rados.rados.0.smithi040.stdout:25: seq_num 24 ranges {0=409600} 2024-09-16T14:48:36.920 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:36.920 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:36.922 INFO:tasks.rados.rados.0.smithi040.stdout:25: writing smithi04040151-25 from 0 to 409600 tid 1 2024-09-16T14:48:36.922 INFO:tasks.rados.rados.0.smithi040.stdout:6: finishing write tid 3 to smithi04040151-6 2024-09-16T14:48:36.922 INFO:tasks.rados.rados.0.smithi040.stdout:6: oid 6 updating version 0 to 1 2024-09-16T14:48:36.922 INFO:tasks.rados.rados.0.smithi040.stdout:6: oid 6 updating version 1 to 2 2024-09-16T14:48:36.922 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 6 v 2 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-16T14:48:36.922 INFO:tasks.rados.rados.0.smithi040.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-09-16T14:48:36.923 INFO:tasks.rados.rados.0.smithi040.stdout:14: finishing write tid 1 to smithi04040151-14 2024-09-16T14:48:36.923 INFO:tasks.rados.rados.0.smithi040.stdout:14: finishing write tid 2 to smithi04040151-14 2024-09-16T14:48:36.923 INFO:tasks.rados.rados.0.smithi040.stdout:15: finishing write tid 1 to smithi04040151-15 2024-09-16T14:48:36.923 INFO:tasks.rados.rados.0.smithi040.stdout:15: finishing write tid 2 to smithi04040151-15 2024-09-16T14:48:36.923 INFO:tasks.rados.rados.0.smithi040.stdout:14: finishing write tid 3 to smithi04040151-14 2024-09-16T14:48:36.923 INFO:tasks.rados.rados.0.smithi040.stdout:14: oid 14 updating version 0 to 5 2024-09-16T14:48:36.923 INFO:tasks.rados.rados.0.smithi040.stdout:14: oid 14 updating version 5 to 6 2024-09-16T14:48:36.923 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 14 v 6 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-16T14:48:36.923 INFO:tasks.rados.rados.0.smithi040.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-09-16T14:48:36.923 INFO:tasks.rados.rados.0.smithi040.stdout:15: finishing write tid 3 to smithi04040151-15 2024-09-16T14:48:36.923 INFO:tasks.rados.rados.0.smithi040.stdout:15: oid 15 updating version 0 to 2 2024-09-16T14:48:36.923 INFO:tasks.rados.rados.0.smithi040.stdout:15: oid 15 version 2 is already newer than 1 2024-09-16T14:48:36.923 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 15 v 2 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-16T14:48:36.923 INFO:tasks.rados.rados.0.smithi040.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-09-16T14:48:36.924 INFO:tasks.rados.rados.0.smithi040.stdout:16: finishing write tid 1 to smithi04040151-16 2024-09-16T14:48:36.924 INFO:tasks.rados.rados.0.smithi040.stdout:17: finishing write tid 1 to smithi04040151-17 2024-09-16T14:48:36.924 INFO:tasks.rados.rados.0.smithi040.stdout:17: finishing write tid 2 to smithi04040151-17 2024-09-16T14:48:36.924 INFO:tasks.rados.rados.0.smithi040.stdout:16: finishing write tid 2 to smithi04040151-16 2024-09-16T14:48:36.924 INFO:tasks.rados.rados.0.smithi040.stdout:16: finishing write tid 3 to smithi04040151-16 2024-09-16T14:48:36.924 INFO:tasks.rados.rados.0.smithi040.stdout:16: oid 16 updating version 0 to 3 2024-09-16T14:48:36.924 INFO:tasks.rados.rados.0.smithi040.stdout:16: oid 16 updating version 3 to 4 2024-09-16T14:48:36.924 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 16 v 4 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-16T14:48:36.924 INFO:tasks.rados.rados.0.smithi040.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-09-16T14:48:36.924 INFO:tasks.rados.rados.0.smithi040.stdout:18: finishing write tid 1 to smithi04040151-18 2024-09-16T14:48:36.924 INFO:tasks.rados.rados.0.smithi040.stdout:18: finishing write tid 2 to smithi04040151-18 2024-09-16T14:48:36.924 INFO:tasks.rados.rados.0.smithi040.stdout:17: finishing write tid 3 to smithi04040151-17 2024-09-16T14:48:36.924 INFO:tasks.rados.rados.0.smithi040.stdout:17: oid 17 updating version 0 to 3 2024-09-16T14:48:36.924 INFO:tasks.rados.rados.0.smithi040.stdout:17: oid 17 updating version 3 to 4 2024-09-16T14:48:36.925 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 17 v 4 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-16T14:48:36.925 INFO:tasks.rados.rados.0.smithi040.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-09-16T14:48:36.925 INFO:tasks.rados.rados.0.smithi040.stdout:19: finishing write tid 1 to smithi04040151-19 2024-09-16T14:48:36.925 INFO:tasks.rados.rados.0.smithi040.stdout:19: finishing write tid 2 to smithi04040151-19 2024-09-16T14:48:36.925 INFO:tasks.rados.rados.0.smithi040.stdout:18: finishing write tid 3 to smithi04040151-18 2024-09-16T14:48:36.925 INFO:tasks.rados.rados.0.smithi040.stdout:18: oid 18 updating version 0 to 4 2024-09-16T14:48:36.925 INFO:tasks.rados.rados.0.smithi040.stdout:18: oid 18 version 4 is already newer than 3 2024-09-16T14:48:36.925 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 18 v 4 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-16T14:48:36.925 INFO:tasks.rados.rados.0.smithi040.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-09-16T14:48:36.925 INFO:tasks.rados.rados.0.smithi040.stdout:20: finishing write tid 1 to smithi04040151-20 2024-09-16T14:48:36.925 INFO:tasks.rados.rados.0.smithi040.stdout:20: finishing write tid 2 to smithi04040151-20 2024-09-16T14:48:36.925 INFO:tasks.rados.rados.0.smithi040.stdout:19: finishing write tid 3 to smithi04040151-19 2024-09-16T14:48:36.925 INFO:tasks.rados.rados.0.smithi040.stdout:19: oid 19 updating version 0 to 3 2024-09-16T14:48:36.926 INFO:tasks.rados.rados.0.smithi040.stdout:19: oid 19 updating version 3 to 4 2024-09-16T14:48:36.926 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 19 v 4 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-16T14:48:36.926 INFO:tasks.rados.rados.0.smithi040.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-09-16T14:48:36.926 INFO:tasks.rados.rados.0.smithi040.stdout:20: finishing write tid 3 to smithi04040151-20 2024-09-16T14:48:36.926 INFO:tasks.rados.rados.0.smithi040.stdout:20: oid 20 updating version 0 to 2 2024-09-16T14:48:36.926 INFO:tasks.rados.rados.0.smithi040.stdout:20: oid 20 version 2 is already newer than 1 2024-09-16T14:48:36.926 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 20 v 2 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-16T14:48:36.926 INFO:tasks.rados.rados.0.smithi040.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-09-16T14:48:36.926 INFO:tasks.rados.rados.0.smithi040.stdout:21: finishing write tid 1 to smithi04040151-21 2024-09-16T14:48:36.926 INFO:tasks.rados.rados.0.smithi040.stdout:21: finishing write tid 2 to smithi04040151-21 2024-09-16T14:48:36.926 INFO:tasks.rados.rados.0.smithi040.stdout:6: done (13 left) 2024-09-16T14:48:36.926 INFO:tasks.rados.rados.0.smithi040.stdout:14: done (12 left) 2024-09-16T14:48:36.926 INFO:tasks.rados.rados.0.smithi040.stdout:15: done (11 left) 2024-09-16T14:48:36.926 INFO:tasks.rados.rados.0.smithi040.stdout:16: done (10 left) 2024-09-16T14:48:36.927 INFO:tasks.rados.rados.0.smithi040.stdout:17: done (9 left) 2024-09-16T14:48:36.927 INFO:tasks.rados.rados.0.smithi040.stdout:18: done (8 left) 2024-09-16T14:48:36.927 INFO:tasks.rados.rados.0.smithi040.stdout:19: done (7 left) 2024-09-16T14:48:36.927 INFO:tasks.rados.rados.0.smithi040.stdout:20: done (6 left) 2024-09-16T14:48:36.927 INFO:tasks.rados.rados.0.smithi040.stdout:26: write initial oid 26 2024-09-16T14:48:36.927 INFO:tasks.rados.rados.0.smithi040.stdout:26: seq_num 25 ranges {0=425984} 2024-09-16T14:48:36.927 INFO:tasks.rados.rados.0.smithi040.stdout:26: writing smithi04040151-26 from 0 to 425984 tid 1 2024-09-16T14:48:36.928 INFO:tasks.rados.rados.0.smithi040.stdout:27: write initial oid 27 2024-09-16T14:48:36.928 INFO:tasks.rados.rados.0.smithi040.stdout:23: finishing write tid 1 to smithi04040151-23 2024-09-16T14:48:36.928 INFO:tasks.rados.rados.0.smithi040.stdout:23: finishing write tid 2 to smithi04040151-23 2024-09-16T14:48:36.928 INFO:tasks.rados.rados.0.smithi040.stdout:27: seq_num 26 ranges {0=442368} 2024-09-16T14:48:36.928 INFO:tasks.rados.rados.0.smithi040.stdout:27: writing smithi04040151-27 from 0 to 442368 tid 1 2024-09-16T14:48:36.928 INFO:tasks.rados.rados.0.smithi040.stdout:28: write initial oid 28 2024-09-16T14:48:36.929 INFO:tasks.rados.rados.0.smithi040.stdout:28: seq_num 27 ranges {0=458752} 2024-09-16T14:48:36.931 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:36.929+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:36.931 INFO:tasks.rados.rados.0.smithi040.stdout:28: writing smithi04040151-28 from 0 to 458752 tid 1 2024-09-16T14:48:36.931 INFO:tasks.rados.rados.0.smithi040.stdout:29: write initial oid 29 2024-09-16T14:48:36.932 INFO:tasks.rados.rados.0.smithi040.stdout:29: seq_num 28 ranges {0=475136} 2024-09-16T14:48:36.935 INFO:tasks.rados.rados.0.smithi040.stdout:29: writing smithi04040151-29 from 0 to 475136 tid 1 2024-09-16T14:48:36.935 INFO:tasks.rados.rados.0.smithi040.stdout:30: write initial oid 30 2024-09-16T14:48:36.935 INFO:tasks.rados.rados.0.smithi040.stdout:30: seq_num 29 ranges {0=491520} 2024-09-16T14:48:36.936 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:48:36.939 INFO:tasks.rados.rados.0.smithi040.stdout:30: writing smithi04040151-30 from 0 to 491520 tid 1 2024-09-16T14:48:36.939 INFO:tasks.rados.rados.0.smithi040.stdout:31: write initial oid 31 2024-09-16T14:48:36.939 INFO:tasks.rados.rados.0.smithi040.stdout:31: seq_num 30 ranges {0=507904} 2024-09-16T14:48:36.942 INFO:tasks.rados.rados.0.smithi040.stdout:31: writing smithi04040151-31 from 0 to 507904 tid 1 2024-09-16T14:48:36.942 INFO:tasks.rados.rados.0.smithi040.stdout:32: write initial oid 32 2024-09-16T14:48:36.942 INFO:tasks.rados.rados.0.smithi040.stdout:32: seq_num 31 ranges {0=524288} 2024-09-16T14:48:36.946 INFO:tasks.rados.rados.0.smithi040.stdout:32: writing smithi04040151-32 from 0 to 524288 tid 1 2024-09-16T14:48:36.946 INFO:tasks.rados.rados.0.smithi040.stdout:33: write initial oid 33 2024-09-16T14:48:36.947 INFO:tasks.rados.rados.0.smithi040.stdout:33: seq_num 32 ranges {0=540672} 2024-09-16T14:48:36.953 INFO:tasks.rados.rados.0.smithi040.stdout:33: writing smithi04040151-33 from 0 to 540672 tid 1 2024-09-16T14:48:36.953 INFO:tasks.rados.rados.0.smithi040.stdout:34: write initial oid 34 2024-09-16T14:48:36.953 INFO:tasks.rados.rados.0.smithi040.stdout:34: seq_num 33 ranges {0=557056} 2024-09-16T14:48:36.954 INFO:tasks.rados.rados.0.smithi040.stdout:34: writing smithi04040151-34 from 0 to 557056 tid 1 2024-09-16T14:48:36.954 INFO:tasks.rados.rados.0.smithi040.stdout:35: write initial oid 35 2024-09-16T14:48:36.954 INFO:tasks.rados.rados.0.smithi040.stdout:35: seq_num 34 ranges {0=573440} 2024-09-16T14:48:36.959 INFO:tasks.rados.rados.0.smithi040.stdout:35: writing smithi04040151-35 from 0 to 573440 tid 1 2024-09-16T14:48:36.959 INFO:tasks.rados.rados.0.smithi040.stdout: waiting on 16 2024-09-16T14:48:36.959 INFO:tasks.rados.rados.0.smithi040.stdout:23: finishing write tid 3 to smithi04040151-23 2024-09-16T14:48:36.959 INFO:tasks.rados.rados.0.smithi040.stdout:23: oid 23 updating version 0 to 4 2024-09-16T14:48:36.959 INFO:tasks.rados.rados.0.smithi040.stdout:23: oid 23 version 4 is already newer than 3 2024-09-16T14:48:36.959 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 23 v 4 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-16T14:48:36.959 INFO:tasks.rados.rados.0.smithi040.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-09-16T14:48:36.959 INFO:tasks.rados.rados.0.smithi040.stdout:21: finishing write tid 3 to smithi04040151-21 2024-09-16T14:48:36.959 INFO:tasks.rados.rados.0.smithi040.stdout:21: oid 21 updating version 0 to 5 2024-09-16T14:48:36.959 INFO:tasks.rados.rados.0.smithi040.stdout:21: oid 21 updating version 5 to 6 2024-09-16T14:48:36.959 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 21 v 6 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-16T14:48:36.959 INFO:tasks.rados.rados.0.smithi040.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-09-16T14:48:36.959 INFO:tasks.rados.rados.0.smithi040.stdout:22: finishing write tid 1 to smithi04040151-22 2024-09-16T14:48:36.960 INFO:tasks.rados.rados.0.smithi040.stdout:22: finishing write tid 2 to smithi04040151-22 2024-09-16T14:48:36.960 INFO:tasks.rados.rados.0.smithi040.stdout:24: finishing write tid 1 to smithi04040151-24 2024-09-16T14:48:36.960 INFO:tasks.rados.rados.0.smithi040.stdout:24: finishing write tid 2 to smithi04040151-24 2024-09-16T14:48:36.960 INFO:tasks.rados.rados.0.smithi040.stdout:24: finishing write tid 3 to smithi04040151-24 2024-09-16T14:48:36.960 INFO:tasks.rados.rados.0.smithi040.stdout:24: oid 24 updating version 0 to 3 2024-09-16T14:48:36.960 INFO:tasks.rados.rados.0.smithi040.stdout:24: oid 24 updating version 3 to 4 2024-09-16T14:48:36.960 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 24 v 4 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-16T14:48:36.960 INFO:tasks.rados.rados.0.smithi040.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-09-16T14:48:36.960 INFO:tasks.rados.rados.0.smithi040.stdout:25: finishing write tid 1 to smithi04040151-25 2024-09-16T14:48:36.961 INFO:tasks.rados.rados.0.smithi040.stdout:22: finishing write tid 3 to smithi04040151-22 2024-09-16T14:48:36.961 INFO:tasks.rados.rados.0.smithi040.stdout:22: oid 22 updating version 0 to 3 2024-09-16T14:48:36.961 INFO:tasks.rados.rados.0.smithi040.stdout:22: oid 22 updating version 3 to 4 2024-09-16T14:48:36.961 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 22 v 4 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-16T14:48:36.961 INFO:tasks.rados.rados.0.smithi040.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-09-16T14:48:36.961 INFO:tasks.rados.rados.0.smithi040.stdout:25: finishing write tid 2 to smithi04040151-25 2024-09-16T14:48:36.961 INFO:tasks.rados.rados.0.smithi040.stdout:26: finishing write tid 1 to smithi04040151-26 2024-09-16T14:48:36.961 INFO:tasks.rados.rados.0.smithi040.stdout:26: finishing write tid 2 to smithi04040151-26 2024-09-16T14:48:36.961 INFO:tasks.rados.rados.0.smithi040.stdout:25: finishing write tid 3 to smithi04040151-25 2024-09-16T14:48:36.961 INFO:tasks.rados.rados.0.smithi040.stdout:25: oid 25 updating version 0 to 1 2024-09-16T14:48:36.961 INFO:tasks.rados.rados.0.smithi040.stdout:25: oid 25 updating version 1 to 2 2024-09-16T14:48:36.961 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 25 v 2 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-16T14:48:36.961 INFO:tasks.rados.rados.0.smithi040.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-09-16T14:48:36.961 INFO:tasks.rados.rados.0.smithi040.stdout:26: finishing write tid 3 to smithi04040151-26 2024-09-16T14:48:36.962 INFO:tasks.rados.rados.0.smithi040.stdout:26: oid 26 updating version 0 to 5 2024-09-16T14:48:36.962 INFO:tasks.rados.rados.0.smithi040.stdout:26: oid 26 updating version 5 to 6 2024-09-16T14:48:36.962 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 26 v 6 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-16T14:48:36.962 INFO:tasks.rados.rados.0.smithi040.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-09-16T14:48:36.962 INFO:tasks.rados.rados.0.smithi040.stdout:28: finishing write tid 1 to smithi04040151-28 2024-09-16T14:48:36.962 INFO:tasks.rados.rados.0.smithi040.stdout:28: finishing write tid 2 to smithi04040151-28 2024-09-16T14:48:36.962 INFO:tasks.rados.rados.0.smithi040.stdout:21: done (15 left) 2024-09-16T14:48:36.962 INFO:tasks.rados.rados.0.smithi040.stdout:22: done (14 left) 2024-09-16T14:48:36.962 INFO:tasks.rados.rados.0.smithi040.stdout:23: done (13 left) 2024-09-16T14:48:36.962 INFO:tasks.rados.rados.0.smithi040.stdout:24: done (12 left) 2024-09-16T14:48:36.962 INFO:tasks.rados.rados.0.smithi040.stdout:25: done (11 left) 2024-09-16T14:48:36.962 INFO:tasks.rados.rados.0.smithi040.stdout:26: done (10 left) 2024-09-16T14:48:36.962 INFO:tasks.rados.rados.0.smithi040.stdout:36: write initial oid 36 2024-09-16T14:48:36.962 INFO:tasks.rados.rados.0.smithi040.stdout:36: seq_num 35 ranges {0=589824} 2024-09-16T14:48:36.963 INFO:tasks.rados.rados.0.smithi040.stdout:36: writing smithi04040151-36 from 0 to 589824 tid 1 2024-09-16T14:48:36.963 INFO:tasks.rados.rados.0.smithi040.stdout:37: write initial oid 37 2024-09-16T14:48:36.963 INFO:tasks.rados.rados.0.smithi040.stdout:37: seq_num 36 ranges {0=606208} 2024-09-16T14:48:36.967 INFO:tasks.rados.rados.0.smithi040.stdout:37: writing smithi04040151-37 from 0 to 606208 tid 1 2024-09-16T14:48:36.967 INFO:tasks.rados.rados.0.smithi040.stdout:38: write initial oid 38 2024-09-16T14:48:36.967 INFO:tasks.rados.rados.0.smithi040.stdout:38: seq_num 37 ranges {0=622592} 2024-09-16T14:48:36.971 INFO:tasks.rados.rados.0.smithi040.stdout:38: writing smithi04040151-38 from 0 to 622592 tid 1 2024-09-16T14:48:36.971 INFO:tasks.rados.rados.0.smithi040.stdout:39: write initial oid 39 2024-09-16T14:48:36.971 INFO:tasks.rados.rados.0.smithi040.stdout:39: seq_num 38 ranges {0=638976} 2024-09-16T14:48:36.975 INFO:tasks.rados.rados.0.smithi040.stdout:39: writing smithi04040151-39 from 0 to 638976 tid 1 2024-09-16T14:48:36.975 INFO:tasks.rados.rados.0.smithi040.stdout:40: write initial oid 40 2024-09-16T14:48:36.975 INFO:tasks.rados.rados.0.smithi040.stdout:40: seq_num 39 ranges {0=671744} 2024-09-16T14:48:36.979 INFO:tasks.rados.rados.0.smithi040.stdout:40: writing smithi04040151-40 from 0 to 671744 tid 1 2024-09-16T14:48:36.979 INFO:tasks.rados.rados.0.smithi040.stdout:41: write initial oid 41 2024-09-16T14:48:36.979 INFO:tasks.rados.rados.0.smithi040.stdout:41: seq_num 40 ranges {0=688128} 2024-09-16T14:48:36.985 INFO:tasks.rados.rados.0.smithi040.stdout:41: writing smithi04040151-41 from 0 to 688128 tid 1 2024-09-16T14:48:36.985 INFO:tasks.rados.rados.0.smithi040.stdout: waiting on 16 2024-09-16T14:48:36.985 INFO:tasks.rados.rados.0.smithi040.stdout:28: finishing write tid 3 to smithi04040151-28 2024-09-16T14:48:36.985 INFO:tasks.rados.rados.0.smithi040.stdout:28: oid 28 updating version 0 to 5 2024-09-16T14:48:36.985 INFO:tasks.rados.rados.0.smithi040.stdout:28: oid 28 updating version 5 to 6 2024-09-16T14:48:36.985 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 28 v 6 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-16T14:48:36.985 INFO:tasks.rados.rados.0.smithi040.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-09-16T14:48:36.985 INFO:tasks.rados.rados.0.smithi040.stdout:30: finishing write tid 1 to smithi04040151-30 2024-09-16T14:48:36.985 INFO:tasks.rados.rados.0.smithi040.stdout:30: finishing write tid 2 to smithi04040151-30 2024-09-16T14:48:36.986 INFO:tasks.rados.rados.0.smithi040.stdout:31: finishing write tid 1 to smithi04040151-31 2024-09-16T14:48:36.986 INFO:tasks.rados.rados.0.smithi040.stdout:30: finishing write tid 3 to smithi04040151-30 2024-09-16T14:48:36.986 INFO:tasks.rados.rados.0.smithi040.stdout:30: oid 30 updating version 0 to 8 2024-09-16T14:48:36.986 INFO:tasks.rados.rados.0.smithi040.stdout:30: oid 30 version 8 is already newer than 7 2024-09-16T14:48:36.986 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 30 v 8 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-16T14:48:36.986 INFO:tasks.rados.rados.0.smithi040.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-09-16T14:48:36.986 INFO:tasks.rados.rados.0.smithi040.stdout:31: finishing write tid 2 to smithi04040151-31 2024-09-16T14:48:36.986 INFO:tasks.rados.rados.0.smithi040.stdout:31: finishing write tid 3 to smithi04040151-31 2024-09-16T14:48:36.986 INFO:tasks.rados.rados.0.smithi040.stdout:31: oid 31 updating version 0 to 8 2024-09-16T14:48:36.986 INFO:tasks.rados.rados.0.smithi040.stdout:31: oid 31 version 8 is already newer than 7 2024-09-16T14:48:36.986 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 31 v 8 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-16T14:48:36.986 INFO:tasks.rados.rados.0.smithi040.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-09-16T14:48:36.986 INFO:tasks.rados.rados.0.smithi040.stdout:33: finishing write tid 1 to smithi04040151-33 2024-09-16T14:48:36.987 INFO:tasks.rados.rados.0.smithi040.stdout:33: finishing write tid 2 to smithi04040151-33 2024-09-16T14:48:36.987 INFO:tasks.rados.rados.0.smithi040.stdout:33: finishing write tid 3 to smithi04040151-33 2024-09-16T14:48:36.987 INFO:tasks.rados.rados.0.smithi040.stdout:33: oid 33 updating version 0 to 4 2024-09-16T14:48:36.987 INFO:tasks.rados.rados.0.smithi040.stdout:33: oid 33 version 4 is already newer than 3 2024-09-16T14:48:36.987 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 33 v 4 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-16T14:48:36.987 INFO:tasks.rados.rados.0.smithi040.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-09-16T14:48:36.987 INFO:tasks.rados.rados.0.smithi040.stdout:34: finishing write tid 1 to smithi04040151-34 2024-09-16T14:48:36.987 INFO:tasks.rados.rados.0.smithi040.stdout:34: finishing write tid 2 to smithi04040151-34 2024-09-16T14:48:36.987 INFO:tasks.rados.rados.0.smithi040.stdout:34: finishing write tid 3 to smithi04040151-34 2024-09-16T14:48:36.987 INFO:tasks.rados.rados.0.smithi040.stdout:34: oid 34 updating version 0 to 10 2024-09-16T14:48:36.987 INFO:tasks.rados.rados.0.smithi040.stdout:34: oid 34 version 10 is already newer than 9 2024-09-16T14:48:36.987 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 34 v 10 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-16T14:48:36.987 INFO:tasks.rados.rados.0.smithi040.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-09-16T14:48:36.988 INFO:tasks.rados.rados.0.smithi040.stdout:35: finishing write tid 1 to smithi04040151-35 2024-09-16T14:48:36.988 INFO:tasks.rados.rados.0.smithi040.stdout:35: finishing write tid 2 to smithi04040151-35 2024-09-16T14:48:36.988 INFO:tasks.rados.rados.0.smithi040.stdout:35: finishing write tid 3 to smithi04040151-35 2024-09-16T14:48:36.988 INFO:tasks.rados.rados.0.smithi040.stdout:35: oid 35 updating version 0 to 3 2024-09-16T14:48:36.988 INFO:tasks.rados.rados.0.smithi040.stdout:35: oid 35 updating version 3 to 4 2024-09-16T14:48:36.988 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 35 v 4 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-16T14:48:36.988 INFO:tasks.rados.rados.0.smithi040.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-09-16T14:48:36.988 INFO:tasks.rados.rados.0.smithi040.stdout:36: finishing write tid 1 to smithi04040151-36 2024-09-16T14:48:36.989 INFO:tasks.rados.rados.0.smithi040.stdout:36: finishing write tid 2 to smithi04040151-36 2024-09-16T14:48:36.989 INFO:tasks.rados.rados.0.smithi040.stdout:36: finishing write tid 3 to smithi04040151-36 2024-09-16T14:48:36.989 INFO:tasks.rados.rados.0.smithi040.stdout:36: oid 36 updating version 0 to 5 2024-09-16T14:48:36.989 INFO:tasks.rados.rados.0.smithi040.stdout:36: oid 36 updating version 5 to 6 2024-09-16T14:48:36.989 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 36 v 6 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-16T14:48:36.989 INFO:tasks.rados.rados.0.smithi040.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-09-16T14:48:36.989 INFO:tasks.rados.rados.0.smithi040.stdout:37: finishing write tid 1 to smithi04040151-37 2024-09-16T14:48:36.989 INFO:tasks.rados.rados.0.smithi040.stdout:28: done (15 left) 2024-09-16T14:48:36.989 INFO:tasks.rados.rados.0.smithi040.stdout:30: done (14 left) 2024-09-16T14:48:36.989 INFO:tasks.rados.rados.0.smithi040.stdout:31: done (13 left) 2024-09-16T14:48:36.989 INFO:tasks.rados.rados.0.smithi040.stdout:33: done (12 left) 2024-09-16T14:48:36.989 INFO:tasks.rados.rados.0.smithi040.stdout:34: done (11 left) 2024-09-16T14:48:36.989 INFO:tasks.rados.rados.0.smithi040.stdout:35: done (10 left) 2024-09-16T14:48:36.990 INFO:tasks.rados.rados.0.smithi040.stdout:36: done (9 left) 2024-09-16T14:48:36.990 INFO:tasks.rados.rados.0.smithi040.stdout:42: write initial oid 42 2024-09-16T14:48:36.990 INFO:tasks.rados.rados.0.smithi040.stdout:42: seq_num 41 ranges {0=704512} 2024-09-16T14:48:36.990 INFO:tasks.rados.rados.0.smithi040.stdout:42: writing smithi04040151-42 from 0 to 704512 tid 1 2024-09-16T14:48:36.990 INFO:tasks.rados.rados.0.smithi040.stdout:43: write initial oid 43 2024-09-16T14:48:36.990 INFO:tasks.rados.rados.0.smithi040.stdout:43: seq_num 42 ranges {0=720896} 2024-09-16T14:48:36.994 INFO:tasks.rados.rados.0.smithi040.stdout:43: writing smithi04040151-43 from 0 to 720896 tid 1 2024-09-16T14:48:36.994 INFO:tasks.rados.rados.0.smithi040.stdout:44: write initial oid 44 2024-09-16T14:48:36.994 INFO:tasks.rados.rados.0.smithi040.stdout:44: seq_num 43 ranges {0=737280} 2024-09-16T14:48:36.999 INFO:tasks.rados.rados.0.smithi040.stdout:44: writing smithi04040151-44 from 0 to 737280 tid 1 2024-09-16T14:48:36.999 INFO:tasks.rados.rados.0.smithi040.stdout:45: write initial oid 45 2024-09-16T14:48:36.999 INFO:tasks.rados.rados.0.smithi040.stdout:45: seq_num 44 ranges {0=753664} 2024-09-16T14:48:37.004 INFO:tasks.rados.rados.0.smithi040.stdout:45: writing smithi04040151-45 from 0 to 753664 tid 1 2024-09-16T14:48:37.004 INFO:tasks.rados.rados.0.smithi040.stdout:46: write initial oid 46 2024-09-16T14:48:37.004 INFO:tasks.rados.rados.0.smithi040.stdout:46: seq_num 45 ranges {0=770048} 2024-09-16T14:48:37.009 INFO:tasks.rados.rados.0.smithi040.stdout:46: writing smithi04040151-46 from 0 to 770048 tid 1 2024-09-16T14:48:37.009 INFO:tasks.rados.rados.0.smithi040.stdout:47: write initial oid 47 2024-09-16T14:48:37.009 INFO:tasks.rados.rados.0.smithi040.stdout:47: seq_num 46 ranges {0=786432} 2024-09-16T14:48:37.014 INFO:tasks.rados.rados.0.smithi040.stdout:47: writing smithi04040151-47 from 0 to 786432 tid 1 2024-09-16T14:48:37.014 INFO:tasks.rados.rados.0.smithi040.stdout:48: write initial oid 48 2024-09-16T14:48:37.014 INFO:tasks.rados.rados.0.smithi040.stdout:48: seq_num 47 ranges {0=425984,425984=376832} 2024-09-16T14:48:37.017 INFO:tasks.rados.rados.0.smithi040.stdout:48: writing smithi04040151-48 from 0 to 425984 tid 1 2024-09-16T14:48:37.022 INFO:tasks.rados.rados.0.smithi040.stdout:48: writing smithi04040151-48 from 425984 to 802816 tid 2 2024-09-16T14:48:37.022 INFO:tasks.rados.rados.0.smithi040.stdout: waiting on 16 2024-09-16T14:48:37.022 INFO:tasks.rados.rados.0.smithi040.stdout:37: finishing write tid 2 to smithi04040151-37 2024-09-16T14:48:37.022 INFO:tasks.rados.rados.0.smithi040.stdout:38: finishing write tid 1 to smithi04040151-38 2024-09-16T14:48:37.022 INFO:tasks.rados.rados.0.smithi040.stdout:37: finishing write tid 3 to smithi04040151-37 2024-09-16T14:48:37.022 INFO:tasks.rados.rados.0.smithi040.stdout:37: oid 37 updating version 0 to 8 2024-09-16T14:48:37.022 INFO:tasks.rados.rados.0.smithi040.stdout:37: oid 37 version 8 is already newer than 7 2024-09-16T14:48:37.022 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 37 v 8 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-16T14:48:37.022 INFO:tasks.rados.rados.0.smithi040.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-09-16T14:48:37.022 INFO:tasks.rados.rados.0.smithi040.stdout:38: finishing write tid 2 to smithi04040151-38 2024-09-16T14:48:37.023 INFO:tasks.rados.rados.0.smithi040.stdout:38: finishing write tid 3 to smithi04040151-38 2024-09-16T14:48:37.023 INFO:tasks.rados.rados.0.smithi040.stdout:38: oid 38 updating version 0 to 8 2024-09-16T14:48:37.023 INFO:tasks.rados.rados.0.smithi040.stdout:38: oid 38 version 8 is already newer than 7 2024-09-16T14:48:37.023 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 38 v 8 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-16T14:48:37.023 INFO:tasks.rados.rados.0.smithi040.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-09-16T14:48:37.023 INFO:tasks.rados.rados.0.smithi040.stdout:39: finishing write tid 1 to smithi04040151-39 2024-09-16T14:48:37.023 INFO:tasks.rados.rados.0.smithi040.stdout:39: finishing write tid 2 to smithi04040151-39 2024-09-16T14:48:37.023 INFO:tasks.rados.rados.0.smithi040.stdout:39: finishing write tid 3 to smithi04040151-39 2024-09-16T14:48:37.023 INFO:tasks.rados.rados.0.smithi040.stdout:39: oid 39 updating version 0 to 1 2024-09-16T14:48:37.023 INFO:tasks.rados.rados.0.smithi040.stdout:39: oid 39 updating version 1 to 2 2024-09-16T14:48:37.023 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 39 v 2 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-16T14:48:37.023 INFO:tasks.rados.rados.0.smithi040.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-09-16T14:48:37.023 INFO:tasks.rados.rados.0.smithi040.stdout:40: finishing write tid 1 to smithi04040151-40 2024-09-16T14:48:37.023 INFO:tasks.rados.rados.0.smithi040.stdout:40: finishing write tid 2 to smithi04040151-40 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:41: finishing write tid 1 to smithi04040151-41 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:41: finishing write tid 2 to smithi04040151-41 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:40: finishing write tid 3 to smithi04040151-40 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:40: oid 40 updating version 0 to 10 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:40: oid 40 version 10 is already newer than 9 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 40 v 10 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:41: finishing write tid 3 to smithi04040151-41 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:41: oid 41 updating version 0 to 4 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:41: oid 41 version 4 is already newer than 3 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 41 v 4 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:42: finishing write tid 1 to smithi04040151-42 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:42: finishing write tid 2 to smithi04040151-42 2024-09-16T14:48:37.024 INFO:tasks.rados.rados.0.smithi040.stdout:42: finishing write tid 3 to smithi04040151-42 2024-09-16T14:48:37.025 INFO:tasks.rados.rados.0.smithi040.stdout:42: oid 42 updating version 0 to 7 2024-09-16T14:48:37.025 INFO:tasks.rados.rados.0.smithi040.stdout:42: oid 42 updating version 7 to 8 2024-09-16T14:48:37.025 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 42 v 8 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-16T14:48:37.025 INFO:tasks.rados.rados.0.smithi040.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-09-16T14:48:37.025 INFO:tasks.rados.rados.0.smithi040.stdout:43: finishing write tid 1 to smithi04040151-43 2024-09-16T14:48:37.025 INFO:tasks.rados.rados.0.smithi040.stdout:43: finishing write tid 2 to smithi04040151-43 2024-09-16T14:48:37.025 INFO:tasks.rados.rados.0.smithi040.stdout:43: finishing write tid 3 to smithi04040151-43 2024-09-16T14:48:37.025 INFO:tasks.rados.rados.0.smithi040.stdout:43: oid 43 updating version 0 to 10 2024-09-16T14:48:37.025 INFO:tasks.rados.rados.0.smithi040.stdout:43: oid 43 version 10 is already newer than 9 2024-09-16T14:48:37.025 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 43 v 10 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-16T14:48:37.025 INFO:tasks.rados.rados.0.smithi040.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-09-16T14:48:37.025 INFO:tasks.rados.rados.0.smithi040.stdout:44: finishing write tid 1 to smithi04040151-44 2024-09-16T14:48:37.025 INFO:tasks.rados.rados.0.smithi040.stdout:44: finishing write tid 2 to smithi04040151-44 2024-09-16T14:48:37.025 INFO:tasks.rados.rados.0.smithi040.stdout:45: finishing write tid 1 to smithi04040151-45 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:45: finishing write tid 2 to smithi04040151-45 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:44: finishing write tid 3 to smithi04040151-44 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:44: oid 44 updating version 0 to 5 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:44: oid 44 updating version 5 to 6 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 44 v 6 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:45: finishing write tid 3 to smithi04040151-45 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:45: oid 45 updating version 0 to 3 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:45: oid 45 updating version 3 to 4 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 45 v 4 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:46: finishing write tid 1 to smithi04040151-46 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:46: finishing write tid 2 to smithi04040151-46 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:46: finishing write tid 3 to smithi04040151-46 2024-09-16T14:48:37.026 INFO:tasks.rados.rados.0.smithi040.stdout:46: oid 46 updating version 0 to 12 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:46: oid 46 version 12 is already newer than 11 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 46 v 12 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:37: done (15 left) 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:38: done (14 left) 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:39: done (13 left) 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:40: done (12 left) 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:41: done (11 left) 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:42: done (10 left) 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:43: done (9 left) 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:44: done (8 left) 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:45: done (7 left) 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:46: done (6 left) 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:49: write initial oid 49 2024-09-16T14:48:37.027 INFO:tasks.rados.rados.0.smithi040.stdout:49: seq_num 48 ranges {0=442368,442368=376832} 2024-09-16T14:48:37.028 INFO:tasks.rados.rados.0.smithi040.stdout:49: writing smithi04040151-49 from 0 to 442368 tid 1 2024-09-16T14:48:37.028 INFO:tasks.rados.rados.0.smithi040.stdout:49: writing smithi04040151-49 from 442368 to 819200 tid 2 2024-09-16T14:48:37.028 INFO:tasks.rados.rados.0.smithi040.stdout:50: write initial oid 50 2024-09-16T14:48:37.028 INFO:tasks.rados.rados.0.smithi040.stdout:50: seq_num 49 ranges {0=458752,458752=376832} 2024-09-16T14:48:37.029 INFO:tasks.rados.rados.0.smithi040.stdout:50: writing smithi04040151-50 from 0 to 458752 tid 1 2024-09-16T14:48:37.032 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:37.030+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:37.033 INFO:tasks.rados.rados.0.smithi040.stdout:50: writing smithi04040151-50 from 458752 to 835584 tid 2 2024-09-16T14:48:37.033 INFO:tasks.rados.rados.0.smithi040.stdout:51: read oid 30 snap -1 2024-09-16T14:48:37.033 INFO:tasks.rados.rados.0.smithi040.stdout:51: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-16T14:48:37.033 INFO:tasks.rados.rados.0.smithi040.stdout:52: read oid 16 snap -1 2024-09-16T14:48:37.033 INFO:tasks.rados.rados.0.smithi040.stdout:52: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-16T14:48:37.034 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 37 current snap is 0 2024-09-16T14:48:37.034 INFO:tasks.rados.rados.0.smithi040.stdout:47: finishing write tid 1 to smithi04040151-47 2024-09-16T14:48:37.034 INFO:tasks.rados.rados.0.smithi040.stdout:47: finishing write tid 2 to smithi04040151-47 2024-09-16T14:48:37.034 INFO:tasks.rados.rados.0.smithi040.stdout:48: finishing write tid 1 to smithi04040151-48 2024-09-16T14:48:37.034 INFO:tasks.rados.rados.0.smithi040.stdout:47: finishing write tid 3 to smithi04040151-47 2024-09-16T14:48:37.034 INFO:tasks.rados.rados.0.smithi040.stdout:47: oid 47 updating version 0 to 7 2024-09-16T14:48:37.034 INFO:tasks.rados.rados.0.smithi040.stdout:47: oid 47 updating version 7 to 8 2024-09-16T14:48:37.034 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 47 v 8 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-16T14:48:37.034 INFO:tasks.rados.rados.0.smithi040.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-09-16T14:48:37.034 INFO:tasks.rados.rados.0.smithi040.stdout:48: finishing write tid 2 to smithi04040151-48 2024-09-16T14:48:37.034 INFO:tasks.rados.rados.0.smithi040.stdout:48: finishing write tid 3 to smithi04040151-48 2024-09-16T14:48:37.034 INFO:tasks.rados.rados.0.smithi040.stdout:48: finishing write tid 4 to smithi04040151-48 2024-09-16T14:48:37.035 INFO:tasks.rados.rados.0.smithi040.stdout:48: oid 48 updating version 0 to 7 2024-09-16T14:48:37.035 INFO:tasks.rados.rados.0.smithi040.stdout:48: oid 48 version 7 is already newer than 6 2024-09-16T14:48:37.035 INFO:tasks.rados.rados.0.smithi040.stdout:48: oid 48 version 7 is already newer than 5 2024-09-16T14:48:37.035 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 48 v 7 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-16T14:48:37.035 INFO:tasks.rados.rados.0.smithi040.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-09-16T14:48:37.035 INFO:tasks.rados.rados.0.smithi040.stdout:53: seq_num 50 ranges {606208=475136,1081344=376832} 2024-09-16T14:48:37.039 INFO:tasks.rados.rados.0.smithi040.stdout:53: writing smithi04040151-37 from 606208 to 1081344 tid 1 2024-09-16T14:48:37.042 INFO:tasks.rados.rados.0.smithi040.stdout:53: writing smithi04040151-37 from 1081344 to 1458176 tid 2 2024-09-16T14:48:37.042 INFO:tasks.rados.rados.0.smithi040.stdout:47: done (10 left) 2024-09-16T14:48:37.042 INFO:tasks.rados.rados.0.smithi040.stdout:48: done (9 left) 2024-09-16T14:48:37.042 INFO:tasks.rados.rados.0.smithi040.stdout:54: copy_from oid 31 from oid 13 current snap is 0 2024-09-16T14:48:37.042 INFO:tasks.rados.rados.0.smithi040.stdout:49: finishing write tid 1 to smithi04040151-49 2024-09-16T14:48:37.043 INFO:tasks.rados.rados.0.smithi040.stdout:52: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-16T14:48:37.054 INFO:tasks.rados.rados.0.smithi040.stdout:49: finishing write tid 2 to smithi04040151-49 2024-09-16T14:48:37.054 INFO:tasks.rados.rados.0.smithi040.stdout:49: finishing write tid 3 to smithi04040151-49 2024-09-16T14:48:37.054 INFO:tasks.rados.rados.0.smithi040.stdout:52: done (9 left) 2024-09-16T14:48:37.054 INFO:tasks.rados.rados.0.smithi040.stdout:55: delete oid 15 current snap is 0 2024-09-16T14:48:37.054 INFO:tasks.rados.rados.0.smithi040.stdout:51: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-16T14:48:37.070 INFO:tasks.rados.rados.0.smithi040.stdout:49: finishing write tid 4 to smithi04040151-49 2024-09-16T14:48:37.071 INFO:tasks.rados.rados.0.smithi040.stdout:49: oid 49 updating version 0 to 3 2024-09-16T14:48:37.071 INFO:tasks.rados.rados.0.smithi040.stdout:49: oid 49 updating version 3 to 5 2024-09-16T14:48:37.071 INFO:tasks.rados.rados.0.smithi040.stdout:49: oid 49 version 5 is already newer than 4 2024-09-16T14:48:37.071 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 49 v 5 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-16T14:48:37.071 INFO:tasks.rados.rados.0.smithi040.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-09-16T14:48:37.071 INFO:tasks.rados.rados.0.smithi040.stdout:50: finishing write tid 1 to smithi04040151-50 2024-09-16T14:48:37.071 INFO:tasks.rados.rados.0.smithi040.stdout:50: finishing write tid 2 to smithi04040151-50 2024-09-16T14:48:37.071 INFO:tasks.rados.rados.0.smithi040.stdout:50: finishing write tid 3 to smithi04040151-50 2024-09-16T14:48:37.071 INFO:tasks.rados.rados.0.smithi040.stdout:50: finishing write tid 4 to smithi04040151-50 2024-09-16T14:48:37.071 INFO:tasks.rados.rados.0.smithi040.stdout:50: oid 50 updating version 0 to 6 2024-09-16T14:48:37.072 INFO:tasks.rados.rados.0.smithi040.stdout:50: oid 50 updating version 6 to 7 2024-09-16T14:48:37.072 INFO:tasks.rados.rados.0.smithi040.stdout:50: oid 50 updating version 7 to 8 2024-09-16T14:48:37.072 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 50 v 8 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-16T14:48:37.072 INFO:tasks.rados.rados.0.smithi040.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-09-16T14:48:37.072 INFO:tasks.rados.rados.0.smithi040.stdout:53: finishing write tid 1 to smithi04040151-37 2024-09-16T14:48:37.072 INFO:tasks.rados.rados.0.smithi040.stdout:53: finishing write tid 2 to smithi04040151-37 2024-09-16T14:48:37.072 INFO:tasks.rados.rados.0.smithi040.stdout:53: finishing write tid 3 to smithi04040151-37 2024-09-16T14:48:37.072 INFO:tasks.rados.rados.0.smithi040.stdout:53: finishing write tid 4 to smithi04040151-37 2024-09-16T14:48:37.072 INFO:tasks.rados.rados.0.smithi040.stdout:53: oid 37 updating version 0 to 13 2024-09-16T14:48:37.073 INFO:tasks.rados.rados.0.smithi040.stdout:53: oid 37 version 13 is already newer than 12 2024-09-16T14:48:37.073 INFO:tasks.rados.rados.0.smithi040.stdout:53: oid 37 version 13 is already newer than 11 2024-09-16T14:48:37.073 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 37 v 13 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-16T14:48:37.073 INFO:tasks.rados.rados.0.smithi040.stdout:53: left oid 37 (ObjNum 50 snap 0 seq_num 50) 2024-09-16T14:48:37.073 INFO:tasks.rados.rados.0.smithi040.stdout:54: finishing copy_from racing read to smithi04040151-31 2024-09-16T14:48:37.073 INFO:tasks.rados.rados.0.smithi040.stdout:54: finishing copy_from to smithi04040151-31 2024-09-16T14:48:37.073 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 31 v 4 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-16T14:48:37.073 INFO:tasks.rados.rados.0.smithi040.stdout:49: done (9 left) 2024-09-16T14:48:37.073 INFO:tasks.rados.rados.0.smithi040.stdout:50: done (8 left) 2024-09-16T14:48:37.073 INFO:tasks.rados.rados.0.smithi040.stdout:51: done (7 left) 2024-09-16T14:48:37.073 INFO:tasks.rados.rados.0.smithi040.stdout:53: done (6 left) 2024-09-16T14:48:37.073 INFO:tasks.rados.rados.0.smithi040.stdout:54: done (5 left) 2024-09-16T14:48:37.073 INFO:tasks.rados.rados.0.smithi040.stdout:55: done (4 left) 2024-09-16T14:48:37.074 INFO:tasks.rados.rados.0.smithi040.stdout:56: snap_create 2024-09-16T14:48:37.083 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:37.083 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:37.092 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:48:37.099 INFO:tasks.rados.rados.0.smithi040.stdout:12: finishing write tid 1 to smithi04040151-12 2024-09-16T14:48:37.099 INFO:tasks.rados.rados.0.smithi040.stdout:12: finishing write tid 2 to smithi04040151-12 2024-09-16T14:48:37.102 INFO:tasks.rados.rados.0.smithi040.stdout:27: finishing write tid 1 to smithi04040151-27 2024-09-16T14:48:37.102 INFO:tasks.rados.rados.0.smithi040.stdout:27: finishing write tid 2 to smithi04040151-27 2024-09-16T14:48:37.103 INFO:tasks.rados.rados.0.smithi040.stdout:29: finishing write tid 1 to smithi04040151-29 2024-09-16T14:48:37.103 INFO:tasks.rados.rados.0.smithi040.stdout:12: finishing write tid 3 to smithi04040151-12 2024-09-16T14:48:37.104 INFO:tasks.rados.rados.0.smithi040.stdout:12: oid 12 updating version 0 to 1 2024-09-16T14:48:37.104 INFO:tasks.rados.rados.0.smithi040.stdout:12: oid 12 updating version 1 to 2 2024-09-16T14:48:37.104 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 12 v 2 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-16T14:48:37.104 INFO:tasks.rados.rados.0.smithi040.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-09-16T14:48:37.104 INFO:tasks.rados.rados.0.smithi040.stdout:27: finishing write tid 3 to smithi04040151-27 2024-09-16T14:48:37.104 INFO:tasks.rados.rados.0.smithi040.stdout:27: oid 27 updating version 0 to 4 2024-09-16T14:48:37.104 INFO:tasks.rados.rados.0.smithi040.stdout:27: oid 27 version 4 is already newer than 3 2024-09-16T14:48:37.105 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 27 v 4 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-16T14:48:37.105 INFO:tasks.rados.rados.0.smithi040.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-09-16T14:48:37.108 INFO:tasks.rados.rados.0.smithi040.stdout:29: finishing write tid 2 to smithi04040151-29 2024-09-16T14:48:37.109 INFO:tasks.rados.rados.0.smithi040.stdout:29: finishing write tid 3 to smithi04040151-29 2024-09-16T14:48:37.109 INFO:tasks.rados.rados.0.smithi040.stdout:29: oid 29 updating version 0 to 6 2024-09-16T14:48:37.109 INFO:tasks.rados.rados.0.smithi040.stdout:29: oid 29 version 6 is already newer than 5 2024-09-16T14:48:37.109 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 29 v 6 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-16T14:48:37.109 INFO:tasks.rados.rados.0.smithi040.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-09-16T14:48:37.110 INFO:tasks.rados.rados.0.smithi040.stdout:32: finishing write tid 1 to smithi04040151-32 2024-09-16T14:48:37.110 INFO:tasks.rados.rados.0.smithi040.stdout:32: finishing write tid 2 to smithi04040151-32 2024-09-16T14:48:37.113 INFO:tasks.rados.rados.0.smithi040.stdout:32: finishing write tid 3 to smithi04040151-32 2024-09-16T14:48:37.113 INFO:tasks.rados.rados.0.smithi040.stdout:32: oid 32 updating version 0 to 8 2024-09-16T14:48:37.113 INFO:tasks.rados.rados.0.smithi040.stdout:32: oid 32 version 8 is already newer than 7 2024-09-16T14:48:37.114 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 32 v 8 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-16T14:48:37.114 INFO:tasks.rados.rados.0.smithi040.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-09-16T14:48:37.133 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:37.132+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:37.205 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:37.205 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:37.214 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:48:37.233 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:37.232+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:37.317 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:37.317 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:37.326 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:48:37.335 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:37.334+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:37.353 INFO:teuthology.orchestra.run.smithi111.stdout: marking collection for removal 2024-09-16T14:48:37.353 INFO:teuthology.orchestra.run.smithi111.stdout:setting '_remove' omap key 2024-09-16T14:48:37.353 INFO:teuthology.orchestra.run.smithi111.stdout:finish_remove_pgs 2.7_head removing 2.7 2024-09-16T14:48:37.355 INFO:teuthology.orchestra.run.smithi111.stdout:Remove successful 2024-09-16T14:48:37.428 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:37.428 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:37.435 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:37.434+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:37.437 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:48:37.537 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:37.535+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:37.578 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:37.578 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:37.586 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:48:37.638 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:37.636+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:37.690 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:37.690 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:37.698 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:48:37.739 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:37.737+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:37.799 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:37.799 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:37.808 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:48:37.839 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:37.838+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:37.880 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.7.6 2024-09-16T14:48:37.883 INFO:tasks.rados.rados.0.smithi040.stdout:12: done (4 left) 2024-09-16T14:48:37.883 INFO:tasks.rados.rados.0.smithi040.stdout:27: done (3 left) 2024-09-16T14:48:37.883 INFO:tasks.rados.rados.0.smithi040.stdout:29: done (2 left) 2024-09-16T14:48:37.883 INFO:tasks.rados.rados.0.smithi040.stdout:32: done (1 left) 2024-09-16T14:48:37.883 INFO:tasks.rados.rados.0.smithi040.stdout:56: done (0 left) 2024-09-16T14:48:37.883 INFO:tasks.rados.rados.0.smithi040.stdout:57: read oid 7 snap 0 2024-09-16T14:48:37.883 INFO:tasks.rados.rados.0.smithi040.stdout:57: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-16T14:48:37.884 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 6 current snap is 1 2024-09-16T14:48:37.884 INFO:tasks.rados.rados.0.smithi040.stdout:58: seq_num 52 ranges {98304=507904,606208=376832} 2024-09-16T14:48:37.884 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is unset 2024-09-16T14:48:37.886 INFO:tasks.rados.rados.0.smithi040.stdout:58: writing smithi04040151-6 from 98304 to 606208 tid 1 2024-09-16T14:48:37.888 INFO:tasks.rados.rados.0.smithi040.stdout:58: writing smithi04040151-6 from 606208 to 983040 tid 2 2024-09-16T14:48:37.888 INFO:tasks.rados.rados.0.smithi040.stdout:59: copy_from oid 23 from oid 50 current snap is 1 2024-09-16T14:48:37.889 INFO:tasks.rados.rados.0.smithi040.stdout:60: copy_from oid 15 from oid 44 current snap is 1 2024-09-16T14:48:37.889 INFO:tasks.rados.rados.0.smithi040.stdout:61: copy_from oid 3 from oid 20 current snap is 1 2024-09-16T14:48:37.889 INFO:tasks.rados.rados.0.smithi040.stdout:62: delete oid 26 current snap is 1 2024-09-16T14:48:37.889 INFO:tasks.rados.rados.0.smithi040.stdout:57: expect (ObjNum 6 snap 0 seq_num 6) 2024-09-16T14:48:37.896 INFO:tasks.rados.rados.0.smithi040.stdout:58: finishing write tid 1 to smithi04040151-6 2024-09-16T14:48:37.898 INFO:tasks.rados.rados.0.smithi040.stdout:57: done (5 left) 2024-09-16T14:48:37.898 INFO:tasks.rados.rados.0.smithi040.stdout:62: done (4 left) 2024-09-16T14:48:37.898 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 14 current snap is 1 2024-09-16T14:48:37.898 INFO:tasks.rados.rados.0.smithi040.stdout:63: seq_num 54 ranges {229376=540672,770048=376832} 2024-09-16T14:48:37.903 INFO:tasks.rados.rados.0.smithi040.stdout:63: writing smithi04040151-14 from 229376 to 770048 tid 1 2024-09-16T14:48:37.906 INFO:tasks.rados.rados.0.smithi040.stdout:63: writing smithi04040151-14 from 770048 to 1146880 tid 2 2024-09-16T14:48:37.906 INFO:tasks.rados.rados.0.smithi040.stdout:64: read oid 28 snap -1 2024-09-16T14:48:37.906 INFO:tasks.rados.rados.0.smithi040.stdout:64: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-16T14:48:37.906 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 31 current snap is 1 2024-09-16T14:48:37.906 INFO:tasks.rados.rados.0.smithi040.stdout:65: seq_num 55 ranges {212992=557056,770048=376832} 2024-09-16T14:48:37.911 INFO:tasks.rados.rados.0.smithi040.stdout:65: writing smithi04040151-31 from 212992 to 770048 tid 1 2024-09-16T14:48:37.914 INFO:tasks.rados.rados.0.smithi040.stdout:65: writing smithi04040151-31 from 770048 to 1146880 tid 2 2024-09-16T14:48:37.914 INFO:tasks.rados.rados.0.smithi040.stdout:66: rollback oid 1 current snap is 1 2024-09-16T14:48:37.914 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 1 to 0 2024-09-16T14:48:37.914 INFO:tasks.rados.rados.0.smithi040.stdout:61: finishing copy_from racing read to smithi04040151-3 2024-09-16T14:48:37.915 INFO:tasks.rados.rados.0.smithi040.stdout:58: finishing write tid 2 to smithi04040151-6 2024-09-16T14:48:37.915 INFO:tasks.rados.rados.0.smithi040.stdout:58: finishing write tid 3 to smithi04040151-6 2024-09-16T14:48:37.915 INFO:tasks.rados.rados.0.smithi040.stdout:59: finishing copy_from racing read to smithi04040151-23 2024-09-16T14:48:37.915 INFO:tasks.rados.rados.0.smithi040.stdout:60: finishing copy_from racing read to smithi04040151-15 2024-09-16T14:48:37.915 INFO:tasks.rados.rados.0.smithi040.stdout:60: finishing copy_from to smithi04040151-15 2024-09-16T14:48:37.915 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 15 v 6 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-16T14:48:37.915 INFO:tasks.rados.rados.0.smithi040.stdout:61: finishing copy_from to smithi04040151-3 2024-09-16T14:48:37.915 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 3 v 2 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-16T14:48:37.915 INFO:tasks.rados.rados.0.smithi040.stdout:58: finishing write tid 4 to smithi04040151-6 2024-09-16T14:48:37.915 INFO:tasks.rados.rados.0.smithi040.stdout:58: oid 6 updating version 0 to 17 2024-09-16T14:48:37.915 INFO:tasks.rados.rados.0.smithi040.stdout:58: oid 6 version 17 is already newer than 16 2024-09-16T14:48:37.916 INFO:tasks.rados.rados.0.smithi040.stdout:58: oid 6 version 17 is already newer than 15 2024-09-16T14:48:37.916 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 6 v 17 (ObjNum 52 snap 1 seq_num 52) dirty exists 2024-09-16T14:48:37.916 INFO:tasks.rados.rados.0.smithi040.stdout:58: left oid 6 (ObjNum 52 snap 1 seq_num 52) 2024-09-16T14:48:37.916 INFO:tasks.rados.rados.0.smithi040.stdout:63: finishing write tid 1 to smithi04040151-14 2024-09-16T14:48:37.916 INFO:tasks.rados.rados.0.smithi040.stdout:59: finishing copy_from to smithi04040151-23 2024-09-16T14:48:37.916 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 23 v 8 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-16T14:48:37.916 INFO:tasks.rados.rados.0.smithi040.stdout:64: expect (ObjNum 27 snap 0 seq_num 27) 2024-09-16T14:48:37.924 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:37.924 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:37.929 INFO:tasks.rados.rados.0.smithi040.stdout:63: finishing write tid 2 to smithi04040151-14 2024-09-16T14:48:37.929 INFO:tasks.rados.rados.0.smithi040.stdout:63: finishing write tid 3 to smithi04040151-14 2024-09-16T14:48:37.930 INFO:tasks.rados.rados.0.smithi040.stdout:63: finishing write tid 4 to smithi04040151-14 2024-09-16T14:48:37.930 INFO:tasks.rados.rados.0.smithi040.stdout:63: oid 14 updating version 0 to 21 2024-09-16T14:48:37.930 INFO:tasks.rados.rados.0.smithi040.stdout:63: oid 14 version 21 is already newer than 20 2024-09-16T14:48:37.930 INFO:tasks.rados.rados.0.smithi040.stdout:63: oid 14 version 21 is already newer than 19 2024-09-16T14:48:37.930 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 14 v 21 (ObjNum 54 snap 1 seq_num 54) dirty exists 2024-09-16T14:48:37.930 INFO:tasks.rados.rados.0.smithi040.stdout:63: left oid 14 (ObjNum 54 snap 1 seq_num 54) 2024-09-16T14:48:37.930 INFO:tasks.rados.rados.0.smithi040.stdout:65: finishing write tid 1 to smithi04040151-31 2024-09-16T14:48:37.930 INFO:tasks.rados.rados.0.smithi040.stdout:58: done (7 left) 2024-09-16T14:48:37.930 INFO:tasks.rados.rados.0.smithi040.stdout:59: done (6 left) 2024-09-16T14:48:37.930 INFO:tasks.rados.rados.0.smithi040.stdout:60: done (5 left) 2024-09-16T14:48:37.930 INFO:tasks.rados.rados.0.smithi040.stdout:61: done (4 left) 2024-09-16T14:48:37.931 INFO:tasks.rados.rados.0.smithi040.stdout:63: done (3 left) 2024-09-16T14:48:37.931 INFO:tasks.rados.rados.0.smithi040.stdout:64: done (2 left) 2024-09-16T14:48:37.931 INFO:tasks.rados.rados.0.smithi040.stdout:67: rollback oid 7 current snap is 1 2024-09-16T14:48:37.931 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 7 to 0 2024-09-16T14:48:37.931 INFO:tasks.rados.rados.0.smithi040.stdout:66: finishing rollback tid 0 to smithi04040151-1 2024-09-16T14:48:37.931 INFO:tasks.rados.rados.0.smithi040.stdout:65: finishing write tid 2 to smithi04040151-31 2024-09-16T14:48:37.931 INFO:tasks.rados.rados.0.smithi040.stdout:66: finishing rollback tid 1 to smithi04040151-1 2024-09-16T14:48:37.931 INFO:tasks.rados.rados.0.smithi040.stdout:65: finishing write tid 3 to smithi04040151-31 2024-09-16T14:48:37.931 INFO:tasks.rados.rados.0.smithi040.stdout:66: finishing rollback tid 2 to smithi04040151-1 2024-09-16T14:48:37.931 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 1 v 12 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-16T14:48:37.931 INFO:tasks.rados.rados.0.smithi040.stdout:65: finishing write tid 4 to smithi04040151-31 2024-09-16T14:48:37.932 INFO:tasks.rados.rados.0.smithi040.stdout:65: oid 31 updating version 0 to 17 2024-09-16T14:48:37.932 INFO:tasks.rados.rados.0.smithi040.stdout:65: oid 31 updating version 17 to 19 2024-09-16T14:48:37.932 INFO:tasks.rados.rados.0.smithi040.stdout:65: oid 31 version 19 is already newer than 18 2024-09-16T14:48:37.932 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 31 v 19 (ObjNum 55 snap 1 seq_num 55) dirty exists 2024-09-16T14:48:37.932 INFO:tasks.rados.rados.0.smithi040.stdout:65: left oid 31 (ObjNum 55 snap 1 seq_num 55) 2024-09-16T14:48:37.932 INFO:tasks.rados.rados.0.smithi040.stdout:65: done (2 left) 2024-09-16T14:48:37.932 INFO:tasks.rados.rados.0.smithi040.stdout:66: done (1 left) 2024-09-16T14:48:37.932 INFO:tasks.rados.rados.0.smithi040.stdout:68: rollback oid 29 current snap is 1 2024-09-16T14:48:37.932 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 29 to 0 2024-09-16T14:48:37.932 INFO:tasks.rados.rados.0.smithi040.stdout:69: read oid 4 snap 0 2024-09-16T14:48:37.932 INFO:tasks.rados.rados.0.smithi040.stdout:69: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-16T14:48:37.933 INFO:tasks.rados.rados.0.smithi040.stdout:70: setattr oid 5 current snap is 1 2024-09-16T14:48:37.933 INFO:tasks.rados.rados.0.smithi040.stdout:71: delete oid 26 current snap is 1 2024-09-16T14:48:37.933 INFO:tasks.rados.rados.0.smithi040.stdout:69: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-16T14:48:37.935 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:48:37.936 INFO:tasks.rados.rados.0.smithi040.stdout:68: finishing rollback tid 0 to smithi04040151-29 2024-09-16T14:48:37.936 INFO:tasks.rados.rados.0.smithi040.stdout:67: finishing rollback tid 0 to smithi04040151-7 2024-09-16T14:48:37.939 INFO:tasks.rados.rados.0.smithi040.stdout:69: done (4 left) 2024-09-16T14:48:37.939 INFO:tasks.rados.rados.0.smithi040.stdout:71: done (3 left) 2024-09-16T14:48:37.940 INFO:tasks.rados.rados.0.smithi040.stdout:72: read oid 10 snap -1 2024-09-16T14:48:37.940 INFO:tasks.rados.rados.0.smithi040.stdout:72: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-16T14:48:37.940 INFO:tasks.rados.rados.0.smithi040.stdout:73: snap_create 2024-09-16T14:48:37.940 INFO:tasks.rados.rados.0.smithi040.stdout:68: finishing rollback tid 1 to smithi04040151-29 2024-09-16T14:48:37.940 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 5 v 6 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-16T14:48:37.940 INFO:tasks.rados.rados.0.smithi040.stdout:68: finishing rollback tid 2 to smithi04040151-29 2024-09-16T14:48:37.940 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 29 v 12 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-16T14:48:37.940 INFO:tasks.rados.rados.0.smithi040.stdout:67: finishing rollback tid 1 to smithi04040151-7 2024-09-16T14:48:37.941 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:37.940+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:37.942 INFO:tasks.rados.rados.0.smithi040.stdout:67: finishing rollback tid 2 to smithi04040151-7 2024-09-16T14:48:37.942 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 7 v 8 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-16T14:48:37.943 INFO:tasks.rados.rados.0.smithi040.stdout:72: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-16T14:48:38.042 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:38.041+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:38.078 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:38.079 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:38.087 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:48:38.144 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:38.143+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:38.190 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:38.190 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:38.198 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:48:38.283 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:38.282+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:38.342 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:38.342 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:38.346 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:38.345+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:38.350 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:48:38.369 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:48:38.447 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:38.446+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:38.464 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:38.464 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:38.474 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:48:38.504 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:48:38.505 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:48:38.521 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:48:38.521 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:48:38.537 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:48:38.537 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:48:38.547 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:38.546+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:38.552 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:48:38.553 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:48:38.569 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:48:38.569 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:48:38.576 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:38.577 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:38.586 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:48:38.587 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:48:38.587 INFO:teuthology.orchestra.run.smithi040.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T14:48:38.588 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:48:38.602 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:48:38.602 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:48:38.619 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:48:38.619 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:48:38.635 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:48:38.636 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:48:38.648 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:38.647+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:38.653 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:48:38.653 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:48:38.668 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:48:38.668 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:48:38.671 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:48:38.671 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:48:38.730 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:38.731 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:38.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:38.731 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [] 2024-09-16T14:48:38.731 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:38.739 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:48:38.782 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:38.781+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:38.841 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:38.842 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:38.842 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:38.842 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:38.849 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:38.848+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:38.850 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:48:38.891 INFO:tasks.rados.rados.0.smithi040.stdout:67: done (4 left) 2024-09-16T14:48:38.891 INFO:tasks.rados.rados.0.smithi040.stdout:68: done (3 left) 2024-09-16T14:48:38.892 INFO:tasks.rados.rados.0.smithi040.stdout:70: done (2 left) 2024-09-16T14:48:38.892 INFO:tasks.rados.rados.0.smithi040.stdout:72: done (1 left) 2024-09-16T14:48:38.892 INFO:tasks.rados.rados.0.smithi040.stdout:73: done (0 left) 2024-09-16T14:48:38.892 INFO:tasks.rados.rados.0.smithi040.stdout:74: read oid 6 snap -1 2024-09-16T14:48:38.892 INFO:tasks.rados.rados.0.smithi040.stdout:74: expect (ObjNum 52 snap 1 seq_num 52) 2024-09-16T14:48:38.892 INFO:tasks.rados.rados.0.smithi040.stdout:75: rmattr oid 33 current snap is 2 2024-09-16T14:48:38.892 INFO:tasks.rados.rados.0.smithi040.stdout:75: done (1 left) 2024-09-16T14:48:38.892 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 28 current snap is 2 2024-09-16T14:48:38.892 INFO:tasks.rados.rados.0.smithi040.stdout:76: seq_num 57 ranges {458752=589824,1048576=376832} 2024-09-16T14:48:38.898 INFO:tasks.rados.rados.0.smithi040.stdout:76: writing smithi04040151-28 from 458752 to 1048576 tid 1 2024-09-16T14:48:38.900 INFO:tasks.rados.rados.0.smithi040.stdout:76: writing smithi04040151-28 from 1048576 to 1425408 tid 2 2024-09-16T14:48:38.900 INFO:tasks.rados.rados.0.smithi040.stdout:77: snap_create 2024-09-16T14:48:38.901 INFO:tasks.rados.rados.0.smithi040.stdout:74: expect (ObjNum 52 snap 1 seq_num 52) 2024-09-16T14:48:38.929 INFO:tasks.rados.rados.0.smithi040.stdout:76: finishing write tid 1 to smithi04040151-28 2024-09-16T14:48:38.930 INFO:tasks.rados.rados.0.smithi040.stdout:76: finishing write tid 2 to smithi04040151-28 2024-09-16T14:48:38.930 INFO:tasks.rados.rados.0.smithi040.stdout:76: finishing write tid 3 to smithi04040151-28 2024-09-16T14:48:38.930 INFO:tasks.rados.rados.0.smithi040.stdout:76: finishing write tid 4 to smithi04040151-28 2024-09-16T14:48:38.930 INFO:tasks.rados.rados.0.smithi040.stdout:76: oid 28 updating version 0 to 12 2024-09-16T14:48:38.930 INFO:tasks.rados.rados.0.smithi040.stdout:76: oid 28 updating version 12 to 13 2024-09-16T14:48:38.930 INFO:tasks.rados.rados.0.smithi040.stdout:76: oid 28 updating version 13 to 14 2024-09-16T14:48:38.930 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 28 v 14 (ObjNum 57 snap 2 seq_num 57) dirty exists 2024-09-16T14:48:38.930 INFO:tasks.rados.rados.0.smithi040.stdout:76: left oid 28 (ObjNum 57 snap 2 seq_num 57) 2024-09-16T14:48:38.950 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:38.949+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:38.991 INFO:teuthology.orchestra.run.smithi111.stdout:get_pg_num_history pg_num_history pg_num_history(e23 pg_nums {1={18=1},2={20=8}} deleted_pools ) 2024-09-16T14:48:38.991 INFO:teuthology.orchestra.run.smithi111.stdout:Importing pgid 2.7 2024-09-16T14:48:38.992 INFO:teuthology.orchestra.run.smithi111.stdout:write_pg epoch 23 info 2.7( empty local-lis/les=20/21 n=0 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) 2024-09-16T14:48:38.993 INFO:teuthology.orchestra.run.smithi111.stdout:Import successful 2024-09-16T14:48:38.999 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:38.999 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:38.999 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:38.999 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:38.999 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:39.003 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:48:39.051 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:39.050+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:39.145 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:39.146 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:39.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:39.146 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:39.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.146 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 32/27 ECSubWrite(tid=21, reqid=client.4500.0:214, at_version=32'12, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:39.146 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.899824+0000", 2024-09-16T14:48:39.146 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.24514740900000001, 2024-09-16T14:48:39.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016897220000000001, 2024-09-16T14:48:39.146 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.146 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:39.147 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899824+0000", 2024-09-16T14:48:39.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899824+0000", 2024-09-16T14:48:39.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899824+0000", 2024-09-16T14:48:39.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:39.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900207+0000", 2024-09-16T14:48:39.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00038250900000000001 2024-09-16T14:48:39.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900222+0000", 2024-09-16T14:48:39.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5319e-05 2024-09-16T14:48:39.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900231+0000", 2024-09-16T14:48:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4420000000000004e-06 2024-09-16T14:48:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900291+0000", 2024-09-16T14:48:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9607e-05 2024-09-16T14:48:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900302+0000", 2024-09-16T14:48:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0607e-05 2024-09-16T14:48:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.901488+0000", 2024-09-16T14:48:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001186269 2024-09-16T14:48:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.901514+0000", 2024-09-16T14:48:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6231999999999999e-05 2024-09-16T14:48:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 32/27 ECSubWrite(tid=22, reqid=client.4500.0:215, at_version=32'13, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.901874+0000", 2024-09-16T14:48:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.24309815700000001, 2024-09-16T14:48:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0024190489999999999, 2024-09-16T14:48:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.901874+0000", 2024-09-16T14:48:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.901874+0000", 2024-09-16T14:48:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.901873+0000", 2024-09-16T14:48:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902117+0000", 2024-09-16T14:48:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024386500000000001 2024-09-16T14:48:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902132+0000", 2024-09-16T14:48:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4941e-05 2024-09-16T14:48:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902142+0000", 2024-09-16T14:48:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0604e-05 2024-09-16T14:48:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902189+0000", 2024-09-16T14:48:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6449000000000002e-05 2024-09-16T14:48:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902199+0000", 2024-09-16T14:48:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0244000000000001e-05 2024-09-16T14:48:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904270+0000", 2024-09-16T14:48:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020705680000000001 2024-09-16T14:48:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904293+0000", 2024-09-16T14:48:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3105e-05 2024-09-16T14:48:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 32/27 ECSubWrite(tid=23, reqid=client.4500.0:216, at_version=32'14, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:39.159 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.902576+0000", 2024-09-16T14:48:39.159 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.24239576299999999, 2024-09-16T14:48:39.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00174631, 2024-09-16T14:48:39.159 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.159 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:39.159 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902576+0000", 2024-09-16T14:48:39.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902576+0000", 2024-09-16T14:48:39.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902575+0000", 2024-09-16T14:48:39.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:39.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902595+0000", 2024-09-16T14:48:39.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9343000000000001e-05 2024-09-16T14:48:39.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902605+0000", 2024-09-16T14:48:39.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0562e-05 2024-09-16T14:48:39.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902613+0000", 2024-09-16T14:48:39.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3300000000000001e-06 2024-09-16T14:48:39.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902660+0000", 2024-09-16T14:48:39.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7271999999999997e-05 2024-09-16T14:48:39.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:39.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902668+0000", 2024-09-16T14:48:39.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0029999999999994e-06 2024-09-16T14:48:39.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:39.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904307+0000", 2024-09-16T14:48:39.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001638919 2024-09-16T14:48:39.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:39.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904322+0000", 2024-09-16T14:48:39.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5580999999999999e-05 2024-09-16T14:48:39.164 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.164 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.164 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.164 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 32/27 ECSubWrite(tid=24, reqid=client.4500.0:216, at_version=0'0, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:48:39.164 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.905148+0000", 2024-09-16T14:48:39.165 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.239823763, 2024-09-16T14:48:39.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000578051, 2024-09-16T14:48:39.165 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.165 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:39.165 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905148+0000", 2024-09-16T14:48:39.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905148+0000", 2024-09-16T14:48:39.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905147+0000", 2024-09-16T14:48:39.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:39.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905159+0000", 2024-09-16T14:48:39.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1471000000000001e-05 2024-09-16T14:48:39.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905173+0000", 2024-09-16T14:48:39.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4472e-05 2024-09-16T14:48:39.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905181+0000", 2024-09-16T14:48:39.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8739999999999995e-06 2024-09-16T14:48:39.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905240+0000", 2024-09-16T14:48:39.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8545000000000003e-05 2024-09-16T14:48:39.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:39.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905249+0000", 2024-09-16T14:48:39.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.054e-06 2024-09-16T14:48:39.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:39.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905706+0000", 2024-09-16T14:48:39.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00045775299999999998 2024-09-16T14:48:39.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:39.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905726+0000", 2024-09-16T14:48:39.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9721999999999999e-05 2024-09-16T14:48:39.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.169 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.169 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.169 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.169 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:39.171 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:48:39.171 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:39.170+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:39.271 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:39.270+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:39.273 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:39.273 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:39.273 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:39.273 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:39.282 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:48:39.373 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:39.371+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:39.424 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:39.425 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:39.425 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:39.425 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:39.425 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:39.433 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:48:39.476 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:39.475+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:39.508 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /var/log/ceph/exp.2.7.6 2024-09-16T14:48:39.536 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 32/27 ECSubWrite(tid=21, reqid=client.4500.0:214, at_version=32'12, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:39.536 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.899969+0000", 2024-09-16T14:48:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.63543564399999997, 2024-09-16T14:48:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040995110000000001, 2024-09-16T14:48:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899969+0000", 2024-09-16T14:48:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899969+0000", 2024-09-16T14:48:39.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899968+0000", 2024-09-16T14:48:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900282+0000", 2024-09-16T14:48:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031321000000000002 2024-09-16T14:48:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900293+0000", 2024-09-16T14:48:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1582e-05 2024-09-16T14:48:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900302+0000", 2024-09-16T14:48:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0170000000000002e-06 2024-09-16T14:48:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900359+0000", 2024-09-16T14:48:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6750999999999999e-05 2024-09-16T14:48:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900369+0000", 2024-09-16T14:48:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9299999999999998e-06 2024-09-16T14:48:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904046+0000", 2024-09-16T14:48:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036775240000000002 2024-09-16T14:48:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904068+0000", 2024-09-16T14:48:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1710999999999999e-05 2024-09-16T14:48:39.540 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 32/27 ECSubWrite(tid=22, reqid=client.4500.0:215, at_version=32'13, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.902030+0000", 2024-09-16T14:48:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.63337418300000004, 2024-09-16T14:48:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023581710000000001, 2024-09-16T14:48:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902030+0000", 2024-09-16T14:48:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902030+0000", 2024-09-16T14:48:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902029+0000", 2024-09-16T14:48:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:39.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902145+0000", 2024-09-16T14:48:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000115652 2024-09-16T14:48:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902158+0000", 2024-09-16T14:48:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3495999999999999e-05 2024-09-16T14:48:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902166+0000", 2024-09-16T14:48:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3429999999999998e-06 2024-09-16T14:48:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902205+0000", 2024-09-16T14:48:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8884000000000002e-05 2024-09-16T14:48:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902213+0000", 2024-09-16T14:48:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9729999999999997e-06 2024-09-16T14:48:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904368+0000", 2024-09-16T14:48:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0021557450000000001 2024-09-16T14:48:39.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904388+0000", 2024-09-16T14:48:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9989000000000001e-05 2024-09-16T14:48:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 32/27 ECSubWrite(tid=23, reqid=client.4500.0:216, at_version=32'14, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.902592+0000", 2024-09-16T14:48:39.546 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.63281261799999999, 2024-09-16T14:48:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00180915, 2024-09-16T14:48:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902592+0000", 2024-09-16T14:48:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902592+0000", 2024-09-16T14:48:39.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902591+0000", 2024-09-16T14:48:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902609+0000", 2024-09-16T14:48:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8482e-05 2024-09-16T14:48:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902623+0000", 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3774000000000001e-05 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902631+0000", 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9149999999999996e-06 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902686+0000", 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5383000000000002e-05 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902695+0000", 2024-09-16T14:48:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1429999999999998e-06 2024-09-16T14:48:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904391+0000", 2024-09-16T14:48:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016966290000000001 2024-09-16T14:48:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904401+0000", 2024-09-16T14:48:39.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5929999999999995e-06 2024-09-16T14:48:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 32/27 ECSubWrite(tid=24, reqid=client.4500.0:216, at_version=0'0, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:48:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.905153+0000", 2024-09-16T14:48:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.63025086299999999, 2024-09-16T14:48:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00051048700000000001, 2024-09-16T14:48:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905153+0000", 2024-09-16T14:48:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905153+0000", 2024-09-16T14:48:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905153+0000", 2024-09-16T14:48:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905161+0000", 2024-09-16T14:48:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0469999999999994e-06 2024-09-16T14:48:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905170+0000", 2024-09-16T14:48:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.217e-06 2024-09-16T14:48:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905175+0000", 2024-09-16T14:48:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4559999999999999e-06 2024-09-16T14:48:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905218+0000", 2024-09-16T14:48:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2435000000000003e-05 2024-09-16T14:48:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905227+0000", 2024-09-16T14:48:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7239999999999998e-06 2024-09-16T14:48:39.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905642+0000", 2024-09-16T14:48:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00041529000000000002 2024-09-16T14:48:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905664+0000", 2024-09-16T14:48:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2008e-05 2024-09-16T14:48:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpRead(3.4s4 32/27 ECSubRead(tid=25, to_read={3:2b1464a3:::smithi04040151-28:head=0,4096,0}, subchunks={3:2b1464a3:::smithi04040151-28:head=[0,1]}, attrs_to_read=))", 2024-09-16T14:48:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.905710+0000", 2024-09-16T14:48:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.62969479900000003, 2024-09-16T14:48:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017045100000000001, 2024-09-16T14:48:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905710+0000", 2024-09-16T14:48:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905710+0000", 2024-09-16T14:48:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905709+0000", 2024-09-16T14:48:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905718+0000", 2024-09-16T14:48:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5540000000000003e-06 2024-09-16T14:48:39.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905734+0000", 2024-09-16T14:48:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5722e-05 2024-09-16T14:48:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905744+0000", 2024-09-16T14:48:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7580000000000004e-06 2024-09-16T14:48:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905793+0000", 2024-09-16T14:48:39.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9415e-05 2024-09-16T14:48:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.905880+0000", 2024-09-16T14:48:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6737000000000007e-05 2024-09-16T14:48:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.560 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.560 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:39.562 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:48:39.575 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:39.574+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:39.588 INFO:tasks.thrashosds.thrasher:checking if target objectstore is bluestore on osd.6 2024-09-16T14:48:39.589 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph-bluestore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/bluestore_tool.$pid.log' --log-level=10 --path /var/lib/ceph/osd/ceph-6 show-label 2024-09-16T14:48:39.675 INFO:teuthology.orchestra.run.smithi111.stdout:inferring bluefs devices from bluestore path 2024-09-16T14:48:39.675 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:39.675 INFO:teuthology.orchestra.run.smithi111.stdout: "/var/lib/ceph/osd/ceph-6/block": { 2024-09-16T14:48:39.675 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_uuid": "6ac36041-045e-4813-ad35-ca4e224811fa", 2024-09-16T14:48:39.675 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 96636764160, 2024-09-16T14:48:39.676 INFO:teuthology.orchestra.run.smithi111.stdout: "btime": "2024-09-16T14:47:41.546432+0000", 2024-09-16T14:48:39.676 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "main", 2024-09-16T14:48:39.676 INFO:teuthology.orchestra.run.smithi111.stdout: "bfm_blocks": "23592960", 2024-09-16T14:48:39.676 INFO:teuthology.orchestra.run.smithi111.stdout: "bfm_blocks_per_key": "128", 2024-09-16T14:48:39.676 INFO:teuthology.orchestra.run.smithi111.stdout: "bfm_bytes_per_block": "4096", 2024-09-16T14:48:39.676 INFO:teuthology.orchestra.run.smithi111.stdout: "bfm_size": "96636764160", 2024-09-16T14:48:39.676 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs": "1", 2024-09-16T14:48:39.676 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_fsid": "aaaa1c0f-de8b-48c8-9412-815c8b70b7b8", 2024-09-16T14:48:39.676 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version_when_created": "ceph version 18.2.4-845-ge4e227b3 (e4e227b37858f5109138494e01e5ba1ba7b8ffe8) reef (stable)", 2024-09-16T14:48:39.676 INFO:teuthology.orchestra.run.smithi111.stdout: "created_at": "2024-09-16T14:47:43.883621Z", 2024-09-16T14:48:39.676 INFO:teuthology.orchestra.run.smithi111.stdout: "kv_backend": "rocksdb", 2024-09-16T14:48:39.676 INFO:teuthology.orchestra.run.smithi111.stdout: "magic": "ceph osd volume v026", 2024-09-16T14:48:39.676 INFO:teuthology.orchestra.run.smithi111.stdout: "mkfs_done": "yes", 2024-09-16T14:48:39.677 INFO:teuthology.orchestra.run.smithi111.stdout: "ready": "ready", 2024-09-16T14:48:39.677 INFO:teuthology.orchestra.run.smithi111.stdout: "require_osd_release": "18", 2024-09-16T14:48:39.677 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": "6" 2024-09-16T14:48:39.677 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:39.677 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:39.677 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:39.674+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:39.679 INFO:tasks.thrashosds.thrasher:checking if target bluestore supports sharding on osd.6 2024-09-16T14:48:39.679 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph-bluestore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/bluestore_tool.$pid.log' --log-level=10 --path /var/lib/ceph/osd/ceph-6 show-sharding 2024-09-16T14:48:39.705 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:39.705 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:39.705 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:39.705 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:39.713 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:48:39.776 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:39.775+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:39.815 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:39.815 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:39.815 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:39.816 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:39.816 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:39.824 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:48:39.900 INFO:tasks.rados.rados.0.smithi040.stdout:74: done (2 left) 2024-09-16T14:48:39.900 INFO:tasks.rados.rados.0.smithi040.stdout:76: done (1 left) 2024-09-16T14:48:39.900 INFO:tasks.rados.rados.0.smithi040.stdout:77: done (0 left) 2024-09-16T14:48:39.900 INFO:tasks.rados.rados.0.smithi040.stdout:78: read oid 31 snap -1 2024-09-16T14:48:39.900 INFO:tasks.rados.rados.0.smithi040.stdout:78: expect (ObjNum 55 snap 1 seq_num 55) 2024-09-16T14:48:39.901 INFO:tasks.rados.rados.0.smithi040.stdout:79: delete oid 25 current snap is 3 2024-09-16T14:48:39.905 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:48:39.906 INFO:tasks.rados.rados.0.smithi040.stdout:79: done (1 left) 2024-09-16T14:48:39.906 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 13 current snap is 3 2024-09-16T14:48:39.906 INFO:tasks.rados.rados.0.smithi040.stdout:80: seq_num 59 ranges {212992=622592,835584=376832} 2024-09-16T14:48:39.909 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:39.908+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:39.911 INFO:tasks.rados.rados.0.smithi040.stdout:80: writing smithi04040151-13 from 212992 to 835584 tid 1 2024-09-16T14:48:39.914 INFO:tasks.rados.rados.0.smithi040.stdout:80: writing smithi04040151-13 from 835584 to 1212416 tid 2 2024-09-16T14:48:39.914 INFO:tasks.rados.rados.0.smithi040.stdout:81: delete oid 11 current snap is 3 2024-09-16T14:48:39.914 INFO:tasks.rados.rados.0.smithi040.stdout:78: expect (ObjNum 55 snap 1 seq_num 55) 2024-09-16T14:48:39.951 INFO:tasks.rados.rados.0.smithi040.stdout:80: finishing write tid 1 to smithi04040151-13 2024-09-16T14:48:39.951 INFO:tasks.rados.rados.0.smithi040.stdout:80: finishing write tid 2 to smithi04040151-13 2024-09-16T14:48:39.951 INFO:tasks.rados.rados.0.smithi040.stdout:80: finishing write tid 3 to smithi04040151-13 2024-09-16T14:48:39.951 INFO:tasks.rados.rados.0.smithi040.stdout:80: finishing write tid 4 to smithi04040151-13 2024-09-16T14:48:39.951 INFO:tasks.rados.rados.0.smithi040.stdout:80: oid 13 updating version 0 to 13 2024-09-16T14:48:39.951 INFO:tasks.rados.rados.0.smithi040.stdout:80: oid 13 version 13 is already newer than 12 2024-09-16T14:48:39.951 INFO:tasks.rados.rados.0.smithi040.stdout:80: oid 13 updating version 13 to 14 2024-09-16T14:48:39.951 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 13 v 14 (ObjNum 59 snap 3 seq_num 59) dirty exists 2024-09-16T14:48:39.951 INFO:tasks.rados.rados.0.smithi040.stdout:80: left oid 13 (ObjNum 59 snap 3 seq_num 59) 2024-09-16T14:48:39.951 INFO:tasks.rados.rados.0.smithi040.stdout:78: done (2 left) 2024-09-16T14:48:39.952 INFO:tasks.rados.rados.0.smithi040.stdout:80: done (1 left) 2024-09-16T14:48:39.952 INFO:tasks.rados.rados.0.smithi040.stdout:81: done (0 left) 2024-09-16T14:48:39.952 INFO:tasks.rados.rados.0.smithi040.stdout:82: snap_remove snap 1 2024-09-16T14:48:39.973 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:39.973 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:39.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:39.973 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:39.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.973 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:4 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'2, mlcod=0'0)", 2024-09-16T14:48:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.364271+0000", 2024-09-16T14:48:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.606598695999999, 2024-09-16T14:48:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036960159999999999, 2024-09-16T14:48:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364271+0000", 2024-09-16T14:48:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.974 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364271+0000", 2024-09-16T14:48:39.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364270+0000", 2024-09-16T14:48:39.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:39.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364277+0000", 2024-09-16T14:48:39.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7519999999999999e-06 2024-09-16T14:48:39.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364290+0000", 2024-09-16T14:48:39.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3159999999999999e-05 2024-09-16T14:48:39.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364297+0000", 2024-09-16T14:48:39.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.776e-06 2024-09-16T14:48:39.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364342+0000", 2024-09-16T14:48:39.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4601999999999997e-05 2024-09-16T14:48:39.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:39.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.364373+0000", 2024-09-16T14:48:39.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1878000000000002e-05 2024-09-16T14:48:39.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:39.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.367937+0000", 2024-09-16T14:48:39.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003563325 2024-09-16T14:48:39.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:39.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.367967+0000", 2024-09-16T14:48:39.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.046e-05 2024-09-16T14:48:39.978 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.978 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.978 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.978 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:18 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'8, mlcod=19'5)", 2024-09-16T14:48:39.978 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.380607+0000", 2024-09-16T14:48:39.978 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.590263352000001, 2024-09-16T14:48:39.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042947610000000002, 2024-09-16T14:48:39.978 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.978 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:39.978 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380607+0000", 2024-09-16T14:48:39.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380607+0000", 2024-09-16T14:48:39.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380606+0000", 2024-09-16T14:48:39.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:39.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380611+0000", 2024-09-16T14:48:39.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3210000000000004e-06 2024-09-16T14:48:39.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380620+0000", 2024-09-16T14:48:39.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5709999999999998e-06 2024-09-16T14:48:39.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380625+0000", 2024-09-16T14:48:39.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1580000000000004e-06 2024-09-16T14:48:39.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380659+0000", 2024-09-16T14:48:39.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3587000000000002e-05 2024-09-16T14:48:39.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:39.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.380679+0000", 2024-09-16T14:48:39.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0242999999999999e-05 2024-09-16T14:48:39.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:39.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.384855+0000", 2024-09-16T14:48:39.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041758089999999999 2024-09-16T14:48:39.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:39.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.384901+0000", 2024-09-16T14:48:39.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6634000000000001e-05 2024-09-16T14:48:39.982 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.982 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.982 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.982 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2024-09-16T14:48:39.983 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.389034+0000", 2024-09-16T14:48:39.983 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.581835772000002, 2024-09-16T14:48:39.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066701460000000001, 2024-09-16T14:48:39.983 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.983 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:39.983 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389034+0000", 2024-09-16T14:48:39.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389034+0000", 2024-09-16T14:48:39.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389033+0000", 2024-09-16T14:48:39.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:39.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389110+0000", 2024-09-16T14:48:39.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6965000000000006e-05 2024-09-16T14:48:39.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389122+0000", 2024-09-16T14:48:39.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.166e-05 2024-09-16T14:48:39.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389129+0000", 2024-09-16T14:48:39.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8120000000000002e-06 2024-09-16T14:48:39.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389166+0000", 2024-09-16T14:48:39.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6986999999999997e-05 2024-09-16T14:48:39.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:39.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389186+0000", 2024-09-16T14:48:39.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0043000000000001e-05 2024-09-16T14:48:39.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:39.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395613+0000", 2024-09-16T14:48:39.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064270109999999998 2024-09-16T14:48:39.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:39.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395704+0000", 2024-09-16T14:48:39.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1595000000000006e-05 2024-09-16T14:48:39.987 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.987 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.987 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.987 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:22 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'12, mlcod=19'8)", 2024-09-16T14:48:39.987 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.389513+0000", 2024-09-16T14:48:39.987 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.581357381, 2024-09-16T14:48:39.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062632759999999999, 2024-09-16T14:48:39.987 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.987 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:39.987 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389513+0000", 2024-09-16T14:48:39.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389513+0000", 2024-09-16T14:48:39.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389512+0000", 2024-09-16T14:48:39.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:39.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389520+0000", 2024-09-16T14:48:39.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2609999999999993e-06 2024-09-16T14:48:39.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389533+0000", 2024-09-16T14:48:39.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2860999999999999e-05 2024-09-16T14:48:39.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389539+0000", 2024-09-16T14:48:39.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2369999999999997e-06 2024-09-16T14:48:39.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389568+0000", 2024-09-16T14:48:39.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9411999999999999e-05 2024-09-16T14:48:39.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:39.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389584+0000", 2024-09-16T14:48:39.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5891000000000001e-05 2024-09-16T14:48:39.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:39.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395712+0000", 2024-09-16T14:48:39.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061272360000000003 2024-09-16T14:48:39.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:39.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395776+0000", 2024-09-16T14:48:39.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4290999999999998e-05 2024-09-16T14:48:39.991 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.991 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.991 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.991 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:23 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'13, mlcod=19'11)", 2024-09-16T14:48:39.991 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.396595+0000", 2024-09-16T14:48:39.991 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.574274500000001, 2024-09-16T14:48:39.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003499107, 2024-09-16T14:48:39.992 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.992 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:39.992 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396595+0000", 2024-09-16T14:48:39.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396595+0000", 2024-09-16T14:48:39.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396595+0000", 2024-09-16T14:48:39.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:39.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396601+0000", 2024-09-16T14:48:39.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4899999999999997e-06 2024-09-16T14:48:39.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396613+0000", 2024-09-16T14:48:39.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1905e-05 2024-09-16T14:48:39.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396618+0000", 2024-09-16T14:48:39.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1039999999999998e-06 2024-09-16T14:48:39.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396656+0000", 2024-09-16T14:48:39.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8408000000000002e-05 2024-09-16T14:48:39.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:39.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.396669+0000", 2024-09-16T14:48:39.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2362e-05 2024-09-16T14:48:39.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:39.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.400046+0000", 2024-09-16T14:48:39.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0033771500000000002 2024-09-16T14:48:39.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:39.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.400095+0000", 2024-09-16T14:48:39.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8581999999999997e-05 2024-09-16T14:48:39.995 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.996 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:39.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:39.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.996 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:32 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'18, mlcod=19'14)", 2024-09-16T14:48:39.996 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.407675+0000", 2024-09-16T14:48:39.996 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.563194541000001, 2024-09-16T14:48:39.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0056403440000000003, 2024-09-16T14:48:39.996 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:39.996 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:39.996 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:39.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:39.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407675+0000", 2024-09-16T14:48:39.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:39.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407675+0000", 2024-09-16T14:48:39.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:39.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:39.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407675+0000", 2024-09-16T14:48:39.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:39.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:39.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407750+0000", 2024-09-16T14:48:39.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4728999999999996e-05 2024-09-16T14:48:39.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:39.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407759+0000", 2024-09-16T14:48:39.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3519999999999996e-06 2024-09-16T14:48:39.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:39.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407764+0000", 2024-09-16T14:48:39.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.994e-06 2024-09-16T14:48:39.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407797+0000", 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2641999999999997e-05 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407816+0000", 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9219e-05 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413272+0000", 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054566129999999999 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:39.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413316+0000", 2024-09-16T14:48:40.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3350999999999999e-05 2024-09-16T14:48:40.000 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.000 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.000 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.000 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2024-09-16T14:48:40.000 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.408061+0000", 2024-09-16T14:48:40.000 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.562808482000001, 2024-09-16T14:48:40.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052755520000000002, 2024-09-16T14:48:40.000 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:40.000 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:40.001 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:40.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:40.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408061+0000", 2024-09-16T14:48:40.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:40.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408061+0000", 2024-09-16T14:48:40.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:40.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408061+0000", 2024-09-16T14:48:40.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:40.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:40.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408066+0000", 2024-09-16T14:48:40.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6210000000000001e-06 2024-09-16T14:48:40.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:40.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408077+0000", 2024-09-16T14:48:40.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0906e-05 2024-09-16T14:48:40.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408081+0000", 2024-09-16T14:48:40.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8689999999999997e-06 2024-09-16T14:48:40.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408114+0000", 2024-09-16T14:48:40.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2795e-05 2024-09-16T14:48:40.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:40.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408133+0000", 2024-09-16T14:48:40.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9017000000000001e-05 2024-09-16T14:48:40.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413318+0000", 2024-09-16T14:48:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051848040000000003 2024-09-16T14:48:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413337+0000", 2024-09-16T14:48:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9173000000000001e-05 2024-09-16T14:48:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:34 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'20, mlcod=19'15)", 2024-09-16T14:48:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.408607+0000", 2024-09-16T14:48:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.562262903000001, 2024-09-16T14:48:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047534309999999998, 2024-09-16T14:48:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408607+0000", 2024-09-16T14:48:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408607+0000", 2024-09-16T14:48:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408606+0000", 2024-09-16T14:48:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408613+0000", 2024-09-16T14:48:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7510000000000002e-06 2024-09-16T14:48:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408626+0000", 2024-09-16T14:48:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2819e-05 2024-09-16T14:48:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408631+0000", 2024-09-16T14:48:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1599999999999997e-06 2024-09-16T14:48:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408673+0000", 2024-09-16T14:48:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2274999999999998e-05 2024-09-16T14:48:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408692+0000", 2024-09-16T14:48:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9219e-05 2024-09-16T14:48:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413340+0000", 2024-09-16T14:48:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046472359999999999 2024-09-16T14:48:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413360+0000", 2024-09-16T14:48:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0882000000000001e-05 2024-09-16T14:48:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:35 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'21, mlcod=19'15)", 2024-09-16T14:48:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.409042+0000", 2024-09-16T14:48:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.561827514000001, 2024-09-16T14:48:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.004339893, 2024-09-16T14:48:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409042+0000", 2024-09-16T14:48:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409042+0000", 2024-09-16T14:48:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409042+0000", 2024-09-16T14:48:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409112+0000", 2024-09-16T14:48:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0382000000000005e-05 2024-09-16T14:48:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409121+0000", 2024-09-16T14:48:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9560000000000003e-06 2024-09-16T14:48:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409126+0000", 2024-09-16T14:48:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3460000000000001e-06 2024-09-16T14:48:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409159+0000", 2024-09-16T14:48:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2351999999999998e-05 2024-09-16T14:48:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409178+0000", 2024-09-16T14:48:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9510000000000001e-05 2024-09-16T14:48:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413362+0000", 2024-09-16T14:48:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041835220000000003 2024-09-16T14:48:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413382+0000", 2024-09-16T14:48:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0463999999999999e-05 2024-09-16T14:48:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:36 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'22, mlcod=19'15)", 2024-09-16T14:48:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.409634+0000", 2024-09-16T14:48:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.561236084000001, 2024-09-16T14:48:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0037732019999999998, 2024-09-16T14:48:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409634+0000", 2024-09-16T14:48:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409634+0000", 2024-09-16T14:48:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409633+0000", 2024-09-16T14:48:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409641+0000", 2024-09-16T14:48:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8220000000000007e-06 2024-09-16T14:48:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409654+0000", 2024-09-16T14:48:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3427999999999999e-05 2024-09-16T14:48:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409669+0000", 2024-09-16T14:48:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4572999999999999e-05 2024-09-16T14:48:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409709+0000", 2024-09-16T14:48:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0392000000000002e-05 2024-09-16T14:48:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409728+0000", 2024-09-16T14:48:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8904000000000001e-05 2024-09-16T14:48:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413385+0000", 2024-09-16T14:48:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036566789999999999 2024-09-16T14:48:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413407+0000", 2024-09-16T14:48:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2357999999999998e-05 2024-09-16T14:48:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:37 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'23, mlcod=19'15)", 2024-09-16T14:48:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.409927+0000", 2024-09-16T14:48:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.560943203000001, 2024-09-16T14:48:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003502027, 2024-09-16T14:48:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409927+0000", 2024-09-16T14:48:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409927+0000", 2024-09-16T14:48:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409926+0000", 2024-09-16T14:48:40.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:40.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:40.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409939+0000", 2024-09-16T14:48:40.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3161e-05 2024-09-16T14:48:40.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:40.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409948+0000", 2024-09-16T14:48:40.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3499999999999997e-06 2024-09-16T14:48:40.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.409952+0000", 2024-09-16T14:48:40.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9820000000000002e-06 2024-09-16T14:48:40.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.410009+0000", 2024-09-16T14:48:40.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7037000000000003e-05 2024-09-16T14:48:40.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:40.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.410023+0000", 2024-09-16T14:48:40.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4718e-05 2024-09-16T14:48:40.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:40.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413408+0000", 2024-09-16T14:48:40.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0033848889999999999 2024-09-16T14:48:40.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413429+0000", 2024-09-16T14:48:40.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0472999999999999e-05 2024-09-16T14:48:40.022 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.022 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.022 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.023 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:158 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'112, mlcod=20'107)", 2024-09-16T14:48:40.023 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:23.394419+0000", 2024-09-16T14:48:40.023 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 16.576450686000001, 2024-09-16T14:48:40.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054479120000000001, 2024-09-16T14:48:40.023 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:40.023 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:40.023 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:40.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:40.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394419+0000", 2024-09-16T14:48:40.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:40.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394419+0000", 2024-09-16T14:48:40.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:40.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394419+0000", 2024-09-16T14:48:40.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:40.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:40.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394481+0000", 2024-09-16T14:48:40.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2515999999999996e-05 2024-09-16T14:48:40.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:40.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394489+0000", 2024-09-16T14:48:40.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:48:40.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394495+0000", 2024-09-16T14:48:40.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3070000000000002e-06 2024-09-16T14:48:40.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394518+0000", 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3818000000000001e-05 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394530+0000", 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1158999999999999e-05 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399835+0000", 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053054829999999997 2024-09-16T14:48:40.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399867+0000", 2024-09-16T14:48:40.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2141000000000003e-05 2024-09-16T14:48:40.027 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.027 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.027 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.027 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:159 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'113, mlcod=20'107)", 2024-09-16T14:48:40.027 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:23.394933+0000", 2024-09-16T14:48:40.027 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 16.575936465000002, 2024-09-16T14:48:40.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049514190000000003, 2024-09-16T14:48:40.027 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:40.028 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:40.028 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:40.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:40.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394933+0000", 2024-09-16T14:48:40.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:40.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394933+0000", 2024-09-16T14:48:40.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:40.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394933+0000", 2024-09-16T14:48:40.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:40.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:40.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394938+0000", 2024-09-16T14:48:40.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5050000000000001e-06 2024-09-16T14:48:40.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:40.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394944+0000", 2024-09-16T14:48:40.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6699999999999997e-06 2024-09-16T14:48:40.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394949+0000", 2024-09-16T14:48:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7439999999999998e-06 2024-09-16T14:48:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394976+0000", 2024-09-16T14:48:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6682e-05 2024-09-16T14:48:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394997+0000", 2024-09-16T14:48:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1291e-05 2024-09-16T14:48:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:40.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399870+0000", 2024-09-16T14:48:40.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00487339 2024-09-16T14:48:40.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399885+0000", 2024-09-16T14:48:40.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4619e-05 2024-09-16T14:48:40.031 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.031 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.031 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.032 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:160 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'114, mlcod=20'109)", 2024-09-16T14:48:40.032 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:23.395425+0000", 2024-09-16T14:48:40.032 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 16.575445053999999, 2024-09-16T14:48:40.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044722859999999998, 2024-09-16T14:48:40.032 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:40.032 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:40.032 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:40.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:40.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395425+0000", 2024-09-16T14:48:40.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:40.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395425+0000", 2024-09-16T14:48:40.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:40.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395424+0000", 2024-09-16T14:48:40.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:40.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:40.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395429+0000", 2024-09-16T14:48:40.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5519999999999998e-06 2024-09-16T14:48:40.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:40.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395436+0000", 2024-09-16T14:48:40.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7859999999999999e-06 2024-09-16T14:48:40.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395441+0000", 2024-09-16T14:48:40.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9899999999999997e-06 2024-09-16T14:48:40.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395466+0000", 2024-09-16T14:48:40.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4739000000000001e-05 2024-09-16T14:48:40.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395478+0000", 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2604e-05 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399886+0000", 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044076769999999996 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399897+0000", 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1396999999999999e-05 2024-09-16T14:48:40.035 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.036 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.036 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.036 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:161 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'115, mlcod=20'109)", 2024-09-16T14:48:40.036 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:23.395800+0000", 2024-09-16T14:48:40.036 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 16.575069931000002, 2024-09-16T14:48:40.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041068930000000003, 2024-09-16T14:48:40.036 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:40.036 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:40.036 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:40.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:40.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395800+0000", 2024-09-16T14:48:40.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:40.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395800+0000", 2024-09-16T14:48:40.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:40.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395800+0000", 2024-09-16T14:48:40.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:40.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:40.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395833+0000", 2024-09-16T14:48:40.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3416e-05 2024-09-16T14:48:40.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:40.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395841+0000", 2024-09-16T14:48:40.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5039999999999996e-06 2024-09-16T14:48:40.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395845+0000", 2024-09-16T14:48:40.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9599999999999999e-06 2024-09-16T14:48:40.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395868+0000", 2024-09-16T14:48:40.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2484e-05 2024-09-16T14:48:40.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:40.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.395879+0000", 2024-09-16T14:48:40.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1222e-05 2024-09-16T14:48:40.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:40.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399898+0000", 2024-09-16T14:48:40.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.004018817 2024-09-16T14:48:40.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399907+0000", 2024-09-16T14:48:40.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9199999999999993e-06 2024-09-16T14:48:40.040 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.040 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.040 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.040 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:162 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'116, mlcod=20'109)", 2024-09-16T14:48:40.041 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:23.396386+0000", 2024-09-16T14:48:40.041 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 16.574484206000001, 2024-09-16T14:48:40.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00367265, 2024-09-16T14:48:40.041 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:40.041 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:40.041 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:40.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:40.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396386+0000", 2024-09-16T14:48:40.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:40.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396386+0000", 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396385+0000", 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396390+0000", 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2140000000000004e-06 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:40.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396398+0000", 2024-09-16T14:48:40.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0570000000000004e-06 2024-09-16T14:48:40.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396402+0000", 2024-09-16T14:48:40.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7709999999999997e-06 2024-09-16T14:48:40.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396802+0000", 2024-09-16T14:48:40.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000399579 2024-09-16T14:48:40.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.396825+0000", 2024-09-16T14:48:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2840000000000002e-05 2024-09-16T14:48:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.400026+0000", 2024-09-16T14:48:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032011639999999998 2024-09-16T14:48:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.400058+0000", 2024-09-16T14:48:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2534999999999999e-05 2024-09-16T14:48:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.044 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.045 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.045 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:165 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'119, mlcod=20'109)", 2024-09-16T14:48:40.045 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:23.397690+0000", 2024-09-16T14:48:40.045 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 16.573179511999999, 2024-09-16T14:48:40.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.004996749, 2024-09-16T14:48:40.045 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:40.045 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:40.045 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:40.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:40.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.397690+0000", 2024-09-16T14:48:40.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:40.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.397690+0000", 2024-09-16T14:48:40.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:40.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.397690+0000", 2024-09-16T14:48:40.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:40.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:40.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.397695+0000", 2024-09-16T14:48:40.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7840000000000003e-06 2024-09-16T14:48:40.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:40.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.397701+0000", 2024-09-16T14:48:40.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6810000000000001e-06 2024-09-16T14:48:40.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.397706+0000", 2024-09-16T14:48:40.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.634e-06 2024-09-16T14:48:40.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399587+0000", 2024-09-16T14:48:40.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0018804920000000001 2024-09-16T14:48:40.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:40.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399605+0000", 2024-09-16T14:48:40.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8156e-05 2024-09-16T14:48:40.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:40.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.402664+0000", 2024-09-16T14:48:40.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030596909999999998 2024-09-16T14:48:40.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.402687+0000", 2024-09-16T14:48:40.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2796e-05 2024-09-16T14:48:40.049 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.049 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.049 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.049 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:211 3.8s0 3:145ad69c:::smithi04040151-6:head [read 0~983040,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T14:48:40.049 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.891138+0000", 2024-09-16T14:48:40.049 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.0797319809999999, 2024-09-16T14:48:40.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046052699999999998, 2024-09-16T14:48:40.049 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:40.049 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:40.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:40.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:40.050 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:40.050 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 211 2024-09-16T14:48:40.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.050 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:40.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:40.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891138+0000", 2024-09-16T14:48:40.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:40.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891138+0000", 2024-09-16T14:48:40.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:40.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891136+0000", 2024-09-16T14:48:40.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:48:40.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:40.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891147+0000", 2024-09-16T14:48:40.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0811e-05 2024-09-16T14:48:40.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:40.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891163+0000", 2024-09-16T14:48:40.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6501999999999999e-05 2024-09-16T14:48:40.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891172+0000", 2024-09-16T14:48:40.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6370000000000005e-06 2024-09-16T14:48:40.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891237+0000", 2024-09-16T14:48:40.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4758999999999994e-05 2024-09-16T14:48:40.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:48:40.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891245+0000", 2024-09-16T14:48:40.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2099999999999993e-06 2024-09-16T14:48:40.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.892786+0000", 2024-09-16T14:48:40.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0015411699999999999 2024-09-16T14:48:40.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:40.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.892854+0000", 2024-09-16T14:48:40.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8100000000000002e-05 2024-09-16T14:48:40.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.895743+0000", 2024-09-16T14:48:40.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028890420000000001 2024-09-16T14:48:40.054 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:212 3.8s0 3:145ad69c:::smithi04040151-6:head [read 0~983040,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.891190+0000", 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.0796802599999999, 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049458080000000003, 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 212 2024-09-16T14:48:40.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.056 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:40.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:40.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891190+0000", 2024-09-16T14:48:40.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:40.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891190+0000", 2024-09-16T14:48:40.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:40.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891189+0000", 2024-09-16T14:48:40.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:40.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:40.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891203+0000", 2024-09-16T14:48:40.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3664e-05 2024-09-16T14:48:40.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:40.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891211+0000", 2024-09-16T14:48:40.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6869999999999994e-06 2024-09-16T14:48:40.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891214+0000", 2024-09-16T14:48:40.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3759999999999999e-06 2024-09-16T14:48:40.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891304+0000", 2024-09-16T14:48:40.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9563999999999998e-05 2024-09-16T14:48:40.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:48:40.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891308+0000", 2024-09-16T14:48:40.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8910000000000003e-06 2024-09-16T14:48:40.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.893081+0000", 2024-09-16T14:48:40.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0017730140000000001 2024-09-16T14:48:40.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:40.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.893126+0000", 2024-09-16T14:48:40.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4206000000000003e-05 2024-09-16T14:48:40.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.896135+0000", 2024-09-16T14:48:40.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030098289999999999 2024-09-16T14:48:40.059 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.060 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.060 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.060 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:213 3.8s0 3:145ad69c:::smithi04040151-6:head [read 0~983040,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T14:48:40.060 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.891233+0000", 2024-09-16T14:48:40.060 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.079636584, 2024-09-16T14:48:40.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053188610000000002, 2024-09-16T14:48:40.060 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:40.060 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:40.060 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:40.060 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:40.060 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:40.061 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 213 2024-09-16T14:48:40.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.061 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:40.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:40.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891233+0000", 2024-09-16T14:48:40.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:40.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891233+0000", 2024-09-16T14:48:40.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:40.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891233+0000", 2024-09-16T14:48:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891235+0000", 2024-09-16T14:48:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2589999999999999e-06 2024-09-16T14:48:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891239+0000", 2024-09-16T14:48:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8340000000000001e-06 2024-09-16T14:48:40.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891242+0000", 2024-09-16T14:48:40.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.841e-06 2024-09-16T14:48:40.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891330+0000", 2024-09-16T14:48:40.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8253999999999999e-05 2024-09-16T14:48:40.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:48:40.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891338+0000", 2024-09-16T14:48:40.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.233e-06 2024-09-16T14:48:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.893273+0000", 2024-09-16T14:48:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001935223 2024-09-16T14:48:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.893313+0000", 2024-09-16T14:48:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9824999999999999e-05 2024-09-16T14:48:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:40.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.896552+0000", 2024-09-16T14:48:40.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032396730000000002 2024-09-16T14:48:40.065 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.065 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.065 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.065 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:40.065 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:40.065 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:40.066 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:40.065+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:40.067 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:48:40.167 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:40.166+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:40.174 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:48:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:48:40.174 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:48:40.174 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:48:40.183 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:48:40.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:40.267+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:40.290 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:48:40.290 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:48:40.290 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:48:40.290 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:48:40.290 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:48:40.298 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:48:40.383 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:40.382+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:40.446 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:48:40.446 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:48:40.446 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:48:40.446 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:48:40.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.447 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 32/27 ECSubWrite(tid=21, reqid=client.4500.0:214, at_version=32'12, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:40.447 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:38.899540+0000", 2024-09-16T14:48:40.447 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.545660078, 2024-09-16T14:48:40.447 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0020185989999999998, 2024-09-16T14:48:40.447 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:40.447 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:40.447 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:40.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.447 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:40.447 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.899540+0000", 2024-09-16T14:48:40.448 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:40.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.448 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:40.448 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.899540+0000", 2024-09-16T14:48:40.448 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:40.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.449 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:40.449 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.899540+0000", 2024-09-16T14:48:40.449 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:40.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.449 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.449 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:40.449 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.899691+0000", 2024-09-16T14:48:40.449 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00015089500000000001 2024-09-16T14:48:40.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.450 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:40.450 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.899716+0000", 2024-09-16T14:48:40.450 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5584000000000001e-05 2024-09-16T14:48:40.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.450 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.450 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.899727+0000", 2024-09-16T14:48:40.450 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1154e-05 2024-09-16T14:48:40.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.451 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:40.451 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.899783+0000", 2024-09-16T14:48:40.451 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5747e-05 2024-09-16T14:48:40.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.451 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:40.451 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.899794+0000", 2024-09-16T14:48:40.451 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0594000000000001e-05 2024-09-16T14:48:40.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.452 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:40.452 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901528+0000", 2024-09-16T14:48:40.452 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001734024 2024-09-16T14:48:40.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.453 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:40.453 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901559+0000", 2024-09-16T14:48:40.453 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0854e-05 2024-09-16T14:48:40.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:40.454 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:40.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:40.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.455 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 32/27 ECSubWrite(tid=22, reqid=client.4500.0:215, at_version=32'13, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:40.455 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:38.901735+0000", 2024-09-16T14:48:40.455 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.5434651109999999, 2024-09-16T14:48:40.455 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0023292809999999999, 2024-09-16T14:48:40.455 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:40.456 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:40.456 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:40.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.456 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:40.456 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901735+0000", 2024-09-16T14:48:40.456 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:40.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.456 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:40.456 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901735+0000", 2024-09-16T14:48:40.457 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:40.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.457 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:40.457 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901734+0000", 2024-09-16T14:48:40.457 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:40.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.457 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:40.457 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901843+0000", 2024-09-16T14:48:40.457 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000109413 2024-09-16T14:48:40.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.458 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:40.458 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901862+0000", 2024-09-16T14:48:40.458 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8964999999999999e-05 2024-09-16T14:48:40.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.458 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.458 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901871+0000", 2024-09-16T14:48:40.458 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7309999999999999e-06 2024-09-16T14:48:40.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.459 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:40.459 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901930+0000", 2024-09-16T14:48:40.459 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8343e-05 2024-09-16T14:48:40.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.459 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:40.459 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901939+0000", 2024-09-16T14:48:40.459 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6050000000000004e-06 2024-09-16T14:48:40.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.460 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:40.460 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.904040+0000", 2024-09-16T14:48:40.460 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0021013569999999999 2024-09-16T14:48:40.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.460 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:40.460 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.904064+0000", 2024-09-16T14:48:40.460 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3842e-05 2024-09-16T14:48:40.460 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:40.460 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:40.460 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:40.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.461 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 32/27 ECSubWrite(tid=23, reqid=client.4500.0:216, at_version=32'14, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:40.461 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:38.902514+0000", 2024-09-16T14:48:40.461 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.5426863210000001, 2024-09-16T14:48:40.461 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0015645240000000001, 2024-09-16T14:48:40.461 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:40.461 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:40.461 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:40.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.461 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:40.462 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.902514+0000", 2024-09-16T14:48:40.462 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:40.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.462 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:40.462 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.902514+0000", 2024-09-16T14:48:40.462 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:40.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.462 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:40.462 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.902513+0000", 2024-09-16T14:48:40.463 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:40.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.463 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:40.463 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.902532+0000", 2024-09-16T14:48:40.463 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8913999999999999e-05 2024-09-16T14:48:40.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.463 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:40.463 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.902547+0000", 2024-09-16T14:48:40.463 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5245000000000001e-05 2024-09-16T14:48:40.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.464 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.464 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.464 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.902556+0000", 2024-09-16T14:48:40.464 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7560000000000005e-06 2024-09-16T14:48:40.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.464 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.464 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:40.464 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.902634+0000", 2024-09-16T14:48:40.464 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7563000000000003e-05 2024-09-16T14:48:40.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.465 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.465 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:40.465 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.902643+0000", 2024-09-16T14:48:40.465 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0850000000000003e-06 2024-09-16T14:48:40.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.465 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.465 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:40.465 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.904067+0000", 2024-09-16T14:48:40.465 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0014246389999999999 2024-09-16T14:48:40.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.466 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:40.466 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.904078+0000", 2024-09-16T14:48:40.466 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1056000000000001e-05 2024-09-16T14:48:40.466 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:40.466 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:40.466 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:40.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.466 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 32/27 ECSubWrite(tid=24, reqid=client.4500.0:216, at_version=0'0, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:48:40.467 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:38.905121+0000", 2024-09-16T14:48:40.467 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.5400794119999999, 2024-09-16T14:48:40.467 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00070334900000000003, 2024-09-16T14:48:40.467 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:40.467 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:40.467 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:40.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.467 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:40.467 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.905121+0000", 2024-09-16T14:48:40.467 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:40.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.468 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:40.468 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.905121+0000", 2024-09-16T14:48:40.468 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:40.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.468 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:40.468 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.905120+0000", 2024-09-16T14:48:40.468 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:40.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.469 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:40.469 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.905131+0000", 2024-09-16T14:48:40.469 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1314999999999999e-05 2024-09-16T14:48:40.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.469 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.469 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:40.469 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.905147+0000", 2024-09-16T14:48:40.469 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5217e-05 2024-09-16T14:48:40.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.469 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.469 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.469 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.905157+0000", 2024-09-16T14:48:40.470 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0291e-05 2024-09-16T14:48:40.470 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.470 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.470 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:40.470 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.905206+0000", 2024-09-16T14:48:40.470 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9555000000000002e-05 2024-09-16T14:48:40.470 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.470 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.470 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:40.470 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.905215+0000", 2024-09-16T14:48:40.470 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0359999999999995e-06 2024-09-16T14:48:40.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.471 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.471 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:40.471 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.905783+0000", 2024-09-16T14:48:40.471 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000567261 2024-09-16T14:48:40.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.471 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.471 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:40.471 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.905824+0000", 2024-09-16T14:48:40.471 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.1424999999999997e-05 2024-09-16T14:48:40.472 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:40.472 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:40.472 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:40.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.472 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs6 33/27 ECSubWrite(tid=166, reqid=client.4500.0:222, at_version=33'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:48:40.472 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:39.902320+0000", 2024-09-16T14:48:40.472 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.54287987500000001, 2024-09-16T14:48:40.472 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00190818, 2024-09-16T14:48:40.472 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:40.472 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:40.473 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:40.473 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.473 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:40.473 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.902320+0000", 2024-09-16T14:48:40.473 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:40.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.473 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.473 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:40.473 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.902320+0000", 2024-09-16T14:48:40.473 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:40.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.474 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.474 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:40.474 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.902320+0000", 2024-09-16T14:48:40.474 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:40.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.474 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.474 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:40.474 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.902408+0000", 2024-09-16T14:48:40.474 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8483e-05 2024-09-16T14:48:40.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.475 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.475 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:40.475 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.902426+0000", 2024-09-16T14:48:40.475 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7561e-05 2024-09-16T14:48:40.475 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.475 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.476 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.476 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.902437+0000", 2024-09-16T14:48:40.476 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1258e-05 2024-09-16T14:48:40.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.476 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.476 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:40.476 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.902506+0000", 2024-09-16T14:48:40.476 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8491999999999994e-05 2024-09-16T14:48:40.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.476 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.476 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:40.477 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.902518+0000", 2024-09-16T14:48:40.477 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2713e-05 2024-09-16T14:48:40.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.477 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:40.477 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.904190+0000", 2024-09-16T14:48:40.477 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001671645 2024-09-16T14:48:40.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.477 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:40.477 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.904228+0000", 2024-09-16T14:48:40.478 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.8399999999999998e-05 2024-09-16T14:48:40.478 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:40.478 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:40.478 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:40.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.478 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs6 33/27 ECSubWrite(tid=167, reqid=client.4500.0:222, at_version=0'0, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:48:40.478 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:39.905227+0000", 2024-09-16T14:48:40.478 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.53997293700000004, 2024-09-16T14:48:40.479 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00081889300000000001, 2024-09-16T14:48:40.479 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:40.479 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:40.479 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:40.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.479 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:40.479 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.905227+0000", 2024-09-16T14:48:40.479 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:40.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.479 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:40.479 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.905227+0000", 2024-09-16T14:48:40.480 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:40.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.480 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:40.480 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.905226+0000", 2024-09-16T14:48:40.480 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:40.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.480 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:40.480 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.905241+0000", 2024-09-16T14:48:40.480 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4808e-05 2024-09-16T14:48:40.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.481 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:40.481 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.905265+0000", 2024-09-16T14:48:40.481 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4258999999999999e-05 2024-09-16T14:48:40.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.481 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:40.481 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.905276+0000", 2024-09-16T14:48:40.481 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0849999999999999e-05 2024-09-16T14:48:40.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.482 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:40.482 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.905329+0000", 2024-09-16T14:48:40.482 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.3176e-05 2024-09-16T14:48:40.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.482 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:40.482 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.905339+0000", 2024-09-16T14:48:40.482 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0176e-05 2024-09-16T14:48:40.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.483 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.483 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:40.483 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.906010+0000", 2024-09-16T14:48:40.483 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00067085500000000004 2024-09-16T14:48:40.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:40.483 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:40.483 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:40.483 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:39.906046+0000", 2024-09-16T14:48:40.483 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6081000000000002e-05 2024-09-16T14:48:40.483 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:40.483 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:40.484 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:40.484 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:40.484 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:40.484 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:48:40.485 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:40.484+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:40.486 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:48:40.527 INFO:teuthology.orchestra.run.smithi111.stdout:m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32 2024-09-16T14:48:40.531 INFO:tasks.thrashosds.thrasher:applying new sharding to bluestore on osd.6 2024-09-16T14:48:40.531 INFO:tasks.thrashosds.thrasher:generate random sharding 2024-09-16T14:48:40.531 INFO:tasks.thrashosds.thrasher:applying new sharding: m(2,8-16) 2024-09-16T14:48:40.531 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph-bluestore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/bluestore_tool.$pid.log' --log-level=10 --path /var/lib/ceph/osd/ceph-6 --sharding 'm(2,8-16)' reshard 2024-09-16T14:48:40.586 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:40.585+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:40.638 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:48:40.639 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:48:40.639 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:48:40.639 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:48:40.643 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:48:40.673 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:48:40.687 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:40.686+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:40.750 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:48:40.750 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:48:40.750 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:48:40.750 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:48:40.750 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:48:40.759 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:48:40.802 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:40.801+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:40.811 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:48:40.811 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:48:40.828 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:48:40.828 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:48:40.845 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:48:40.845 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:48:40.860 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:48:40.861 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:48:40.868 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:40.868 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:40.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:48:40.878 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:48:40.878 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:48:40.880 INFO:teuthology.orchestra.run.smithi111.stdout:reshard success 2024-09-16T14:48:40.894 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:48:40.894 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:48:40.895 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:40.893+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:40.895 INFO:teuthology.orchestra.run.smithi040.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T14:48:40.904 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is unset 2024-09-16T14:48:40.904 INFO:tasks.rados.rados.0.smithi040.stdout:82: done (0 left) 2024-09-16T14:48:40.904 INFO:tasks.rados.rados.0.smithi040.stdout:83: copy_from oid 35 from oid 22 current snap is 3 2024-09-16T14:48:40.904 INFO:tasks.rados.rados.0.smithi040.stdout:84: read oid 15 snap -1 2024-09-16T14:48:40.904 INFO:tasks.rados.rados.0.smithi040.stdout:84: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-16T14:48:40.905 INFO:tasks.rados.rados.0.smithi040.stdout:85: delete oid 24 current snap is 3 2024-09-16T14:48:40.914 INFO:tasks.rados.rados.0.smithi040.stdout:84: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-16T14:48:40.921 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:48:40.922 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:48:40.938 INFO:tasks.rados.rados.0.smithi040.stdout:83: finishing copy_from racing read to smithi04040151-35 2024-09-16T14:48:40.939 INFO:tasks.rados.rados.0.smithi040.stdout:83: finishing copy_from to smithi04040151-35 2024-09-16T14:48:40.939 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 35 v 4 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-16T14:48:40.939 INFO:tasks.rados.rados.0.smithi040.stdout:83: done (2 left) 2024-09-16T14:48:40.939 INFO:tasks.rados.rados.0.smithi040.stdout:84: done (1 left) 2024-09-16T14:48:40.939 INFO:tasks.rados.rados.0.smithi040.stdout:85: done (0 left) 2024-09-16T14:48:40.939 INFO:tasks.rados.rados.0.smithi040.stdout:86: setattr oid 20 current snap is 3 2024-09-16T14:48:40.940 INFO:tasks.rados.rados.0.smithi040.stdout:87: read oid 22 snap 2 2024-09-16T14:48:40.940 INFO:tasks.rados.rados.0.smithi040.stdout:87: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-16T14:48:40.940 INFO:tasks.rados.rados.0.smithi040.stdout:88: copy_from oid 39 from oid 12 current snap is 3 2024-09-16T14:48:40.940 INFO:tasks.rados.rados.0.smithi040.stdout:89: read oid 10 snap -1 2024-09-16T14:48:40.940 INFO:tasks.rados.rados.0.smithi040.stdout:89: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-16T14:48:40.940 INFO:tasks.rados.rados.0.smithi040.stdout:90: snap_remove snap 0 2024-09-16T14:48:40.941 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:48:40.942 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:48:40.961 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:48:40.962 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:48:40.978 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:48:40.979 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:48:40.989 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:40.988+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:40.995 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:48:40.996 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:48:40.999 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:48:40.999 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:48:41.014 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:41.014 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:41.022 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:48:41.090 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:41.089+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:41.171 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:41.171 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:41.179 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:48:41.191 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:41.190+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:41.286 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:41.287 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:41.292 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:41.291+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:41.295 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:48:41.344 INFO:tasks.thrashosds.thrasher:running fsck to verify new sharding on osd.6 2024-09-16T14:48:41.344 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph-bluestore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/bluestore_tool.$pid.log' --log-level=10 --path /var/lib/ceph/osd/ceph-6 fsck 2024-09-16T14:48:41.402 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:41.402 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:41.411 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:48:41.428 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:41.427+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:41.493 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:41.492+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:41.517 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:41.517 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:41.526 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:48:41.594 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:41.593+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:41.673 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:41.673 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:41.682 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:48:41.694 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:41.693+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:41.795 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:41.794+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:41.797 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:41.798 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:41.807 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:48:41.897 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:41.895+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:41.905 INFO:tasks.rados.rados.0.smithi040.stdout:90: done (4 left) 2024-09-16T14:48:41.905 INFO:tasks.rados.rados.0.smithi040.stdout:91: setattr oid 35 current snap is 3 2024-09-16T14:48:41.905 INFO:tasks.rados.rados.0.smithi040.stdout:89: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-16T14:48:41.912 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 20 v 6 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-16T14:48:41.912 INFO:tasks.rados.rados.0.smithi040.stdout:88: finishing copy_from racing read to smithi04040151-39 2024-09-16T14:48:41.913 INFO:tasks.rados.rados.0.smithi040.stdout:87: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-16T14:48:41.916 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:41.916 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:41.926 INFO:tasks.rados.rados.0.smithi040.stdout:88: finishing copy_from to smithi04040151-39 2024-09-16T14:48:41.926 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 39 v 2 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-16T14:48:41.928 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:48:41.929 INFO:tasks.rados.rados.0.smithi040.stdout:86: done (4 left) 2024-09-16T14:48:41.929 INFO:tasks.rados.rados.0.smithi040.stdout:87: done (3 left) 2024-09-16T14:48:41.929 INFO:tasks.rados.rados.0.smithi040.stdout:88: done (2 left) 2024-09-16T14:48:41.929 INFO:tasks.rados.rados.0.smithi040.stdout:89: done (1 left) 2024-09-16T14:48:41.929 INFO:tasks.rados.rados.0.smithi040.stdout:92: copy_from oid 17 from oid 32 current snap is 3 2024-09-16T14:48:41.930 INFO:tasks.rados.rados.0.smithi040.stdout:93: read oid 19 snap -1 2024-09-16T14:48:41.930 INFO:tasks.rados.rados.0.smithi040.stdout:93: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-16T14:48:41.930 INFO:tasks.rados.rados.0.smithi040.stdout:94: rollback oid 31 current snap is 3 2024-09-16T14:48:41.930 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 31 to 2 2024-09-16T14:48:41.930 INFO:tasks.rados.rados.0.smithi040.stdout:95: copy_from oid 18 from oid 38 current snap is 3 2024-09-16T14:48:41.930 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 43 current snap is 3 2024-09-16T14:48:41.930 INFO:tasks.rados.rados.0.smithi040.stdout:96: seq_num 62 ranges {720896=671744,1392640=376832} 2024-09-16T14:48:41.937 INFO:tasks.rados.rados.0.smithi040.stdout:96: writing smithi04040151-43 from 720896 to 1392640 tid 1 2024-09-16T14:48:41.941 INFO:tasks.rados.rados.0.smithi040.stdout:96: writing smithi04040151-43 from 1392640 to 1769472 tid 2 2024-09-16T14:48:41.941 INFO:tasks.rados.rados.0.smithi040.stdout:97: copy_from oid 39 from oid 40 current snap is 3 2024-09-16T14:48:41.941 INFO:tasks.rados.rados.0.smithi040.stdout:93: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-16T14:48:41.953 INFO:tasks.rados.rados.0.smithi040.stdout:94: finishing rollback tid 0 to smithi04040151-31 2024-09-16T14:48:41.954 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 35 v 9 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-16T14:48:41.954 INFO:tasks.rados.rados.0.smithi040.stdout:94: finishing rollback tid 1 to smithi04040151-31 2024-09-16T14:48:41.954 INFO:tasks.rados.rados.0.smithi040.stdout:94: finishing rollback tid 2 to smithi04040151-31 2024-09-16T14:48:41.954 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 31 v 32 (ObjNum 55 snap 1 seq_num 55) dirty exists 2024-09-16T14:48:41.954 INFO:tasks.rados.rados.0.smithi040.stdout:92: finishing copy_from racing read to smithi04040151-17 2024-09-16T14:48:41.954 INFO:tasks.rados.rados.0.smithi040.stdout:95: finishing copy_from racing read to smithi04040151-18 2024-09-16T14:48:41.954 INFO:tasks.rados.rados.0.smithi040.stdout:91: done (6 left) 2024-09-16T14:48:41.954 INFO:tasks.rados.rados.0.smithi040.stdout:93: done (5 left) 2024-09-16T14:48:41.954 INFO:tasks.rados.rados.0.smithi040.stdout:94: done (4 left) 2024-09-16T14:48:41.954 INFO:tasks.rados.rados.0.smithi040.stdout:98: rollback oid 33 current snap is 3 2024-09-16T14:48:41.954 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 33 to 2 2024-09-16T14:48:41.954 INFO:tasks.rados.rados.0.smithi040.stdout:92: finishing copy_from to smithi04040151-17 2024-09-16T14:48:41.955 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 17 v 8 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-16T14:48:41.955 INFO:tasks.rados.rados.0.smithi040.stdout:95: finishing copy_from to smithi04040151-18 2024-09-16T14:48:41.955 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 18 v 8 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-16T14:48:41.955 INFO:tasks.rados.rados.0.smithi040.stdout:96: finishing write tid 1 to smithi04040151-43 2024-09-16T14:48:41.955 INFO:tasks.rados.rados.0.smithi040.stdout:96: finishing write tid 2 to smithi04040151-43 2024-09-16T14:48:41.955 INFO:tasks.rados.rados.0.smithi040.stdout:96: finishing write tid 3 to smithi04040151-43 2024-09-16T14:48:41.955 INFO:tasks.rados.rados.0.smithi040.stdout:97: finishing copy_from racing read to smithi04040151-39 2024-09-16T14:48:41.955 INFO:tasks.rados.rados.0.smithi040.stdout:96: finishing write tid 4 to smithi04040151-43 2024-09-16T14:48:41.955 INFO:tasks.rados.rados.0.smithi040.stdout:96: oid 43 updating version 0 to 27 2024-09-16T14:48:41.955 INFO:tasks.rados.rados.0.smithi040.stdout:96: oid 43 updating version 27 to 29 2024-09-16T14:48:41.955 INFO:tasks.rados.rados.0.smithi040.stdout:96: oid 43 version 29 is already newer than 28 2024-09-16T14:48:41.955 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 43 v 29 (ObjNum 62 snap 3 seq_num 62) dirty exists 2024-09-16T14:48:41.955 INFO:tasks.rados.rados.0.smithi040.stdout:96: left oid 43 (ObjNum 62 snap 3 seq_num 62) 2024-09-16T14:48:41.955 INFO:tasks.rados.rados.0.smithi040.stdout:92: done (4 left) 2024-09-16T14:48:41.956 INFO:tasks.rados.rados.0.smithi040.stdout:95: done (3 left) 2024-09-16T14:48:41.956 INFO:tasks.rados.rados.0.smithi040.stdout:96: done (2 left) 2024-09-16T14:48:41.956 INFO:tasks.rados.rados.0.smithi040.stdout:99: setattr oid 37 current snap is 3 2024-09-16T14:48:41.956 INFO:tasks.rados.rados.0.smithi040.stdout:97: finishing copy_from to smithi04040151-39 2024-09-16T14:48:41.956 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 39 v 10 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-16T14:48:41.957 INFO:tasks.rados.rados.0.smithi040.stdout:97: done (2 left) 2024-09-16T14:48:41.957 INFO:tasks.rados.rados.0.smithi040.stdout:100: rollback oid 50 current snap is 3 2024-09-16T14:48:41.957 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 50 to 2 2024-09-16T14:48:41.957 INFO:tasks.rados.rados.0.smithi040.stdout:101: read oid 39 snap -1 2024-09-16T14:48:41.957 INFO:tasks.rados.rados.0.smithi040.stdout:101: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-16T14:48:41.957 INFO:tasks.rados.rados.0.smithi040.stdout:102: read oid 9 snap -1 2024-09-16T14:48:41.957 INFO:tasks.rados.rados.0.smithi040.stdout:102: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-16T14:48:41.958 INFO:tasks.rados.rados.0.smithi040.stdout:103: rollback oid 40 current snap is 3 2024-09-16T14:48:41.958 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 40 to 2 2024-09-16T14:48:41.958 INFO:tasks.rados.rados.0.smithi040.stdout:104: rollback oid 4 current snap is 3 2024-09-16T14:48:41.958 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 4 to 2 2024-09-16T14:48:41.958 INFO:tasks.rados.rados.0.smithi040.stdout:105: rollback oid 47 current snap is 3 2024-09-16T14:48:41.958 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 47 to 2 2024-09-16T14:48:41.958 INFO:tasks.rados.rados.0.smithi040.stdout:106: delete oid 16 current snap is 3 2024-09-16T14:48:41.962 INFO:tasks.rados.rados.0.smithi040.stdout:98: finishing rollback tid 0 to smithi04040151-33 2024-09-16T14:48:41.963 INFO:tasks.rados.rados.0.smithi040.stdout:106: done (8 left) 2024-09-16T14:48:41.963 INFO:tasks.rados.rados.0.smithi040.stdout:107: rollback oid 34 current snap is 3 2024-09-16T14:48:41.963 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 34 to 2 2024-09-16T14:48:41.963 INFO:tasks.rados.rados.0.smithi040.stdout:108: read oid 1 snap -1 2024-09-16T14:48:41.963 INFO:tasks.rados.rados.0.smithi040.stdout:108: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-16T14:48:41.963 INFO:tasks.rados.rados.0.smithi040.stdout:109: delete oid 30 current snap is 3 2024-09-16T14:48:41.964 INFO:tasks.rados.rados.0.smithi040.stdout:102: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-16T14:48:41.969 INFO:tasks.rados.rados.0.smithi040.stdout:98: finishing rollback tid 1 to smithi04040151-33 2024-09-16T14:48:41.969 INFO:tasks.rados.rados.0.smithi040.stdout:105: finishing rollback tid 0 to smithi04040151-47 2024-09-16T14:48:41.969 INFO:tasks.rados.rados.0.smithi040.stdout:98: finishing rollback tid 2 to smithi04040151-33 2024-09-16T14:48:41.969 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 33 v 12 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-16T14:48:41.970 INFO:tasks.rados.rados.0.smithi040.stdout:104: finishing rollback tid 0 to smithi04040151-4 2024-09-16T14:48:41.970 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 37 v 31 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-16T14:48:41.970 INFO:tasks.rados.rados.0.smithi040.stdout:105: finishing rollback tid 1 to smithi04040151-47 2024-09-16T14:48:41.970 INFO:tasks.rados.rados.0.smithi040.stdout:101: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-16T14:48:41.989 INFO:tasks.rados.rados.0.smithi040.stdout:103: finishing rollback tid 0 to smithi04040151-40 2024-09-16T14:48:41.989 INFO:tasks.rados.rados.0.smithi040.stdout:100: finishing rollback tid 0 to smithi04040151-50 2024-09-16T14:48:41.989 INFO:tasks.rados.rados.0.smithi040.stdout:98: done (10 left) 2024-09-16T14:48:41.989 INFO:tasks.rados.rados.0.smithi040.stdout:99: done (9 left) 2024-09-16T14:48:41.989 INFO:tasks.rados.rados.0.smithi040.stdout:101: done (8 left) 2024-09-16T14:48:41.989 INFO:tasks.rados.rados.0.smithi040.stdout:102: done (7 left) 2024-09-16T14:48:41.989 INFO:tasks.rados.rados.0.smithi040.stdout:109: done (6 left) 2024-09-16T14:48:41.989 INFO:tasks.rados.rados.0.smithi040.stdout:110: snap_create 2024-09-16T14:48:41.989 INFO:tasks.rados.rados.0.smithi040.stdout:104: finishing rollback tid 1 to smithi04040151-4 2024-09-16T14:48:41.989 INFO:tasks.rados.rados.0.smithi040.stdout:104: finishing rollback tid 2 to smithi04040151-4 2024-09-16T14:48:41.989 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 4 v 13 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-16T14:48:41.990 INFO:tasks.rados.rados.0.smithi040.stdout:100: finishing rollback tid 1 to smithi04040151-50 2024-09-16T14:48:41.990 INFO:tasks.rados.rados.0.smithi040.stdout:103: finishing rollback tid 1 to smithi04040151-40 2024-09-16T14:48:41.990 INFO:tasks.rados.rados.0.smithi040.stdout:100: finishing rollback tid 2 to smithi04040151-50 2024-09-16T14:48:41.990 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 50 v 12 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-16T14:48:41.990 INFO:tasks.rados.rados.0.smithi040.stdout:105: finishing rollback tid 2 to smithi04040151-47 2024-09-16T14:48:41.990 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 47 v 18 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-16T14:48:41.990 INFO:tasks.rados.rados.0.smithi040.stdout:103: finishing rollback tid 2 to smithi04040151-40 2024-09-16T14:48:41.990 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 40 v 28 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-16T14:48:41.990 INFO:tasks.rados.rados.0.smithi040.stdout:108: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-16T14:48:41.990 INFO:tasks.rados.rados.0.smithi040.stdout:107: finishing rollback tid 0 to smithi04040151-34 2024-09-16T14:48:41.991 INFO:tasks.rados.rados.0.smithi040.stdout:107: finishing rollback tid 1 to smithi04040151-34 2024-09-16T14:48:41.991 INFO:tasks.rados.rados.0.smithi040.stdout:107: finishing rollback tid 2 to smithi04040151-34 2024-09-16T14:48:41.991 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 34 v 36 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-16T14:48:41.997 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:41.996+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:42.044 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:42.044 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:42.053 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:48:42.098 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:42.097+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:42.154 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:42.154 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:42.162 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:48:42.195 INFO:teuthology.orchestra.run.smithi111.stdout:fsck success 2024-09-16T14:48:42.199 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:42.198+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:42.202 INFO:tasks.thrashosds.thrasher:resharding successfully completed 2024-09-16T14:48:42.203 INFO:tasks.thrashosds.thrasher:Reviving osd 6 2024-09-16T14:48:42.203 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-16T14:48:42.203 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-16T14:48:42.245 INFO:tasks.ceph.osd.6:Started 2024-09-16T14:48:42.246 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:48:42.268 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:42.268 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:42.277 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:48:42.300 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:42.299+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:42.346 INFO:teuthology.orchestra.run.smithi111.stderr:no valid command found; 10 closest matches: 2024-09-16T14:48:42.346 INFO:teuthology.orchestra.run.smithi111.stderr:0 2024-09-16T14:48:42.346 INFO:teuthology.orchestra.run.smithi111.stderr:1 2024-09-16T14:48:42.347 INFO:teuthology.orchestra.run.smithi111.stderr:2 2024-09-16T14:48:42.347 INFO:teuthology.orchestra.run.smithi111.stderr:abort 2024-09-16T14:48:42.347 INFO:teuthology.orchestra.run.smithi111.stderr:assert 2024-09-16T14:48:42.347 INFO:teuthology.orchestra.run.smithi111.stderr:config diff 2024-09-16T14:48:42.347 INFO:teuthology.orchestra.run.smithi111.stderr:config diff get 2024-09-16T14:48:42.347 INFO:teuthology.orchestra.run.smithi111.stderr:config get 2024-09-16T14:48:42.347 INFO:teuthology.orchestra.run.smithi111.stderr:config help [] 2024-09-16T14:48:42.347 INFO:teuthology.orchestra.run.smithi111.stderr:config set ... 2024-09-16T14:48:42.347 INFO:teuthology.orchestra.run.smithi111.stderr:admin_socket: invalid command 2024-09-16T14:48:42.349 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T14:48:42.349 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2024-09-16T14:48:42.400 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:42.399+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:42.421 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:42.421 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:42.429 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:48:42.502 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:42.501+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:42.531 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:42.531 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:42.541 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:48:42.603 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:42.602+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:42.643 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:42.643 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:42.651 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:48:42.704 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:42.703+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:42.754 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:42.754 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:42.762 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:48:42.805 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:42.804+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:42.832 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:42.831+0000 7fcdec02c640 -1 Falling back to public interface 2024-09-16T14:48:42.877 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:42.877 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:42.886 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:48:42.906 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:42.905+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:42.913 INFO:tasks.rados.rados.0.smithi040.stdout:100: done (6 left) 2024-09-16T14:48:42.913 INFO:tasks.rados.rados.0.smithi040.stdout:103: done (5 left) 2024-09-16T14:48:42.913 INFO:tasks.rados.rados.0.smithi040.stdout:104: done (4 left) 2024-09-16T14:48:42.913 INFO:tasks.rados.rados.0.smithi040.stdout:105: done (3 left) 2024-09-16T14:48:42.913 INFO:tasks.rados.rados.0.smithi040.stdout:107: done (2 left) 2024-09-16T14:48:42.913 INFO:tasks.rados.rados.0.smithi040.stdout:108: done (1 left) 2024-09-16T14:48:42.913 INFO:tasks.rados.rados.0.smithi040.stdout:110: done (0 left) 2024-09-16T14:48:42.913 INFO:tasks.rados.rados.0.smithi040.stdout:111: rmattr oid 8 current snap is 4 2024-09-16T14:48:42.913 INFO:tasks.rados.rados.0.smithi040.stdout:111: done (0 left) 2024-09-16T14:48:42.914 INFO:tasks.rados.rados.0.smithi040.stdout:112: setattr oid 26 current snap is 4 2024-09-16T14:48:42.914 INFO:tasks.rados.rados.0.smithi040.stdout:113: snap_remove snap 2 2024-09-16T14:48:42.929 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:48:42.989 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:42.990 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:42.998 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:48:43.000 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:48:43.042 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:43.041+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:43.102 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:43.102 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:43.108 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:43.107+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:43.111 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:48:43.128 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:48:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:48:43.146 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:48:43.146 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:48:43.162 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:48:43.162 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:48:43.178 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:48:43.178 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:48:43.195 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:48:43.195 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:48:43.208 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:43.207+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:43.211 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:48:43.212 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:48:43.212 INFO:teuthology.orchestra.run.smithi040.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T14:48:43.229 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:48:43.229 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:48:43.245 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:48:43.245 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:48:43.254 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:43.254 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:43.261 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:48:43.261 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:48:43.262 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:48:43.278 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:48:43.278 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:48:43.298 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:48:43.299 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:48:43.299 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:48:43.299 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:48:43.346 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:43.345+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:43.405 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:43.405 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:43.405 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:43.405 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:43.405 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:43.410 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:43.409+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:43.413 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:48:43.511 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:43.510+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:43.557 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:43.557 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:43.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:43.557 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:43.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.557 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 32/27 ECSubWrite(tid=21, reqid=client.4500.0:214, at_version=32'12, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:43.557 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.899824+0000", 2024-09-16T14:48:43.557 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6550843640000004, 2024-09-16T14:48:43.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016897220000000001, 2024-09-16T14:48:43.558 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.558 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.558 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899824+0000", 2024-09-16T14:48:43.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899824+0000", 2024-09-16T14:48:43.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899824+0000", 2024-09-16T14:48:43.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:43.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900207+0000", 2024-09-16T14:48:43.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00038250900000000001 2024-09-16T14:48:43.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900222+0000", 2024-09-16T14:48:43.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5319e-05 2024-09-16T14:48:43.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900231+0000", 2024-09-16T14:48:43.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4420000000000004e-06 2024-09-16T14:48:43.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900291+0000", 2024-09-16T14:48:43.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9607e-05 2024-09-16T14:48:43.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900302+0000", 2024-09-16T14:48:43.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0607e-05 2024-09-16T14:48:43.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.901488+0000", 2024-09-16T14:48:43.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001186269 2024-09-16T14:48:43.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.901514+0000", 2024-09-16T14:48:43.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6231999999999999e-05 2024-09-16T14:48:43.562 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.562 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.562 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.562 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 32/27 ECSubWrite(tid=22, reqid=client.4500.0:215, at_version=32'13, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:43.562 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.901874+0000", 2024-09-16T14:48:43.562 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6530351119999995, 2024-09-16T14:48:43.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0024190489999999999, 2024-09-16T14:48:43.563 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.563 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.563 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.901874+0000", 2024-09-16T14:48:43.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.901874+0000", 2024-09-16T14:48:43.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.901873+0000", 2024-09-16T14:48:43.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:43.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902117+0000", 2024-09-16T14:48:43.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00024386500000000001 2024-09-16T14:48:43.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902132+0000", 2024-09-16T14:48:43.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4941e-05 2024-09-16T14:48:43.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902142+0000", 2024-09-16T14:48:43.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0604e-05 2024-09-16T14:48:43.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902189+0000", 2024-09-16T14:48:43.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6449000000000002e-05 2024-09-16T14:48:43.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902199+0000", 2024-09-16T14:48:43.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0244000000000001e-05 2024-09-16T14:48:43.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904270+0000", 2024-09-16T14:48:43.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020705680000000001 2024-09-16T14:48:43.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904293+0000", 2024-09-16T14:48:43.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3105e-05 2024-09-16T14:48:43.567 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.567 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.567 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.567 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 32/27 ECSubWrite(tid=23, reqid=client.4500.0:216, at_version=32'14, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:43.567 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.902576+0000", 2024-09-16T14:48:43.567 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.6523327180000003, 2024-09-16T14:48:43.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00174631, 2024-09-16T14:48:43.567 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.568 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.568 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902576+0000", 2024-09-16T14:48:43.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902576+0000", 2024-09-16T14:48:43.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902575+0000", 2024-09-16T14:48:43.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:43.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902595+0000", 2024-09-16T14:48:43.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9343000000000001e-05 2024-09-16T14:48:43.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902605+0000", 2024-09-16T14:48:43.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0562e-05 2024-09-16T14:48:43.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902613+0000", 2024-09-16T14:48:43.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3300000000000001e-06 2024-09-16T14:48:43.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902660+0000", 2024-09-16T14:48:43.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7271999999999997e-05 2024-09-16T14:48:43.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902668+0000", 2024-09-16T14:48:43.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0029999999999994e-06 2024-09-16T14:48:43.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904307+0000", 2024-09-16T14:48:43.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001638919 2024-09-16T14:48:43.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904322+0000", 2024-09-16T14:48:43.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5580999999999999e-05 2024-09-16T14:48:43.572 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.572 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.572 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.572 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 33/27 ECSubWrite(tid=166, reqid=client.4500.0:222, at_version=33'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:48:43.572 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:39.902497+0000", 2024-09-16T14:48:43.572 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.6524119580000001, 2024-09-16T14:48:43.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001638549, 2024-09-16T14:48:43.572 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.573 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.573 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.902497+0000", 2024-09-16T14:48:43.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.902497+0000", 2024-09-16T14:48:43.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.902496+0000", 2024-09-16T14:48:43.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:43.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.902531+0000", 2024-09-16T14:48:43.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4304000000000002e-05 2024-09-16T14:48:43.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.902542+0000", 2024-09-16T14:48:43.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0933e-05 2024-09-16T14:48:43.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.902550+0000", 2024-09-16T14:48:43.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5120000000000008e-06 2024-09-16T14:48:43.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.902598+0000", 2024-09-16T14:48:43.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7726999999999999e-05 2024-09-16T14:48:43.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.902608+0000", 2024-09-16T14:48:43.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9070000000000002e-06 2024-09-16T14:48:43.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.904106+0000", 2024-09-16T14:48:43.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001498482 2024-09-16T14:48:43.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.904135+0000", 2024-09-16T14:48:43.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9071999999999999e-05 2024-09-16T14:48:43.576 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.577 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.577 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 33/27 ECSubWrite(tid=73, reqid=client.4500.0:223, at_version=33'12, trim_to=0'0, roll_forward_to=30'10))", 2024-09-16T14:48:43.577 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:39.913116+0000", 2024-09-16T14:48:43.577 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.641793254, 2024-09-16T14:48:43.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014959109999999999, 2024-09-16T14:48:43.577 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.577 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.577 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.913116+0000", 2024-09-16T14:48:43.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.913116+0000", 2024-09-16T14:48:43.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.913114+0000", 2024-09-16T14:48:43.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:43.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.913514+0000", 2024-09-16T14:48:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00039931800000000001 2024-09-16T14:48:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.913531+0000", 2024-09-16T14:48:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6775000000000001e-05 2024-09-16T14:48:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.913539+0000", 2024-09-16T14:48:43.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9430000000000006e-06 2024-09-16T14:48:43.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.913587+0000", 2024-09-16T14:48:43.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7366e-05 2024-09-16T14:48:43.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.913593+0000", 2024-09-16T14:48:43.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2759999999999997e-06 2024-09-16T14:48:43.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.914593+0000", 2024-09-16T14:48:43.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00099974200000000008 2024-09-16T14:48:43.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.914611+0000", 2024-09-16T14:48:43.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8647e-05 2024-09-16T14:48:43.581 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.581 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.581 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.582 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 33/27 ECSubWrite(tid=74, reqid=client.4500.0:224, at_version=33'13, trim_to=0'0, roll_forward_to=30'10))", 2024-09-16T14:48:43.582 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:39.915493+0000", 2024-09-16T14:48:43.582 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.6394159459999997, 2024-09-16T14:48:43.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0013931760000000001, 2024-09-16T14:48:43.582 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.582 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.582 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915493+0000", 2024-09-16T14:48:43.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915493+0000", 2024-09-16T14:48:43.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915492+0000", 2024-09-16T14:48:43.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:43.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915652+0000", 2024-09-16T14:48:43.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00015936100000000001 2024-09-16T14:48:43.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915666+0000", 2024-09-16T14:48:43.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4817000000000001e-05 2024-09-16T14:48:43.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915674+0000", 2024-09-16T14:48:43.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1000000000000004e-06 2024-09-16T14:48:43.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915727+0000", 2024-09-16T14:48:43.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2567e-05 2024-09-16T14:48:43.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915737+0000", 2024-09-16T14:48:43.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5699999999999999e-06 2024-09-16T14:48:43.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.916865+0000", 2024-09-16T14:48:43.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011283689999999999 2024-09-16T14:48:43.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.916886+0000", 2024-09-16T14:48:43.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1106000000000001e-05 2024-09-16T14:48:43.586 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.586 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.586 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.586 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpRead(3.cs4 34/27 ECSubRead(tid=32, to_read={3:37d15b23:::smithi04040151-15:head=0,184320,0}, subchunks={3:37d15b23:::smithi04040151-15:head=[0,1]}, attrs_to_read=))", 2024-09-16T14:48:43.586 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.906820+0000", 2024-09-16T14:48:43.586 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.6480892379999998, 2024-09-16T14:48:43.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0027119140000000002, 2024-09-16T14:48:43.586 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.587 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.587 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906820+0000", 2024-09-16T14:48:43.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906820+0000", 2024-09-16T14:48:43.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906819+0000", 2024-09-16T14:48:43.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:43.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906832+0000", 2024-09-16T14:48:43.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2656999999999999e-05 2024-09-16T14:48:43.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906881+0000", 2024-09-16T14:48:43.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9431000000000002e-05 2024-09-16T14:48:43.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906902+0000", 2024-09-16T14:48:43.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0480999999999999e-05 2024-09-16T14:48:43.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906966+0000", 2024-09-16T14:48:43.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3795999999999997e-05 2024-09-16T14:48:43.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909532+0000", 2024-09-16T14:48:43.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0025659400000000001 2024-09-16T14:48:43.590 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.590 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.590 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.590 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpRead(3.cs4 34/27 ECSubRead(tid=33, to_read={3:37d15b23:::smithi04040151-15:head=0,184320,0}, subchunks={3:37d15b23:::smithi04040151-15:head=[0,1]}, attrs_to_read=))", 2024-09-16T14:48:43.590 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.907055+0000", 2024-09-16T14:48:43.590 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.6478535430000001, 2024-09-16T14:48:43.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0026192630000000001, 2024-09-16T14:48:43.590 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.590 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.590 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907055+0000", 2024-09-16T14:48:43.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907055+0000", 2024-09-16T14:48:43.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907054+0000", 2024-09-16T14:48:43.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:43.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907066+0000", 2024-09-16T14:48:43.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2143e-05 2024-09-16T14:48:43.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907081+0000", 2024-09-16T14:48:43.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4752e-05 2024-09-16T14:48:43.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907091+0000", 2024-09-16T14:48:43.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0135999999999999e-05 2024-09-16T14:48:43.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909564+0000", 2024-09-16T14:48:43.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002472981 2024-09-16T14:48:43.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909675+0000", 2024-09-16T14:48:43.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000110194 2024-09-16T14:48:43.594 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.594 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.594 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.594 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=79, reqid=osd.3.0:79, at_version=34'18, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:48:43.594 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:43.594 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.647071747, 2024-09-16T14:48:43.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069050049999999997, 2024-09-16T14:48:43.594 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.594 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.594 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:43.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:43.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:43.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:43.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907863+0000", 2024-09-16T14:48:43.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5972000000000001e-05 2024-09-16T14:48:43.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907876+0000", 2024-09-16T14:48:43.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3230999999999999e-05 2024-09-16T14:48:43.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907900+0000", 2024-09-16T14:48:43.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3716000000000001e-05 2024-09-16T14:48:43.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907956+0000", 2024-09-16T14:48:43.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6307999999999997e-05 2024-09-16T14:48:43.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907965+0000", 2024-09-16T14:48:43.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1530000000000003e-06 2024-09-16T14:48:43.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914715+0000", 2024-09-16T14:48:43.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067495990000000002 2024-09-16T14:48:43.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914742+0000", 2024-09-16T14:48:43.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.728e-05 2024-09-16T14:48:43.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.598 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.599 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=80, reqid=osd.3.0:80, at_version=34'20, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:48:43.599 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:48:43.599 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.6456550769999998, 2024-09-16T14:48:43.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065560849999999997, 2024-09-16T14:48:43.599 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.599 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.599 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:48:43.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:48:43.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909253+0000", 2024-09-16T14:48:43.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:43.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909278+0000", 2024-09-16T14:48:43.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4905e-05 2024-09-16T14:48:43.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909295+0000", 2024-09-16T14:48:43.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6713000000000001e-05 2024-09-16T14:48:43.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909304+0000", 2024-09-16T14:48:43.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6739999999999992e-06 2024-09-16T14:48:43.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909367+0000", 2024-09-16T14:48:43.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2700999999999995e-05 2024-09-16T14:48:43.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909391+0000", 2024-09-16T14:48:43.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3614000000000001e-05 2024-09-16T14:48:43.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915784+0000", 2024-09-16T14:48:43.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063938479999999997 2024-09-16T14:48:43.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915810+0000", 2024-09-16T14:48:43.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5457999999999999e-05 2024-09-16T14:48:43.603 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.603 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.603 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.603 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:48:43.604 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:43.604 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.6450274880000002, 2024-09-16T14:48:43.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007542137, 2024-09-16T14:48:43.604 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.604 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.604 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:43.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:43.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:43.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:43.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909917+0000", 2024-09-16T14:48:43.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6513000000000001e-05 2024-09-16T14:48:43.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:48:43.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:48:43.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909941+0000", 2024-09-16T14:48:43.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1859999999999992e-06 2024-09-16T14:48:43.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910004+0000", 2024-09-16T14:48:43.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2706999999999994e-05 2024-09-16T14:48:43.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910012+0000", 2024-09-16T14:48:43.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.157e-06 2024-09-16T14:48:43.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917395+0000", 2024-09-16T14:48:43.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073827659999999998 2024-09-16T14:48:43.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917423+0000", 2024-09-16T14:48:43.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8568000000000002e-05 2024-09-16T14:48:43.608 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.608 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.608 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.608 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:48:43.608 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:48:43.608 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.6408652479999999, 2024-09-16T14:48:43.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065747590000000003, 2024-09-16T14:48:43.609 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.609 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.609 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:48:43.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:48:43.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914043+0000", 2024-09-16T14:48:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914071+0000", 2024-09-16T14:48:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7633000000000001e-05 2024-09-16T14:48:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914086+0000", 2024-09-16T14:48:43.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4973999999999999e-05 2024-09-16T14:48:43.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914095+0000", 2024-09-16T14:48:43.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2119999999999992e-06 2024-09-16T14:48:43.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915626+0000", 2024-09-16T14:48:43.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001530998 2024-09-16T14:48:43.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915635+0000", 2024-09-16T14:48:43.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0140000000000004e-06 2024-09-16T14:48:43.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920590+0000", 2024-09-16T14:48:43.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049547740000000003 2024-09-16T14:48:43.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920618+0000", 2024-09-16T14:48:43.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8767000000000001e-05 2024-09-16T14:48:43.613 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.613 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.613 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.613 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es5 34/27 ECSubWrite(tid=74, reqid=client.4500.0:229, at_version=34'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:48:43.613 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914267+0000", 2024-09-16T14:48:43.613 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.64064152, 2024-09-16T14:48:43.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031688250000000001, 2024-09-16T14:48:43.614 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.614 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.614 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914267+0000", 2024-09-16T14:48:43.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914267+0000", 2024-09-16T14:48:43.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914266+0000", 2024-09-16T14:48:43.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:43.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914322+0000", 2024-09-16T14:48:43.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5646000000000002e-05 2024-09-16T14:48:43.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914342+0000", 2024-09-16T14:48:43.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0213000000000001e-05 2024-09-16T14:48:43.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914353+0000", 2024-09-16T14:48:43.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1177e-05 2024-09-16T14:48:43.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914412+0000", 2024-09-16T14:48:43.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8366999999999999e-05 2024-09-16T14:48:43.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914424+0000", 2024-09-16T14:48:43.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2303999999999999e-05 2024-09-16T14:48:43.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917421+0000", 2024-09-16T14:48:43.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0029973040000000001 2024-09-16T14:48:43.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917436+0000", 2024-09-16T14:48:43.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.486e-05 2024-09-16T14:48:43.617 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.617 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.618 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.618 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:48:43.618 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:43.618 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.6385427579999998, 2024-09-16T14:48:43.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006879947, 2024-09-16T14:48:43.618 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.618 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.618 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:43.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:43.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:43.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:43.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:43.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8702000000000002e-05 2024-09-16T14:48:43.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916394+0000", 2024-09-16T14:48:43.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6350000000000001e-06 2024-09-16T14:48:43.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916403+0000", 2024-09-16T14:48:43.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3950000000000001e-06 2024-09-16T14:48:43.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916459+0000", 2024-09-16T14:48:43.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5889e-05 2024-09-16T14:48:43.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916467+0000", 2024-09-16T14:48:43.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2770000000000005e-06 2024-09-16T14:48:43.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923224+0000", 2024-09-16T14:48:43.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067573099999999999 2024-09-16T14:48:43.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923246+0000", 2024-09-16T14:48:43.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1882000000000002e-05 2024-09-16T14:48:43.622 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.622 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.622 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.622 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=82, reqid=osd.3.0:80, at_version=0'0, trim_to=0'0, roll_forward_to=34'20))", 2024-09-16T14:48:43.623 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:48:43.623 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.6369688930000001, 2024-09-16T14:48:43.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005378954, 2024-09-16T14:48:43.623 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.623 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.623 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:48:43.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:48:43.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917939+0000", 2024-09-16T14:48:43.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:43.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917952+0000", 2024-09-16T14:48:43.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2278e-05 2024-09-16T14:48:43.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917963+0000", 2024-09-16T14:48:43.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.102e-05 2024-09-16T14:48:43.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917970+0000", 2024-09-16T14:48:43.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9589999999999998e-06 2024-09-16T14:48:43.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.918179+0000", 2024-09-16T14:48:43.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020897 2024-09-16T14:48:43.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.918188+0000", 2024-09-16T14:48:43.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9600000000000006e-06 2024-09-16T14:48:43.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923288+0000", 2024-09-16T14:48:43.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051001830000000003 2024-09-16T14:48:43.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923319+0000", 2024-09-16T14:48:43.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1131999999999999e-05 2024-09-16T14:48:43.627 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.627 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.627 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.627 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 34/27 ECSubWrite(tid=83, reqid=client.4500.0:234, at_version=0'0, trim_to=0'0, roll_forward_to=34'22))", 2024-09-16T14:48:43.627 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.919181+0000", 2024-09-16T14:48:43.627 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.6357277159999999, 2024-09-16T14:48:43.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042759809999999999, 2024-09-16T14:48:43.627 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.628 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.628 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919181+0000", 2024-09-16T14:48:43.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919181+0000", 2024-09-16T14:48:43.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919180+0000", 2024-09-16T14:48:43.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:43.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919190+0000", 2024-09-16T14:48:43.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5259999999999999e-06 2024-09-16T14:48:43.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919203+0000", 2024-09-16T14:48:43.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3198e-05 2024-09-16T14:48:43.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919211+0000", 2024-09-16T14:48:43.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2109999999999998e-06 2024-09-16T14:48:43.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919268+0000", 2024-09-16T14:48:43.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7003999999999998e-05 2024-09-16T14:48:43.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919280+0000", 2024-09-16T14:48:43.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1505000000000001e-05 2024-09-16T14:48:43.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923431+0000", 2024-09-16T14:48:43.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041514309999999997 2024-09-16T14:48:43.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923457+0000", 2024-09-16T14:48:43.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5941000000000001e-05 2024-09-16T14:48:43.631 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.631 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.631 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.632 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es5 34/27 ECSubWrite(tid=75, reqid=client.4500.0:229, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-16T14:48:43.632 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.920854+0000", 2024-09-16T14:48:43.632 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.6340552399999999, 2024-09-16T14:48:43.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050284889999999997, 2024-09-16T14:48:43.632 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.632 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.632 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920854+0000", 2024-09-16T14:48:43.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920854+0000", 2024-09-16T14:48:43.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920853+0000", 2024-09-16T14:48:43.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:43.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920863+0000", 2024-09-16T14:48:43.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0560999999999999e-05 2024-09-16T14:48:43.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920875+0000", 2024-09-16T14:48:43.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1588e-05 2024-09-16T14:48:43.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920883+0000", 2024-09-16T14:48:43.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7889999999999995e-06 2024-09-16T14:48:43.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920933+0000", 2024-09-16T14:48:43.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0759e-05 2024-09-16T14:48:43.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920943+0000", 2024-09-16T14:48:43.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.978e-06 2024-09-16T14:48:43.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925863+0000", 2024-09-16T14:48:43.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049191779999999997 2024-09-16T14:48:43.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925882+0000", 2024-09-16T14:48:43.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9525e-05 2024-09-16T14:48:43.636 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.636 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.636 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.636 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 34/27 ECSubWrite(tid=169, reqid=osd.7.0:168, at_version=0'0, trim_to=0'0, roll_forward_to=34'8))", 2024-09-16T14:48:43.636 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.921043+0000", 2024-09-16T14:48:43.636 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.6338656149999999, 2024-09-16T14:48:43.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048257229999999996, 2024-09-16T14:48:43.637 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.637 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.637 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921043+0000", 2024-09-16T14:48:43.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921043+0000", 2024-09-16T14:48:43.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921042+0000", 2024-09-16T14:48:43.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:43.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921057+0000", 2024-09-16T14:48:43.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4270999999999999e-05 2024-09-16T14:48:43.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921073+0000", 2024-09-16T14:48:43.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6821999999999999e-05 2024-09-16T14:48:43.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921200+0000", 2024-09-16T14:48:43.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012666899999999999 2024-09-16T14:48:43.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921252+0000", 2024-09-16T14:48:43.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1892999999999999e-05 2024-09-16T14:48:43.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921260+0000", 2024-09-16T14:48:43.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0649999999999999e-06 2024-09-16T14:48:43.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925844+0000", 2024-09-16T14:48:43.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045839169999999999 2024-09-16T14:48:43.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925869+0000", 2024-09-16T14:48:43.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4909000000000002e-05 2024-09-16T14:48:43.641 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.641 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.641 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.641 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 34/27 ECSubWrite(tid=27, reqid=osd.1.0:26, at_version=0'0, trim_to=0'0, roll_forward_to=34'16))", 2024-09-16T14:48:43.641 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.923892+0000", 2024-09-16T14:48:43.641 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.6310168040000002, 2024-09-16T14:48:43.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0024927920000000002, 2024-09-16T14:48:43.641 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.641 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.641 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923892+0000", 2024-09-16T14:48:43.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923892+0000", 2024-09-16T14:48:43.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923891+0000", 2024-09-16T14:48:43.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:43.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923901+0000", 2024-09-16T14:48:43.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5980000000000003e-06 2024-09-16T14:48:43.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923913+0000", 2024-09-16T14:48:43.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1851999999999999e-05 2024-09-16T14:48:43.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923921+0000", 2024-09-16T14:48:43.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5820000000000002e-06 2024-09-16T14:48:43.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923957+0000", 2024-09-16T14:48:43.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6084999999999998e-05 2024-09-16T14:48:43.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923968+0000", 2024-09-16T14:48:43.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0699e-05 2024-09-16T14:48:43.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.926359+0000", 2024-09-16T14:48:43.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023904640000000001 2024-09-16T14:48:43.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.926385+0000", 2024-09-16T14:48:43.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6290000000000001e-05 2024-09-16T14:48:43.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.645 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.645 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 34/27 ECSubWrite(tid=77, reqid=client.4500.0:235, at_version=34'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:48:43.646 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.941418+0000", 2024-09-16T14:48:43.646 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 2.6134905719999999, 2024-09-16T14:48:43.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0013741490000000001, 2024-09-16T14:48:43.646 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:43.646 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:43.646 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:43.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:43.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941418+0000", 2024-09-16T14:48:43.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:43.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941418+0000", 2024-09-16T14:48:43.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:43.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:43.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941417+0000", 2024-09-16T14:48:43.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:43.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:43.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941469+0000", 2024-09-16T14:48:43.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1928999999999997e-05 2024-09-16T14:48:43.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:43.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941490+0000", 2024-09-16T14:48:43.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0886e-05 2024-09-16T14:48:43.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:43.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941500+0000", 2024-09-16T14:48:43.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5640000000000002e-06 2024-09-16T14:48:43.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:43.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941548+0000", 2024-09-16T14:48:43.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8158999999999997e-05 2024-09-16T14:48:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941557+0000", 2024-09-16T14:48:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2499999999999995e-06 2024-09-16T14:48:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.942762+0000", 2024-09-16T14:48:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0012049630000000001 2024-09-16T14:48:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:43.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.942792+0000", 2024-09-16T14:48:43.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0380999999999999e-05 2024-09-16T14:48:43.650 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.650 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.650 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.650 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:43.650 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:43.650 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:43.651 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:48:43.652 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:43.650+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:43.752 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:43.751+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:43.753 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:43.753 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:43.754 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:43.754 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:43.762 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:48:43.853 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:43.852+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:43.904 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:43.904 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:43.905 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:43.905 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:43.905 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:43.913 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:48:43.914 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is set 2024-09-16T14:48:43.915 INFO:tasks.rados.rados.0.smithi040.stdout:113: done (1 left) 2024-09-16T14:48:43.915 INFO:tasks.rados.rados.0.smithi040.stdout:114: copy_from oid 29 from oid 16 current snap is 4 2024-09-16T14:48:43.915 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 26 v 37 (ObjNum 49 snap 0 seq_num 211206834) dirty exists 2024-09-16T14:48:43.915 INFO:tasks.rados.rados.0.smithi040.stdout:112: done (1 left) 2024-09-16T14:48:43.916 INFO:tasks.rados.rados.0.smithi040.stdout:115: snap_remove snap 3 2024-09-16T14:48:43.982 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:43.981+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:44.028 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:44.028 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:44.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:44.028 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:44.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.028 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 32/27 ECSubWrite(tid=21, reqid=client.4500.0:214, at_version=32'12, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:44.028 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.899969+0000", 2024-09-16T14:48:44.028 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.1260832729999999, 2024-09-16T14:48:44.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040995110000000001, 2024-09-16T14:48:44.029 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.029 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.029 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899969+0000", 2024-09-16T14:48:44.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899969+0000", 2024-09-16T14:48:44.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899968+0000", 2024-09-16T14:48:44.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:44.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900282+0000", 2024-09-16T14:48:44.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031321000000000002 2024-09-16T14:48:44.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900293+0000", 2024-09-16T14:48:44.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1582e-05 2024-09-16T14:48:44.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900302+0000", 2024-09-16T14:48:44.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0170000000000002e-06 2024-09-16T14:48:44.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900359+0000", 2024-09-16T14:48:44.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6750999999999999e-05 2024-09-16T14:48:44.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900369+0000", 2024-09-16T14:48:44.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9299999999999998e-06 2024-09-16T14:48:44.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904046+0000", 2024-09-16T14:48:44.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036775240000000002 2024-09-16T14:48:44.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904068+0000", 2024-09-16T14:48:44.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1710999999999999e-05 2024-09-16T14:48:44.033 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.033 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.033 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.033 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 32/27 ECSubWrite(tid=22, reqid=client.4500.0:215, at_version=32'13, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:44.033 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.902030+0000", 2024-09-16T14:48:44.033 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.1240218119999996, 2024-09-16T14:48:44.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023581710000000001, 2024-09-16T14:48:44.034 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.034 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.034 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902030+0000", 2024-09-16T14:48:44.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902030+0000", 2024-09-16T14:48:44.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902029+0000", 2024-09-16T14:48:44.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902145+0000", 2024-09-16T14:48:44.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000115652 2024-09-16T14:48:44.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902158+0000", 2024-09-16T14:48:44.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3495999999999999e-05 2024-09-16T14:48:44.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902166+0000", 2024-09-16T14:48:44.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3429999999999998e-06 2024-09-16T14:48:44.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902205+0000", 2024-09-16T14:48:44.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8884000000000002e-05 2024-09-16T14:48:44.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902213+0000", 2024-09-16T14:48:44.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9729999999999997e-06 2024-09-16T14:48:44.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904368+0000", 2024-09-16T14:48:44.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0021557450000000001 2024-09-16T14:48:44.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904388+0000", 2024-09-16T14:48:44.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9989000000000001e-05 2024-09-16T14:48:44.037 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.038 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.038 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.038 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 32/27 ECSubWrite(tid=23, reqid=client.4500.0:216, at_version=32'14, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:44.038 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.902592+0000", 2024-09-16T14:48:44.038 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.1234602469999997, 2024-09-16T14:48:44.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00180915, 2024-09-16T14:48:44.038 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.038 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.038 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902592+0000", 2024-09-16T14:48:44.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902592+0000", 2024-09-16T14:48:44.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902591+0000", 2024-09-16T14:48:44.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902609+0000", 2024-09-16T14:48:44.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8482e-05 2024-09-16T14:48:44.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902623+0000", 2024-09-16T14:48:44.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3774000000000001e-05 2024-09-16T14:48:44.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902631+0000", 2024-09-16T14:48:44.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9149999999999996e-06 2024-09-16T14:48:44.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902686+0000", 2024-09-16T14:48:44.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5383000000000002e-05 2024-09-16T14:48:44.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.902695+0000", 2024-09-16T14:48:44.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1429999999999998e-06 2024-09-16T14:48:44.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904391+0000", 2024-09-16T14:48:44.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016966290000000001 2024-09-16T14:48:44.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904401+0000", 2024-09-16T14:48:44.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5929999999999995e-06 2024-09-16T14:48:44.042 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.042 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.042 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.042 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpRead(3.7s4 33/27 ECSubRead(tid=70, to_read={3:e91f0d6c:::smithi04040151-31:head=0,286720,0}, subchunks={3:e91f0d6c:::smithi04040151-31:head=[0,1]}, attrs_to_read=))", 2024-09-16T14:48:44.042 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:39.900545+0000", 2024-09-16T14:48:44.042 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.1255067260000002, 2024-09-16T14:48:44.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001082455, 2024-09-16T14:48:44.043 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.043 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.043 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.900545+0000", 2024-09-16T14:48:44.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.900545+0000", 2024-09-16T14:48:44.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.900545+0000", 2024-09-16T14:48:44.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:44.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.900552+0000", 2024-09-16T14:48:44.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.25e-06 2024-09-16T14:48:44.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.900567+0000", 2024-09-16T14:48:44.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4813e-05 2024-09-16T14:48:44.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.900577+0000", 2024-09-16T14:48:44.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8700000000000004e-06 2024-09-16T14:48:44.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.900637+0000", 2024-09-16T14:48:44.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0365999999999998e-05 2024-09-16T14:48:44.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.901628+0000", 2024-09-16T14:48:44.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00099028700000000007 2024-09-16T14:48:44.046 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.046 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.046 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.046 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 33/27 ECSubWrite(tid=166, reqid=client.4500.0:222, at_version=33'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:48:44.046 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:39.903030+0000", 2024-09-16T14:48:44.046 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.1230219879999996, 2024-09-16T14:48:44.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0012266309999999999, 2024-09-16T14:48:44.046 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.046 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.047 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.903030+0000", 2024-09-16T14:48:44.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.903030+0000", 2024-09-16T14:48:44.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.903030+0000", 2024-09-16T14:48:44.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.903051+0000", 2024-09-16T14:48:44.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1069000000000001e-05 2024-09-16T14:48:44.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.903062+0000", 2024-09-16T14:48:44.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1336e-05 2024-09-16T14:48:44.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.903070+0000", 2024-09-16T14:48:44.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9629999999999998e-06 2024-09-16T14:48:44.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.903159+0000", 2024-09-16T14:48:44.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8802999999999997e-05 2024-09-16T14:48:44.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.903167+0000", 2024-09-16T14:48:44.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0979999999999994e-06 2024-09-16T14:48:44.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.904231+0000", 2024-09-16T14:48:44.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010639989999999999 2024-09-16T14:48:44.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.904257+0000", 2024-09-16T14:48:44.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5703000000000001e-05 2024-09-16T14:48:44.050 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.050 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.050 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.051 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 33/27 ECSubWrite(tid=73, reqid=client.4500.0:223, at_version=33'12, trim_to=0'0, roll_forward_to=30'10))", 2024-09-16T14:48:44.051 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:39.912621+0000", 2024-09-16T14:48:44.051 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.1134314160000001, 2024-09-16T14:48:44.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0013766570000000001, 2024-09-16T14:48:44.051 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.051 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.051 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.912621+0000", 2024-09-16T14:48:44.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.912621+0000", 2024-09-16T14:48:44.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.912620+0000", 2024-09-16T14:48:44.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.912956+0000", 2024-09-16T14:48:44.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033668400000000002 2024-09-16T14:48:44.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.912968+0000", 2024-09-16T14:48:44.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1559999999999999e-05 2024-09-16T14:48:44.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.912975+0000", 2024-09-16T14:48:44.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3359999999999997e-06 2024-09-16T14:48:44.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.913034+0000", 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8479e-05 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.913041+0000", 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9689999999999997e-06 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.913966+0000", 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000925438 2024-09-16T14:48:44.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.913997+0000", 2024-09-16T14:48:44.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0994000000000002e-05 2024-09-16T14:48:44.055 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.055 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.055 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.055 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 33/27 ECSubWrite(tid=74, reqid=client.4500.0:224, at_version=33'13, trim_to=0'0, roll_forward_to=30'10))", 2024-09-16T14:48:44.055 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:39.915257+0000", 2024-09-16T14:48:44.055 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.1107948739999998, 2024-09-16T14:48:44.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001591804, 2024-09-16T14:48:44.056 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.056 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.056 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915257+0000", 2024-09-16T14:48:44.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915257+0000", 2024-09-16T14:48:44.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915256+0000", 2024-09-16T14:48:44.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915490+0000", 2024-09-16T14:48:44.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000234062 2024-09-16T14:48:44.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915505+0000", 2024-09-16T14:48:44.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4185000000000001e-05 2024-09-16T14:48:44.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915512+0000", 2024-09-16T14:48:44.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5909999999999998e-06 2024-09-16T14:48:44.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915565+0000", 2024-09-16T14:48:44.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3108e-05 2024-09-16T14:48:44.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.915580+0000", 2024-09-16T14:48:44.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4555999999999999e-05 2024-09-16T14:48:44.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.916823+0000", 2024-09-16T14:48:44.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001243117 2024-09-16T14:48:44.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:39.916849+0000", 2024-09-16T14:48:44.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5913000000000001e-05 2024-09-16T14:48:44.060 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.060 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.060 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.060 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 34/27 ECSubWrite(tid=79, reqid=osd.3.0:79, at_version=34'18, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:48:44.060 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.907896+0000", 2024-09-16T14:48:44.060 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1181557880000002, 2024-09-16T14:48:44.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00146948, 2024-09-16T14:48:44.060 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.060 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.061 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907896+0000", 2024-09-16T14:48:44.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907896+0000", 2024-09-16T14:48:44.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907896+0000", 2024-09-16T14:48:44.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907923+0000", 2024-09-16T14:48:44.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6883e-05 2024-09-16T14:48:44.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907935+0000", 2024-09-16T14:48:44.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2357999999999999e-05 2024-09-16T14:48:44.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907944+0000", 2024-09-16T14:48:44.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9900000000000003e-06 2024-09-16T14:48:44.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.908005+0000", 2024-09-16T14:48:44.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0702000000000003e-05 2024-09-16T14:48:44.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.908014+0000", 2024-09-16T14:48:44.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4010000000000003e-06 2024-09-16T14:48:44.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909339+0000", 2024-09-16T14:48:44.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001325052 2024-09-16T14:48:44.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909366+0000", 2024-09-16T14:48:44.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6397000000000002e-05 2024-09-16T14:48:44.064 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.064 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.064 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.064 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 34/27 ECSubWrite(tid=80, reqid=osd.3.0:80, at_version=34'20, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:48:44.065 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:48:44.065 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1166369509999998, 2024-09-16T14:48:44.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055013639999999999, 2024-09-16T14:48:44.065 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.065 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.065 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:48:44.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:48:44.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909414+0000", 2024-09-16T14:48:44.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909442+0000", 2024-09-16T14:48:44.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7662999999999999e-05 2024-09-16T14:48:44.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909452+0000", 2024-09-16T14:48:44.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0773e-05 2024-09-16T14:48:44.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909463+0000", 2024-09-16T14:48:44.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0144e-05 2024-09-16T14:48:44.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909522+0000", 2024-09-16T14:48:44.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9651000000000002e-05 2024-09-16T14:48:44.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909531+0000", 2024-09-16T14:48:44.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8529999999999998e-06 2024-09-16T14:48:44.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914874+0000", 2024-09-16T14:48:44.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053432829999999999 2024-09-16T14:48:44.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914916+0000", 2024-09-16T14:48:44.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1928000000000002e-05 2024-09-16T14:48:44.069 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.069 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.069 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.070 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:48:44.070 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:48:44.070 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1165681410000001, 2024-09-16T14:48:44.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079855010000000008, 2024-09-16T14:48:44.070 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.070 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.070 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:48:44.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:48:44.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909483+0000", 2024-09-16T14:48:44.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909521+0000", 2024-09-16T14:48:44.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7185999999999999e-05 2024-09-16T14:48:44.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909543+0000", 2024-09-16T14:48:44.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2802999999999998e-05 2024-09-16T14:48:44.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909551+0000", 2024-09-16T14:48:44.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5039999999999996e-06 2024-09-16T14:48:44.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909927+0000", 2024-09-16T14:48:44.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037637400000000001 2024-09-16T14:48:44.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:48:44.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2040000000000002e-06 2024-09-16T14:48:44.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917442+0000", 2024-09-16T14:48:44.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007508659 2024-09-16T14:48:44.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917469+0000", 2024-09-16T14:48:44.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7197999999999999e-05 2024-09-16T14:48:44.074 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.074 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.074 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1124957950000001, 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067701159999999996, 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:44.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:44.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913579+0000", 2024-09-16T14:48:44.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2829000000000001e-05 2024-09-16T14:48:44.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913591+0000", 2024-09-16T14:48:44.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2264e-05 2024-09-16T14:48:44.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913600+0000", 2024-09-16T14:48:44.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9959999999999999e-06 2024-09-16T14:48:44.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913648+0000", 2024-09-16T14:48:44.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7759000000000001e-05 2024-09-16T14:48:44.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913657+0000", 2024-09-16T14:48:44.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6709999999999994e-06 2024-09-16T14:48:44.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920296+0000", 2024-09-16T14:48:44.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066385120000000001 2024-09-16T14:48:44.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.079 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.079 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920326+0000", 2024-09-16T14:48:44.079 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0405000000000001e-05 2024-09-16T14:48:44.079 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.079 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.079 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.079 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 34/27 ECSubWrite(tid=74, reqid=client.4500.0:229, at_version=34'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:48:44.079 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:44.079 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1116798370000001, 2024-09-16T14:48:44.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060056090000000003, 2024-09-16T14:48:44.080 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.080 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.080 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:44.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.080 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.080 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.080 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.080 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:44.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:44.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914468+0000", 2024-09-16T14:48:44.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6398999999999996e-05 2024-09-16T14:48:44.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914483+0000", 2024-09-16T14:48:44.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5238e-05 2024-09-16T14:48:44.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914494+0000", 2024-09-16T14:48:44.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0192e-05 2024-09-16T14:48:44.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914622+0000", 2024-09-16T14:48:44.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012855599999999999 2024-09-16T14:48:44.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914635+0000", 2024-09-16T14:48:44.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2510000000000001e-05 2024-09-16T14:48:44.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920339+0000", 2024-09-16T14:48:44.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057041549999999998 2024-09-16T14:48:44.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920378+0000", 2024-09-16T14:48:44.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8918999999999998e-05 2024-09-16T14:48:44.084 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.084 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.084 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.084 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:48:44.085 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:44.085 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1096670610000001, 2024-09-16T14:48:44.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069011319999999999, 2024-09-16T14:48:44.085 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.085 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.085 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:44.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:44.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:44.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:44.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916410+0000", 2024-09-16T14:48:44.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5343000000000001e-05 2024-09-16T14:48:44.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916418+0000", 2024-09-16T14:48:44.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2360000000000004e-06 2024-09-16T14:48:44.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916428+0000", 2024-09-16T14:48:44.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6779999999999995e-06 2024-09-16T14:48:44.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916492+0000", 2024-09-16T14:48:44.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4219000000000003e-05 2024-09-16T14:48:44.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916502+0000", 2024-09-16T14:48:44.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6690000000000001e-06 2024-09-16T14:48:44.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923254+0000", 2024-09-16T14:48:44.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067518070000000003 2024-09-16T14:48:44.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923286+0000", 2024-09-16T14:48:44.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2367999999999999e-05 2024-09-16T14:48:44.089 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.089 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.089 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.089 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 34/27 ECSubWrite(tid=82, reqid=osd.3.0:80, at_version=0'0, trim_to=0'0, roll_forward_to=34'20))", 2024-09-16T14:48:44.089 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.917072+0000", 2024-09-16T14:48:44.090 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1089799089999999, 2024-09-16T14:48:44.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0033776309999999999, 2024-09-16T14:48:44.090 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.090 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.090 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917072+0000", 2024-09-16T14:48:44.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917072+0000", 2024-09-16T14:48:44.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917072+0000", 2024-09-16T14:48:44.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917080+0000", 2024-09-16T14:48:44.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8349999999999993e-06 2024-09-16T14:48:44.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917089+0000", 2024-09-16T14:48:44.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5140000000000001e-06 2024-09-16T14:48:44.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917097+0000", 2024-09-16T14:48:44.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7749999999999993e-06 2024-09-16T14:48:44.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917213+0000", 2024-09-16T14:48:44.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011579200000000001 2024-09-16T14:48:44.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917237+0000", 2024-09-16T14:48:44.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4573e-05 2024-09-16T14:48:44.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920423+0000", 2024-09-16T14:48:44.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0031861580000000001 2024-09-16T14:48:44.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920450+0000", 2024-09-16T14:48:44.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6359e-05 2024-09-16T14:48:44.094 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.094 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.094 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.094 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 34/27 ECSubWrite(tid=83, reqid=client.4500.0:234, at_version=0'0, trim_to=0'0, roll_forward_to=34'22))", 2024-09-16T14:48:44.094 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.919388+0000", 2024-09-16T14:48:44.095 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.106663857, 2024-09-16T14:48:44.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0037609840000000002, 2024-09-16T14:48:44.095 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.095 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.095 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919388+0000", 2024-09-16T14:48:44.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919388+0000", 2024-09-16T14:48:44.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919387+0000", 2024-09-16T14:48:44.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919415+0000", 2024-09-16T14:48:44.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7571000000000001e-05 2024-09-16T14:48:44.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919434+0000", 2024-09-16T14:48:44.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9273999999999999e-05 2024-09-16T14:48:44.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919444+0000", 2024-09-16T14:48:44.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9879999999999999e-06 2024-09-16T14:48:44.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919495+0000", 2024-09-16T14:48:44.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0698999999999997e-05 2024-09-16T14:48:44.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.919506+0000", 2024-09-16T14:48:44.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.095e-05 2024-09-16T14:48:44.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923123+0000", 2024-09-16T14:48:44.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036172930000000002 2024-09-16T14:48:44.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923149+0000", 2024-09-16T14:48:44.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6035000000000001e-05 2024-09-16T14:48:44.099 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.099 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.099 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.099 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 34/27 ECSubWrite(tid=75, reqid=client.4500.0:229, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-16T14:48:44.099 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.920873+0000", 2024-09-16T14:48:44.099 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1051786469999998, 2024-09-16T14:48:44.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049917640000000001, 2024-09-16T14:48:44.100 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.100 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.100 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920873+0000", 2024-09-16T14:48:44.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920873+0000", 2024-09-16T14:48:44.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920872+0000", 2024-09-16T14:48:44.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920884+0000", 2024-09-16T14:48:44.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1501999999999999e-05 2024-09-16T14:48:44.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920898+0000", 2024-09-16T14:48:44.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4391e-05 2024-09-16T14:48:44.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.102 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.102 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920916+0000", 2024-09-16T14:48:44.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8057e-05 2024-09-16T14:48:44.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.102 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.102 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920965+0000", 2024-09-16T14:48:44.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8962000000000003e-05 2024-09-16T14:48:44.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.102 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920975+0000", 2024-09-16T14:48:44.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3989999999999993e-06 2024-09-16T14:48:44.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925839+0000", 2024-09-16T14:48:44.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048646330000000002 2024-09-16T14:48:44.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925865+0000", 2024-09-16T14:48:44.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5703000000000001e-05 2024-09-16T14:48:44.103 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.104 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.104 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.104 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/27 ECSubWrite(tid=169, reqid=osd.7.0:168, at_version=0'0, trim_to=0'0, roll_forward_to=34'8))", 2024-09-16T14:48:44.104 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.921046+0000", 2024-09-16T14:48:44.104 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1050059220000001, 2024-09-16T14:48:44.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048734829999999996, 2024-09-16T14:48:44.104 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.104 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.105 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921046+0000", 2024-09-16T14:48:44.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921046+0000", 2024-09-16T14:48:44.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921045+0000", 2024-09-16T14:48:44.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921056+0000", 2024-09-16T14:48:44.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0733e-05 2024-09-16T14:48:44.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921067+0000", 2024-09-16T14:48:44.106 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1048e-05 2024-09-16T14:48:44.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921076+0000", 2024-09-16T14:48:44.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:48:44.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921135+0000", 2024-09-16T14:48:44.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8681000000000003e-05 2024-09-16T14:48:44.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921144+0000", 2024-09-16T14:48:44.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9399999999999997e-06 2024-09-16T14:48:44.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925903+0000", 2024-09-16T14:48:44.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047588860000000004 2024-09-16T14:48:44.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925920+0000", 2024-09-16T14:48:44.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6147999999999999e-05 2024-09-16T14:48:44.108 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.109 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.109 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.109 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 34/27 ECSubWrite(tid=27, reqid=osd.1.0:26, at_version=0'0, trim_to=0'0, roll_forward_to=34'16))", 2024-09-16T14:48:44.109 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.923906+0000", 2024-09-16T14:48:44.109 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.1021461029999999, 2024-09-16T14:48:44.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002426344, 2024-09-16T14:48:44.109 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.110 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.110 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923906+0000", 2024-09-16T14:48:44.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923906+0000", 2024-09-16T14:48:44.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923905+0000", 2024-09-16T14:48:44.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923916+0000", 2024-09-16T14:48:44.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0655e-05 2024-09-16T14:48:44.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923933+0000", 2024-09-16T14:48:44.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7336000000000001e-05 2024-09-16T14:48:44.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923943+0000", 2024-09-16T14:48:44.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9210000000000004e-06 2024-09-16T14:48:44.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.924006+0000", 2024-09-16T14:48:44.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3260000000000001e-05 2024-09-16T14:48:44.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.924019+0000", 2024-09-16T14:48:44.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2542e-05 2024-09-16T14:48:44.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.926304+0000", 2024-09-16T14:48:44.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022853019999999999 2024-09-16T14:48:44.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.926332+0000", 2024-09-16T14:48:44.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8218e-05 2024-09-16T14:48:44.114 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.114 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.114 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.114 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as3 34/27 ECSubWrite(tid=77, reqid=client.4500.0:235, at_version=34'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:48:44.114 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.941590+0000", 2024-09-16T14:48:44.114 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.0844620489999999, 2024-09-16T14:48:44.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0022632279999999999, 2024-09-16T14:48:44.114 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.114 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.114 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941590+0000", 2024-09-16T14:48:44.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941590+0000", 2024-09-16T14:48:44.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941589+0000", 2024-09-16T14:48:44.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941637+0000", 2024-09-16T14:48:44.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7552000000000001e-05 2024-09-16T14:48:44.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941656+0000", 2024-09-16T14:48:44.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9627e-05 2024-09-16T14:48:44.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941666+0000", 2024-09-16T14:48:44.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9720000000000004e-06 2024-09-16T14:48:44.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941723+0000", 2024-09-16T14:48:44.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6439e-05 2024-09-16T14:48:44.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.941732+0000", 2024-09-16T14:48:44.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7159999999999998e-06 2024-09-16T14:48:44.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.943820+0000", 2024-09-16T14:48:44.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020879829999999999 2024-09-16T14:48:44.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.943853+0000", 2024-09-16T14:48:44.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2777999999999997e-05 2024-09-16T14:48:44.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.119 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.119 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.120 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.120 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.5s7 34/27 ECSubWrite(tid=35, reqid=client.4500.0:239, at_version=34'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:48:44.120 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.945632+0000", 2024-09-16T14:48:44.120 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.080420325, 2024-09-16T14:48:44.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0015030779999999999, 2024-09-16T14:48:44.121 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.121 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.121 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.121 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.945632+0000", 2024-09-16T14:48:44.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.945632+0000", 2024-09-16T14:48:44.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.945631+0000", 2024-09-16T14:48:44.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.945955+0000", 2024-09-16T14:48:44.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032387399999999998 2024-09-16T14:48:44.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.945987+0000", 2024-09-16T14:48:44.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1458000000000003e-05 2024-09-16T14:48:44.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.945999+0000", 2024-09-16T14:48:44.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2218000000000001e-05 2024-09-16T14:48:44.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.946062+0000", 2024-09-16T14:48:44.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2941999999999995e-05 2024-09-16T14:48:44.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.946068+0000", 2024-09-16T14:48:44.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2310000000000001e-06 2024-09-16T14:48:44.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.947110+0000", 2024-09-16T14:48:44.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001042142 2024-09-16T14:48:44.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.947135+0000", 2024-09-16T14:48:44.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4473000000000001e-05 2024-09-16T14:48:44.127 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.128 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.128 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.128 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.128 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.128 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:44.129 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:48:44.130 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:44.128+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:44.231 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:44.230+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:44.274 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:44.274 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:44.274 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:44.274 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:44.282 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:48:44.332 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:44.331+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:44.383 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:44.384 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:44.384 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:44.384 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:44.384 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:44.392 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:48:44.433 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:44.432+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:44.496 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:44.496 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:44.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:44.496 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:44.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.497 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2024-09-16T14:48:44.497 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.389034+0000", 2024-09-16T14:48:44.497 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.105171348999999, 2024-09-16T14:48:44.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066701460000000001, 2024-09-16T14:48:44.497 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.497 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:44.497 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389034+0000", 2024-09-16T14:48:44.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389034+0000", 2024-09-16T14:48:44.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389033+0000", 2024-09-16T14:48:44.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389110+0000", 2024-09-16T14:48:44.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6965000000000006e-05 2024-09-16T14:48:44.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389122+0000", 2024-09-16T14:48:44.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.166e-05 2024-09-16T14:48:44.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389129+0000", 2024-09-16T14:48:44.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8120000000000002e-06 2024-09-16T14:48:44.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389166+0000", 2024-09-16T14:48:44.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6986999999999997e-05 2024-09-16T14:48:44.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:44.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389186+0000", 2024-09-16T14:48:44.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0043000000000001e-05 2024-09-16T14:48:44.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:44.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395613+0000", 2024-09-16T14:48:44.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064270109999999998 2024-09-16T14:48:44.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395704+0000", 2024-09-16T14:48:44.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1595000000000006e-05 2024-09-16T14:48:44.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.501 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.501 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:22 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'12, mlcod=19'8)", 2024-09-16T14:48:44.501 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.389513+0000", 2024-09-16T14:48:44.501 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.104692958000001, 2024-09-16T14:48:44.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062632759999999999, 2024-09-16T14:48:44.501 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.502 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:44.502 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389513+0000", 2024-09-16T14:48:44.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389513+0000", 2024-09-16T14:48:44.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389512+0000", 2024-09-16T14:48:44.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389520+0000", 2024-09-16T14:48:44.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2609999999999993e-06 2024-09-16T14:48:44.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389533+0000", 2024-09-16T14:48:44.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2860999999999999e-05 2024-09-16T14:48:44.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389539+0000", 2024-09-16T14:48:44.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2369999999999997e-06 2024-09-16T14:48:44.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389568+0000", 2024-09-16T14:48:44.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9411999999999999e-05 2024-09-16T14:48:44.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:44.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389584+0000", 2024-09-16T14:48:44.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5891000000000001e-05 2024-09-16T14:48:44.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:44.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395712+0000", 2024-09-16T14:48:44.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061272360000000003 2024-09-16T14:48:44.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395776+0000", 2024-09-16T14:48:44.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4290999999999998e-05 2024-09-16T14:48:44.506 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.506 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.506 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.506 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:32 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'18, mlcod=19'14)", 2024-09-16T14:48:44.506 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.407675+0000", 2024-09-16T14:48:44.506 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.086530117999999, 2024-09-16T14:48:44.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0056403440000000003, 2024-09-16T14:48:44.506 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.506 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:44.506 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407675+0000", 2024-09-16T14:48:44.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407675+0000", 2024-09-16T14:48:44.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407675+0000", 2024-09-16T14:48:44.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407750+0000", 2024-09-16T14:48:44.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4728999999999996e-05 2024-09-16T14:48:44.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407759+0000", 2024-09-16T14:48:44.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3519999999999996e-06 2024-09-16T14:48:44.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407764+0000", 2024-09-16T14:48:44.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.994e-06 2024-09-16T14:48:44.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407797+0000", 2024-09-16T14:48:44.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2641999999999997e-05 2024-09-16T14:48:44.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:44.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.407816+0000", 2024-09-16T14:48:44.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9219e-05 2024-09-16T14:48:44.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:44.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413272+0000", 2024-09-16T14:48:44.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054566129999999999 2024-09-16T14:48:44.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413316+0000", 2024-09-16T14:48:44.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3350999999999999e-05 2024-09-16T14:48:44.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.510 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.511 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:33 1.0 e19/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'19, mlcod=19'14)", 2024-09-16T14:48:44.511 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.408061+0000", 2024-09-16T14:48:44.511 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.086144058999999, 2024-09-16T14:48:44.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052755520000000002, 2024-09-16T14:48:44.511 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.511 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:44.511 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408061+0000", 2024-09-16T14:48:44.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408061+0000", 2024-09-16T14:48:44.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408061+0000", 2024-09-16T14:48:44.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408066+0000", 2024-09-16T14:48:44.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6210000000000001e-06 2024-09-16T14:48:44.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408077+0000", 2024-09-16T14:48:44.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0906e-05 2024-09-16T14:48:44.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408081+0000", 2024-09-16T14:48:44.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8689999999999997e-06 2024-09-16T14:48:44.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408114+0000", 2024-09-16T14:48:44.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2795e-05 2024-09-16T14:48:44.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:44.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.408133+0000", 2024-09-16T14:48:44.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9017000000000001e-05 2024-09-16T14:48:44.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:44.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413318+0000", 2024-09-16T14:48:44.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051848040000000003 2024-09-16T14:48:44.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.413337+0000", 2024-09-16T14:48:44.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9173000000000001e-05 2024-09-16T14:48:44.515 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.515 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.515 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.515 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:158 1.0 e20/18 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 20'112, mlcod=20'107)", 2024-09-16T14:48:44.515 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:23.394419+0000", 2024-09-16T14:48:44.515 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 21.099786262999999, 2024-09-16T14:48:44.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054479120000000001, 2024-09-16T14:48:44.515 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.516 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:44.516 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394419+0000", 2024-09-16T14:48:44.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394419+0000", 2024-09-16T14:48:44.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394419+0000", 2024-09-16T14:48:44.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394481+0000", 2024-09-16T14:48:44.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2515999999999996e-05 2024-09-16T14:48:44.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394489+0000", 2024-09-16T14:48:44.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0059999999999992e-06 2024-09-16T14:48:44.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394495+0000", 2024-09-16T14:48:44.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3070000000000002e-06 2024-09-16T14:48:44.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394518+0000", 2024-09-16T14:48:44.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3818000000000001e-05 2024-09-16T14:48:44.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:44.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.394530+0000", 2024-09-16T14:48:44.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1158999999999999e-05 2024-09-16T14:48:44.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:44.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399835+0000", 2024-09-16T14:48:44.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053054829999999997 2024-09-16T14:48:44.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:23.399867+0000", 2024-09-16T14:48:44.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2141000000000003e-05 2024-09-16T14:48:44.520 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.520 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.520 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.520 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:213 3.8s0 3:145ad69c:::smithi04040151-6:head [read 0~983040,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-16T14:48:44.520 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.891233+0000", 2024-09-16T14:48:44.520 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.6029721610000003, 2024-09-16T14:48:44.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053188610000000002, 2024-09-16T14:48:44.521 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.521 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:44.521 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:44.521 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:44.521 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:44.521 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 213 2024-09-16T14:48:44.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.521 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891233+0000", 2024-09-16T14:48:44.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891233+0000", 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891233+0000", 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891235+0000", 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2589999999999999e-06 2024-09-16T14:48:44.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891239+0000", 2024-09-16T14:48:44.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8340000000000001e-06 2024-09-16T14:48:44.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891242+0000", 2024-09-16T14:48:44.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.841e-06 2024-09-16T14:48:44.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891330+0000", 2024-09-16T14:48:44.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8253999999999999e-05 2024-09-16T14:48:44.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:48:44.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.891338+0000", 2024-09-16T14:48:44.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.233e-06 2024-09-16T14:48:44.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.893273+0000", 2024-09-16T14:48:44.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001935223 2024-09-16T14:48:44.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:44.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.893313+0000", 2024-09-16T14:48:44.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9824999999999999e-05 2024-09-16T14:48:44.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.896552+0000", 2024-09-16T14:48:44.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032396730000000002 2024-09-16T14:48:44.525 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.525 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.525 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.526 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:229 3.es0 3:7ecfd044:::smithi04040151-35:head [copy-from ver 4] snapc 4=[4,2] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:48:44.526 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:48:44.526 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.5900358109999999, 2024-09-16T14:48:44.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016506461, 2024-09-16T14:48:44.526 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.526 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:44.526 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:44.526 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:44.526 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:44.526 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 229 2024-09-16T14:48:44.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.526 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:48:44.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:48:44.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904168+0000", 2024-09-16T14:48:44.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:48:44.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904187+0000", 2024-09-16T14:48:44.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9449999999999998e-05 2024-09-16T14:48:44.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904206+0000", 2024-09-16T14:48:44.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8698e-05 2024-09-16T14:48:44.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904215+0000", 2024-09-16T14:48:44.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:48:44.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904276+0000", 2024-09-16T14:48:44.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1130999999999995e-05 2024-09-16T14:48:44.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:48:44.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904284+0000", 2024-09-16T14:48:44.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0609999999999996e-06 2024-09-16T14:48:44.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906272+0000", 2024-09-16T14:48:44.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019884210000000002 2024-09-16T14:48:44.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:44.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906391+0000", 2024-09-16T14:48:44.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000118843 2024-09-16T14:48:44.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.912595+0000", 2024-09-16T14:48:44.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062038190000000002 2024-09-16T14:48:44.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917584+0000", 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049889069999999999 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920592+0000", 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030081639999999998 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920676+0000", 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3974999999999995e-05 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.531 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:230 3.es0 3:7ecfd044:::smithi04040151-35:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:48:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:48:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.5899743280000003, 2024-09-16T14:48:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0096834569999999995, 2024-09-16T14:48:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 230 2024-09-16T14:48:44.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.533 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:48:44.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:48:44.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:48:44.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904235+0000", 2024-09-16T14:48:44.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4059999999999998e-06 2024-09-16T14:48:44.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904244+0000", 2024-09-16T14:48:44.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-16T14:48:44.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904250+0000", 2024-09-16T14:48:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0719999999999996e-06 2024-09-16T14:48:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904338+0000", 2024-09-16T14:48:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7923000000000005e-05 2024-09-16T14:48:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:48:44.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904344+0000", 2024-09-16T14:48:44.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8669999999999999e-06 2024-09-16T14:48:44.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906575+0000", 2024-09-16T14:48:44.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002230994 2024-09-16T14:48:44.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:48:44.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906632+0000", 2024-09-16T14:48:44.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7357e-05 2024-09-16T14:48:44.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913716+0000", 2024-09-16T14:48:44.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070835070000000002 2024-09-16T14:48:44.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:44.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913822+0000", 2024-09-16T14:48:44.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010566299999999999 2024-09-16T14:48:44.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913915+0000", 2024-09-16T14:48:44.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3018999999999998e-05 2024-09-16T14:48:44.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.538 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.538 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s1 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:48:44.538 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.910113+0000", 2024-09-16T14:48:44.538 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.58409216, 2024-09-16T14:48:44.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051394459999999998, 2024-09-16T14:48:44.538 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.538 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.538 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910113+0000", 2024-09-16T14:48:44.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910113+0000", 2024-09-16T14:48:44.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910113+0000", 2024-09-16T14:48:44.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:44.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910129+0000", 2024-09-16T14:48:44.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6101000000000001e-05 2024-09-16T14:48:44.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910139+0000", 2024-09-16T14:48:44.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0105999999999999e-05 2024-09-16T14:48:44.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910146+0000", 2024-09-16T14:48:44.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2960000000000004e-06 2024-09-16T14:48:44.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910191+0000", 2024-09-16T14:48:44.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5593000000000002e-05 2024-09-16T14:48:44.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910198+0000", 2024-09-16T14:48:44.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3849999999999999e-06 2024-09-16T14:48:44.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915218+0000", 2024-09-16T14:48:44.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050206499999999998 2024-09-16T14:48:44.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915253+0000", 2024-09-16T14:48:44.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4433e-05 2024-09-16T14:48:44.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.543 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.543 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs1 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:48:44.543 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.913964+0000", 2024-09-16T14:48:44.543 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.5802414759999999, 2024-09-16T14:48:44.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063214940000000004, 2024-09-16T14:48:44.543 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.543 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.543 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913964+0000", 2024-09-16T14:48:44.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913964+0000", 2024-09-16T14:48:44.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913964+0000", 2024-09-16T14:48:44.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:44.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913996+0000", 2024-09-16T14:48:44.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2218000000000002e-05 2024-09-16T14:48:44.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914007+0000", 2024-09-16T14:48:44.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1216999999999999e-05 2024-09-16T14:48:44.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914018+0000", 2024-09-16T14:48:44.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0902e-05 2024-09-16T14:48:44.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914075+0000", 2024-09-16T14:48:44.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6864999999999999e-05 2024-09-16T14:48:44.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914084+0000", 2024-09-16T14:48:44.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1540000000000008e-06 2024-09-16T14:48:44.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920256+0000", 2024-09-16T14:48:44.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061714339999999999 2024-09-16T14:48:44.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920286+0000", 2024-09-16T14:48:44.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9910999999999999e-05 2024-09-16T14:48:44.548 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.548 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.548 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.548 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s1 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:48:44.548 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916482+0000", 2024-09-16T14:48:44.548 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.5777235579999997, 2024-09-16T14:48:44.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069103619999999998, 2024-09-16T14:48:44.548 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.548 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.549 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916482+0000", 2024-09-16T14:48:44.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916482+0000", 2024-09-16T14:48:44.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916482+0000", 2024-09-16T14:48:44.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916501+0000", 2024-09-16T14:48:44.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9494999999999999e-05 2024-09-16T14:48:44.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916511+0000", 2024-09-16T14:48:44.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4269999999999997e-06 2024-09-16T14:48:44.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916518+0000", 2024-09-16T14:48:44.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0310000000000001e-06 2024-09-16T14:48:44.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916572+0000", 2024-09-16T14:48:44.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4691000000000002e-05 2024-09-16T14:48:44.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916583+0000", 2024-09-16T14:48:44.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.039e-05 2024-09-16T14:48:44.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923344+0000", 2024-09-16T14:48:44.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067608720000000002 2024-09-16T14:48:44.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923392+0000", 2024-09-16T14:48:44.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8779999999999997e-05 2024-09-16T14:48:44.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.552 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.553 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:237 3.as0 3:53e83d93:::smithi04040151-22:4 [read 0~360448,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:48:44.553 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.940246+0000", 2024-09-16T14:48:44.553 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.5539592739999999, 2024-09-16T14:48:44.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055339860000000003, 2024-09-16T14:48:44.553 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.553 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:44.553 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:44.553 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:44.553 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:44.553 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 237 2024-09-16T14:48:44.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.554 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940246+0000", 2024-09-16T14:48:44.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940246+0000", 2024-09-16T14:48:44.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940246+0000", 2024-09-16T14:48:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940248+0000", 2024-09-16T14:48:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2249999999999999e-06 2024-09-16T14:48:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940253+0000", 2024-09-16T14:48:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9660000000000004e-06 2024-09-16T14:48:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940256+0000", 2024-09-16T14:48:44.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9110000000000002e-06 2024-09-16T14:48:44.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.943337+0000", 2024-09-16T14:48:44.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003080756 2024-09-16T14:48:44.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:44.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.943432+0000", 2024-09-16T14:48:44.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5610000000000001e-05 2024-09-16T14:48:44.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.945780+0000", 2024-09-16T14:48:44.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002347769 2024-09-16T14:48:44.557 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.557 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.557 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.557 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:238 3.as0 3:53e83d93:::smithi04040151-22:4 [read 0~360448,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:48:44.557 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.940260+0000", 2024-09-16T14:48:44.557 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.553945981, 2024-09-16T14:48:44.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068824810000000002, 2024-09-16T14:48:44.558 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.558 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:44.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:44.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:44.558 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:44.558 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 238 2024-09-16T14:48:44.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.558 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940260+0000", 2024-09-16T14:48:44.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940260+0000", 2024-09-16T14:48:44.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940259+0000", 2024-09-16T14:48:44.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:44.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940261+0000", 2024-09-16T14:48:44.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.717e-06 2024-09-16T14:48:44.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940264+0000", 2024-09-16T14:48:44.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2349999999999999e-06 2024-09-16T14:48:44.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940267+0000", 2024-09-16T14:48:44.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-06 2024-09-16T14:48:44.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.946049+0000", 2024-09-16T14:48:44.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057822469999999999 2024-09-16T14:48:44.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:44.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.946135+0000", 2024-09-16T14:48:44.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5871000000000005e-05 2024-09-16T14:48:44.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.947142+0000", 2024-09-16T14:48:44.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010067699999999999 2024-09-16T14:48:44.561 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.561 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.562 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.562 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs4 37/27 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:44.562 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:48:44.562 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.57343751700000001, 2024-09-16T14:48:44.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0098104279999999995, 2024-09-16T14:48:44.562 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.562 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.562 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:48:44.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:48:44.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:48:44.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920785+0000", 2024-09-16T14:48:44.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7218e-05 2024-09-16T14:48:44.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920793+0000", 2024-09-16T14:48:44.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9300000000000003e-06 2024-09-16T14:48:44.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920800+0000", 2024-09-16T14:48:44.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1860000000000003e-06 2024-09-16T14:48:44.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923692+0000", 2024-09-16T14:48:44.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028921329999999999 2024-09-16T14:48:44.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923701+0000", 2024-09-16T14:48:44.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6109999999999994e-06 2024-09-16T14:48:44.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930554+0000", 2024-09-16T14:48:44.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068534019999999998 2024-09-16T14:48:44.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930578+0000", 2024-09-16T14:48:44.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4232000000000001e-05 2024-09-16T14:48:44.566 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.566 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.566 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.567 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s4 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:44.567 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923896+0000", 2024-09-16T14:48:44.567 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.57030942699999998, 2024-09-16T14:48:44.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006707055, 2024-09-16T14:48:44.567 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.567 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.567 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923896+0000", 2024-09-16T14:48:44.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923896+0000", 2024-09-16T14:48:44.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923896+0000", 2024-09-16T14:48:44.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923920+0000", 2024-09-16T14:48:44.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3713000000000001e-05 2024-09-16T14:48:44.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923928+0000", 2024-09-16T14:48:44.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6810000000000004e-06 2024-09-16T14:48:44.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923936+0000", 2024-09-16T14:48:44.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.537e-06 2024-09-16T14:48:44.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923983+0000", 2024-09-16T14:48:44.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7686999999999999e-05 2024-09-16T14:48:44.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923993+0000", 2024-09-16T14:48:44.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3619999999999995e-06 2024-09-16T14:48:44.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930584+0000", 2024-09-16T14:48:44.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065914190000000003 2024-09-16T14:48:44.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930603+0000", 2024-09-16T14:48:44.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8936999999999998e-05 2024-09-16T14:48:44.572 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.572 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.572 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.572 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s1 37/27 ECSubWrite(tid=33, reqid=osd.1.0:33, at_version=37'24, trim_to=0'0, roll_forward_to=35'22))", 2024-09-16T14:48:44.572 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924967+0000", 2024-09-16T14:48:44.573 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.56923848600000004, 2024-09-16T14:48:44.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005671316, 2024-09-16T14:48:44.573 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.573 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.573 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924967+0000", 2024-09-16T14:48:44.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924967+0000", 2024-09-16T14:48:44.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924967+0000", 2024-09-16T14:48:44.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:44.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924990+0000", 2024-09-16T14:48:44.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3068e-05 2024-09-16T14:48:44.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925000+0000", 2024-09-16T14:48:44.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9000000000000001e-06 2024-09-16T14:48:44.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925007+0000", 2024-09-16T14:48:44.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2130000000000002e-06 2024-09-16T14:48:44.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925077+0000", 2024-09-16T14:48:44.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9980000000000004e-05 2024-09-16T14:48:44.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925086+0000", 2024-09-16T14:48:44.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.755e-06 2024-09-16T14:48:44.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930615+0000", 2024-09-16T14:48:44.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055289229999999998 2024-09-16T14:48:44.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930638+0000", 2024-09-16T14:48:44.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3680999999999999e-05 2024-09-16T14:48:44.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.577 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.578 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s1 37/27 ECSubWrite(tid=119, reqid=osd.3.0:119, at_version=37'39, trim_to=0'0, roll_forward_to=36'37))", 2024-09-16T14:48:44.578 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.933173+0000", 2024-09-16T14:48:44.578 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.56103283699999995, 2024-09-16T14:48:44.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006508443, 2024-09-16T14:48:44.578 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.578 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.578 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933173+0000", 2024-09-16T14:48:44.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933173+0000", 2024-09-16T14:48:44.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933172+0000", 2024-09-16T14:48:44.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933191+0000", 2024-09-16T14:48:44.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8766999999999998e-05 2024-09-16T14:48:44.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933202+0000", 2024-09-16T14:48:44.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1337e-05 2024-09-16T14:48:44.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933210+0000", 2024-09-16T14:48:44.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1049999999999995e-06 2024-09-16T14:48:44.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933258+0000", 2024-09-16T14:48:44.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7753000000000002e-05 2024-09-16T14:48:44.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933268+0000", 2024-09-16T14:48:44.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5009999999999993e-06 2024-09-16T14:48:44.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.939661+0000", 2024-09-16T14:48:44.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063929349999999998 2024-09-16T14:48:44.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.939681+0000", 2024-09-16T14:48:44.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0618000000000001e-05 2024-09-16T14:48:44.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.582 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.582 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s4 37/27 ECSubWrite(tid=17, reqid=osd.4.0:16, at_version=0'0, trim_to=0'0, roll_forward_to=37'14))", 2024-09-16T14:48:44.582 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.933901+0000", 2024-09-16T14:48:44.583 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.56030446099999998, 2024-09-16T14:48:44.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0058206229999999996, 2024-09-16T14:48:44.583 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.583 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.583 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933901+0000", 2024-09-16T14:48:44.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933901+0000", 2024-09-16T14:48:44.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933901+0000", 2024-09-16T14:48:44.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933909+0000", 2024-09-16T14:48:44.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4330000000000004e-06 2024-09-16T14:48:44.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933917+0000", 2024-09-16T14:48:44.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3219999999999993e-06 2024-09-16T14:48:44.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933926+0000", 2024-09-16T14:48:44.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0909999999999993e-06 2024-09-16T14:48:44.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933971+0000", 2024-09-16T14:48:44.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5429000000000002e-05 2024-09-16T14:48:44.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933986+0000", 2024-09-16T14:48:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4912999999999999e-05 2024-09-16T14:48:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.939696+0000", 2024-09-16T14:48:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057099509999999996 2024-09-16T14:48:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.939722+0000", 2024-09-16T14:48:44.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5826999999999999e-05 2024-09-16T14:48:44.587 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.587 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.587 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.587 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s1 37/27 ECSubWrite(tid=120, reqid=osd.3.0:120, at_version=37'41, trim_to=0'0, roll_forward_to=36'37))", 2024-09-16T14:48:44.587 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.934770+0000", 2024-09-16T14:48:44.587 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.55943505800000004, 2024-09-16T14:48:44.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0076309480000000002, 2024-09-16T14:48:44.587 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.588 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.588 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934770+0000", 2024-09-16T14:48:44.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934770+0000", 2024-09-16T14:48:44.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934770+0000", 2024-09-16T14:48:44.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934786+0000", 2024-09-16T14:48:44.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6401000000000001e-05 2024-09-16T14:48:44.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934796+0000", 2024-09-16T14:48:44.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0267e-05 2024-09-16T14:48:44.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934804+0000", 2024-09-16T14:48:44.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1040000000000007e-06 2024-09-16T14:48:44.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934848+0000", 2024-09-16T14:48:44.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4070000000000003e-05 2024-09-16T14:48:44.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934858+0000", 2024-09-16T14:48:44.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7079999999999992e-06 2024-09-16T14:48:44.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942379+0000", 2024-09-16T14:48:44.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0075206379999999996 2024-09-16T14:48:44.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942401+0000", 2024-09-16T14:48:44.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2597999999999999e-05 2024-09-16T14:48:44.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.592 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.592 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s1 37/27 ECSubWrite(tid=125, reqid=osd.3.0:125, at_version=37'43, trim_to=0'0, roll_forward_to=37'41))", 2024-09-16T14:48:44.592 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.943670+0000", 2024-09-16T14:48:44.592 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.55053545299999995, 2024-09-16T14:48:44.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053773099999999997, 2024-09-16T14:48:44.593 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:44.593 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.593 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:44.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:44.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943670+0000", 2024-09-16T14:48:44.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:44.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943670+0000", 2024-09-16T14:48:44.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:44.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:44.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943669+0000", 2024-09-16T14:48:44.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:44.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943690+0000", 2024-09-16T14:48:44.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0744e-05 2024-09-16T14:48:44.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:44.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943701+0000", 2024-09-16T14:48:44.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1408e-05 2024-09-16T14:48:44.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943709+0000", 2024-09-16T14:48:44.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6520000000000006e-06 2024-09-16T14:48:44.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:44.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943772+0000", 2024-09-16T14:48:44.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2464000000000004e-05 2024-09-16T14:48:44.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943780+0000", 2024-09-16T14:48:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5089999999999993e-06 2024-09-16T14:48:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949027+0000", 2024-09-16T14:48:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052465180000000004 2024-09-16T14:48:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949047+0000", 2024-09-16T14:48:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0767999999999998e-05 2024-09-16T14:48:44.596 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.597 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.597 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.597 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:44.597 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:44.597 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:44.598 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:44.597+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:44.599 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:48:44.699 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:44.698+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:44.718 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:48:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:48:44.718 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:48:44.719 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:48:44.728 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:48:44.800 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:44.799+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:44.834 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:48:44.834 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:48:44.835 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:48:44.835 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:48:44.835 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:48:44.843 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:48:44.901 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:44.900+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:44.923 INFO:tasks.rados.rados.0.smithi040.stdout:115: done (1 left) 2024-09-16T14:48:44.923 INFO:tasks.rados.rados.0.smithi040.stdout:116: rollback oid 5 current snap is 4 2024-09-16T14:48:44.923 INFO:tasks.rados.rados.0.smithi040.stdout:116: done (1 left) 2024-09-16T14:48:44.923 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 35 current snap is 4 2024-09-16T14:48:44.923 INFO:tasks.rados.rados.0.smithi040.stdout:114: finishing copy_from to smithi04040151-29 2024-09-16T14:48:44.923 INFO:tasks.rados.rados.0.smithi040.stdout:114: got expected ENOENT (src dne) 2024-09-16T14:48:44.923 INFO:tasks.rados.rados.0.smithi040.stdout:117: seq_num 65 ranges {360448=720896,1081344=376832} 2024-09-16T14:48:44.928 INFO:tasks.rados.rados.0.smithi040.stdout:117: writing smithi04040151-35 from 360448 to 1081344 tid 1 2024-09-16T14:48:44.931 INFO:tasks.rados.rados.0.smithi040.stdout:117: writing smithi04040151-35 from 1081344 to 1458176 tid 2 2024-09-16T14:48:44.931 INFO:tasks.rados.rados.0.smithi040.stdout:118: setattr oid 3 current snap is 4 2024-09-16T14:48:44.931 INFO:tasks.rados.rados.0.smithi040.stdout:114: finishing copy_from racing read to smithi04040151-29 2024-09-16T14:48:44.933 INFO:tasks.rados.rados.0.smithi040.stdout:114: done (2 left) 2024-09-16T14:48:44.933 INFO:tasks.rados.rados.0.smithi040.stdout:119: read oid 21 snap -1 2024-09-16T14:48:44.933 INFO:tasks.rados.rados.0.smithi040.stdout:119: expect (ObjNum 20 snap 0 seq_num 20) 2024-09-16T14:48:44.933 INFO:tasks.rados.rados.0.smithi040.stdout:120: read oid 25 snap -1 2024-09-16T14:48:44.933 INFO:tasks.rados.rados.0.smithi040.stdout:120: expect deleted 2024-09-16T14:48:44.933 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 19 current snap is 4 2024-09-16T14:48:44.933 INFO:tasks.rados.rados.0.smithi040.stdout:121: seq_num 66 ranges {311296=737280,1048576=376832} 2024-09-16T14:48:44.939 INFO:tasks.rados.rados.0.smithi040.stdout:121: writing smithi04040151-19 from 311296 to 1048576 tid 1 2024-09-16T14:48:44.943 INFO:tasks.rados.rados.0.smithi040.stdout:121: writing smithi04040151-19 from 1048576 to 1425408 tid 2 2024-09-16T14:48:44.943 INFO:tasks.rados.rados.0.smithi040.stdout:122: read oid 11 snap -1 2024-09-16T14:48:44.943 INFO:tasks.rados.rados.0.smithi040.stdout:122: expect deleted 2024-09-16T14:48:44.943 INFO:tasks.rados.rados.0.smithi040.stdout:123: rmattr oid 24 current snap is 4 2024-09-16T14:48:44.943 INFO:tasks.rados.rados.0.smithi040.stdout:123: done (6 left) 2024-09-16T14:48:44.943 INFO:tasks.rados.rados.0.smithi040.stdout:124: rollback oid 47 current snap is 4 2024-09-16T14:48:44.943 INFO:tasks.rados.rados.0.smithi040.stdout:124: done (6 left) 2024-09-16T14:48:44.943 INFO:tasks.rados.rados.0.smithi040.stdout:125: rollback oid 29 current snap is 4 2024-09-16T14:48:44.943 INFO:tasks.rados.rados.0.smithi040.stdout:125: done (6 left) 2024-09-16T14:48:44.943 INFO:tasks.rados.rados.0.smithi040.stdout:126: copy_from oid 18 from oid 10 current snap is 4 2024-09-16T14:48:44.943 INFO:tasks.rados.rados.0.smithi040.stdout:127: read oid 26 snap -1 2024-09-16T14:48:44.943 INFO:tasks.rados.rados.0.smithi040.stdout:127: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-16T14:48:44.944 INFO:tasks.rados.rados.0.smithi040.stdout:128: setattr oid 42 current snap is 4 2024-09-16T14:48:44.944 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 3 v 25 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-16T14:48:44.944 INFO:tasks.rados.rados.0.smithi040.stdout:117: finishing write tid 1 to smithi04040151-35 2024-09-16T14:48:44.944 INFO:tasks.rados.rados.0.smithi040.stdout:117: finishing write tid 2 to smithi04040151-35 2024-09-16T14:48:44.944 INFO:tasks.rados.rados.0.smithi040.stdout:117: finishing write tid 3 to smithi04040151-35 2024-09-16T14:48:44.945 INFO:tasks.rados.rados.0.smithi040.stdout:118: done (8 left) 2024-09-16T14:48:44.945 INFO:tasks.rados.rados.0.smithi040.stdout:120: done (7 left) 2024-09-16T14:48:44.945 INFO:tasks.rados.rados.0.smithi040.stdout:129: delete oid 22 current snap is 4 2024-09-16T14:48:44.946 INFO:tasks.rados.rados.0.smithi040.stdout:117: finishing write tid 4 to smithi04040151-35 2024-09-16T14:48:44.946 INFO:tasks.rados.rados.0.smithi040.stdout:117: oid 35 updating version 0 to 13 2024-09-16T14:48:44.946 INFO:tasks.rados.rados.0.smithi040.stdout:117: oid 35 version 13 is already newer than 12 2024-09-16T14:48:44.946 INFO:tasks.rados.rados.0.smithi040.stdout:117: oid 35 updating version 13 to 14 2024-09-16T14:48:44.946 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 35 v 14 (ObjNum 65 snap 4 seq_num 65) dirty exists 2024-09-16T14:48:44.947 INFO:tasks.rados.rados.0.smithi040.stdout:117: left oid 35 (ObjNum 65 snap 4 seq_num 65) 2024-09-16T14:48:44.948 INFO:tasks.rados.rados.0.smithi040.stdout:127: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-16T14:48:44.950 INFO:tasks.rados.rados.0.smithi040.stdout:119: expect (ObjNum 20 snap 0 seq_num 20) 2024-09-16T14:48:44.960 INFO:tasks.rados.rados.0.smithi040.stdout:121: finishing write tid 1 to smithi04040151-19 2024-09-16T14:48:44.961 INFO:tasks.rados.rados.0.smithi040.stdout:121: finishing write tid 2 to smithi04040151-19 2024-09-16T14:48:44.961 INFO:tasks.rados.rados.0.smithi040.stdout:117: done (7 left) 2024-09-16T14:48:44.961 INFO:tasks.rados.rados.0.smithi040.stdout:119: done (6 left) 2024-09-16T14:48:44.961 INFO:tasks.rados.rados.0.smithi040.stdout:122: done (5 left) 2024-09-16T14:48:44.961 INFO:tasks.rados.rados.0.smithi040.stdout:127: done (4 left) 2024-09-16T14:48:44.961 INFO:tasks.rados.rados.0.smithi040.stdout:129: done (3 left) 2024-09-16T14:48:44.961 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 41 current snap is 4 2024-09-16T14:48:44.961 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 42 v 26 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-16T14:48:44.961 INFO:tasks.rados.rados.0.smithi040.stdout:121: finishing write tid 3 to smithi04040151-19 2024-09-16T14:48:44.961 INFO:tasks.rados.rados.0.smithi040.stdout:130: seq_num 68 ranges {688128=770048,1458176=376832} 2024-09-16T14:48:44.963 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:48:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:48:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:48:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:48:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 32/27 ECSubWrite(tid=22, reqid=client.4500.0:215, at_version=32'13, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:38.901735+0000", 2024-09-16T14:48:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 6.0600718010000003, 2024-09-16T14:48:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0023292809999999999, 2024-09-16T14:48:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901735+0000", 2024-09-16T14:48:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901735+0000", 2024-09-16T14:48:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901734+0000", 2024-09-16T14:48:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901843+0000", 2024-09-16T14:48:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000109413 2024-09-16T14:48:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901862+0000", 2024-09-16T14:48:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8964999999999999e-05 2024-09-16T14:48:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901871+0000", 2024-09-16T14:48:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7309999999999999e-06 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901930+0000", 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8343e-05 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.901939+0000", 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6050000000000004e-06 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.904040+0000", 2024-09-16T14:48:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0021013569999999999 2024-09-16T14:48:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.904064+0000", 2024-09-16T14:48:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3842e-05 2024-09-16T14:48:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs6 37/27 ECSubWrite(tid=187, reqid=osd.7.0:187, at_version=37'12, trim_to=0'0, roll_forward_to=35'10))", 2024-09-16T14:48:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.915624+0000", 2024-09-16T14:48:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.0461828179999999, 2024-09-16T14:48:44.969 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0034771300000000002, 2024-09-16T14:48:44.969 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:44.969 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.969 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:44.969 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.969 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:44.969 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915624+0000", 2024-09-16T14:48:44.969 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.969 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.969 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:44.969 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915624+0000", 2024-09-16T14:48:44.969 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915624+0000", 2024-09-16T14:48:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915644+0000", 2024-09-16T14:48:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0636e-05 2024-09-16T14:48:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915721+0000", 2024-09-16T14:48:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.6613000000000007e-05 2024-09-16T14:48:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915735+0000", 2024-09-16T14:48:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3696000000000001e-05 2024-09-16T14:48:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915848+0000", 2024-09-16T14:48:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000113393 2024-09-16T14:48:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915858+0000", 2024-09-16T14:48:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0071000000000001e-05 2024-09-16T14:48:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.919086+0000", 2024-09-16T14:48:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0032279209999999999 2024-09-16T14:48:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.919101+0000", 2024-09-16T14:48:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5177999999999999e-05 2024-09-16T14:48:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.973 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.973 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 37/27 ECSubWrite(tid=188, reqid=osd.7.0:188, at_version=37'20, trim_to=0'0, roll_forward_to=35'18))", 2024-09-16T14:48:44.973 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.915787+0000", 2024-09-16T14:48:44.973 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.0460196159999999, 2024-09-16T14:48:44.973 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0033016009999999999, 2024-09-16T14:48:44.973 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:44.973 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.973 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:44.973 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.973 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:44.973 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915787+0000", 2024-09-16T14:48:44.973 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.974 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.974 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.974 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:44.974 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915787+0000", 2024-09-16T14:48:44.974 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.974 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.974 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.974 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:44.974 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915787+0000", 2024-09-16T14:48:44.974 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.974 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.974 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.974 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:44.974 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915860+0000", 2024-09-16T14:48:44.975 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3461999999999996e-05 2024-09-16T14:48:44.975 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.975 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.975 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:44.975 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915870+0000", 2024-09-16T14:48:44.975 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0370000000000001e-05 2024-09-16T14:48:44.975 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.975 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.975 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.975 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915879+0000", 2024-09-16T14:48:44.975 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.935e-06 2024-09-16T14:48:44.975 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.975 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.975 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:44.976 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915979+0000", 2024-09-16T14:48:44.976 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9277000000000005e-05 2024-09-16T14:48:44.976 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.976 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.976 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.915988+0000", 2024-09-16T14:48:44.976 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6930000000000002e-06 2024-09-16T14:48:44.976 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.976 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.976 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.919062+0000", 2024-09-16T14:48:44.976 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.003073573 2024-09-16T14:48:44.976 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.977 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:44.977 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.919089+0000", 2024-09-16T14:48:44.977 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6984000000000001e-05 2024-09-16T14:48:44.977 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.977 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:44.977 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.977 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.977 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.977 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 37/27 ECSubWrite(tid=33, reqid=osd.1.0:33, at_version=37'24, trim_to=0'0, roll_forward_to=35'22))", 2024-09-16T14:48:44.977 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.920475+0000", 2024-09-16T14:48:44.977 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.0413317099999999, 2024-09-16T14:48:44.977 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0034691589999999999, 2024-09-16T14:48:44.977 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:44.977 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.978 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:44.978 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.978 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:44.978 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920475+0000", 2024-09-16T14:48:44.978 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.978 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.978 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.978 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:44.978 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920475+0000", 2024-09-16T14:48:44.978 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.978 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.978 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.978 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:44.978 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920474+0000", 2024-09-16T14:48:44.979 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:44.979 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.979 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.979 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:44.979 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920502+0000", 2024-09-16T14:48:44.979 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7549999999999999e-05 2024-09-16T14:48:44.979 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.979 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.979 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:44.979 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920515+0000", 2024-09-16T14:48:44.979 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3728e-05 2024-09-16T14:48:44.979 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.979 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.979 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.980 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920524+0000", 2024-09-16T14:48:44.980 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9230000000000008e-06 2024-09-16T14:48:44.980 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.980 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.980 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:44.980 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920595+0000", 2024-09-16T14:48:44.980 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.0436000000000001e-05 2024-09-16T14:48:44.980 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.980 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.980 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.980 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920604+0000", 2024-09-16T14:48:44.980 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4700000000000008e-06 2024-09-16T14:48:44.980 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.981 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.981 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.981 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.923910+0000", 2024-09-16T14:48:44.981 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0033056130000000002 2024-09-16T14:48:44.981 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.981 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.981 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:44.981 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.923944+0000", 2024-09-16T14:48:44.981 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.4591000000000001e-05 2024-09-16T14:48:44.981 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.981 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:44.981 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.981 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.981 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s2 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.921392+0000", 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.040415114, 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0025447339999999999, 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921392+0000", 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:44.982 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921392+0000", 2024-09-16T14:48:44.983 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.983 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.983 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.983 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:44.983 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921391+0000", 2024-09-16T14:48:44.983 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.983 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.983 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.983 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:44.983 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921412+0000", 2024-09-16T14:48:44.983 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0825000000000001e-05 2024-09-16T14:48:44.983 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.983 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.983 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:44.984 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921423+0000", 2024-09-16T14:48:44.984 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1085e-05 2024-09-16T14:48:44.984 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.984 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.984 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.984 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921431+0000", 2024-09-16T14:48:44.984 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.6140000000000003e-06 2024-09-16T14:48:44.984 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.984 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.984 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:44.984 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921488+0000", 2024-09-16T14:48:44.984 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6827999999999999e-05 2024-09-16T14:48:44.984 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.984 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.985 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.985 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921497+0000", 2024-09-16T14:48:44.985 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3170000000000008e-06 2024-09-16T14:48:44.985 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.985 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.985 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.985 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.923910+0000", 2024-09-16T14:48:44.985 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0024134349999999998 2024-09-16T14:48:44.985 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.985 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.985 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:44.985 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.923936+0000", 2024-09-16T14:48:44.985 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6101999999999998e-05 2024-09-16T14:48:44.985 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.986 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:44.986 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.986 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.986 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.986 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs6 37/27 ECSubWrite(tid=189, reqid=osd.7.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'12))", 2024-09-16T14:48:44.986 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.921747+0000", 2024-09-16T14:48:44.986 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.0400595109999999, 2024-09-16T14:48:44.986 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0022272199999999998, 2024-09-16T14:48:44.986 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:44.986 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.986 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:44.986 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.986 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:44.986 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921747+0000", 2024-09-16T14:48:44.987 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.987 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.987 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:44.987 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921747+0000", 2024-09-16T14:48:44.987 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.987 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.987 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:44.987 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921747+0000", 2024-09-16T14:48:44.987 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.987 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.987 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:44.988 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921757+0000", 2024-09-16T14:48:44.988 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8870000000000004e-06 2024-09-16T14:48:44.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.988 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.988 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:44.988 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921767+0000", 2024-09-16T14:48:44.988 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7690000000000008e-06 2024-09-16T14:48:44.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.988 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.988 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.988 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921774+0000", 2024-09-16T14:48:44.988 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7090000000000002e-06 2024-09-16T14:48:44.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.988 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921945+0000", 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00017034600000000001 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921953+0000", 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7229999999999993e-06 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.923956+0000", 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0020028950000000002 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.990 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:44.990 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.923975+0000", 2024-09-16T14:48:44.990 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8351000000000001e-05 2024-09-16T14:48:44.990 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.990 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:44.990 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.990 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.990 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.990 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.bs1 37/27 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:44.990 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.921872+0000", 2024-09-16T14:48:44.990 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.0399351219999999, 2024-09-16T14:48:44.990 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0029324770000000002, 2024-09-16T14:48:44.990 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:44.990 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.991 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:44.991 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.991 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:44.991 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921872+0000", 2024-09-16T14:48:44.991 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.991 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.991 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:44.991 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921872+0000", 2024-09-16T14:48:44.991 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.991 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.992 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:44.992 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921871+0000", 2024-09-16T14:48:44.992 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.992 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.992 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.992 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:44.992 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.922210+0000", 2024-09-16T14:48:44.992 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00033898700000000001 2024-09-16T14:48:44.992 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.992 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.992 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:44.992 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.922382+0000", 2024-09-16T14:48:44.992 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00017168500000000001 2024-09-16T14:48:44.993 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.993 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.993 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.993 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.922389+0000", 2024-09-16T14:48:44.993 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.9299999999999997e-06 2024-09-16T14:48:44.993 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.993 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.993 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:44.993 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.922440+0000", 2024-09-16T14:48:44.993 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1031e-05 2024-09-16T14:48:44.993 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.993 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.993 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.922450+0000", 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0069e-05 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924781+0000", 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0023304799999999998 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924804+0000", 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3669999999999999e-05 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:44.994 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.995 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.995 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.995 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 37/27 ECSubWrite(tid=190, reqid=osd.7.0:188, at_version=0'0, trim_to=0'0, roll_forward_to=37'20))", 2024-09-16T14:48:44.995 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.921896+0000", 2024-09-16T14:48:44.995 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.0399108290000001, 2024-09-16T14:48:44.995 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0022546139999999998, 2024-09-16T14:48:44.995 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:44.995 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:44.995 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:44.995 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.995 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:44.995 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921896+0000", 2024-09-16T14:48:44.995 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.995 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.996 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.996 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:44.996 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921896+0000", 2024-09-16T14:48:44.996 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:44.996 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.996 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.996 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:44.996 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921896+0000", 2024-09-16T14:48:44.996 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:44.996 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.996 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.996 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:44.996 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921904+0000", 2024-09-16T14:48:44.997 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.303e-06 2024-09-16T14:48:44.997 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.997 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.997 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:44.997 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921913+0000", 2024-09-16T14:48:44.997 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8959999999999992e-06 2024-09-16T14:48:44.997 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.997 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.997 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:44.997 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921920+0000", 2024-09-16T14:48:44.997 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.0890000000000002e-06 2024-09-16T14:48:44.997 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.997 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.997 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:44.998 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921982+0000", 2024-09-16T14:48:44.998 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.2460999999999997e-05 2024-09-16T14:48:44.998 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.998 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.998 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:44.998 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:48:44.998 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4870000000000002e-06 2024-09-16T14:48:44.998 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.998 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.998 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:44.999 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924125+0000", 2024-09-16T14:48:44.999 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0021342819999999999 2024-09-16T14:48:44.999 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.999 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.999 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:44.999 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924151+0000", 2024-09-16T14:48:44.999 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5525999999999999e-05 2024-09-16T14:48:44.999 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.999 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:44.999 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:44.999 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:44.999 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:44.999 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:48:44.999 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.924050+0000", 2024-09-16T14:48:45.000 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.0377569010000001, 2024-09-16T14:48:45.000 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0050080109999999997, 2024-09-16T14:48:45.000 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:45.000 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:45.000 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:45.000 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.000 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:45.000 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924050+0000", 2024-09-16T14:48:45.000 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.000 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.000 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.000 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:45.000 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924050+0000", 2024-09-16T14:48:45.001 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.001 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.001 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.001 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:45.001 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924049+0000", 2024-09-16T14:48:45.001 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:45.001 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.001 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.001 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:45.001 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924073+0000", 2024-09-16T14:48:45.001 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3495000000000001e-05 2024-09-16T14:48:45.001 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.001 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.001 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:45.002 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924084+0000", 2024-09-16T14:48:45.002 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0821e-05 2024-09-16T14:48:45.002 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.002 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.002 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:45.002 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924092+0000", 2024-09-16T14:48:45.002 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4749999999999993e-06 2024-09-16T14:48:45.002 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.002 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.002 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:45.002 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924140+0000", 2024-09-16T14:48:45.002 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7488999999999999e-05 2024-09-16T14:48:45.002 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.002 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.003 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:45.003 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924149+0000", 2024-09-16T14:48:45.003 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6020000000000006e-06 2024-09-16T14:48:45.003 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.003 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.003 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:45.003 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.929014+0000", 2024-09-16T14:48:45.003 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0048650120000000002 2024-09-16T14:48:45.003 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.003 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.003 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:45.003 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.929058+0000", 2024-09-16T14:48:45.003 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.3593000000000001e-05 2024-09-16T14:48:45.003 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.004 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:45.004 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.004 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.004 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.004 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/27 ECSubWrite(tid=48, reqid=osd.5.0:47, at_version=0'0, trim_to=0'0, roll_forward_to=37'11))", 2024-09-16T14:48:45.004 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.925548+0000", 2024-09-16T14:48:45.004 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.036258814, 2024-09-16T14:48:45.004 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.004252507, 2024-09-16T14:48:45.004 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:45.004 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:45.004 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:45.004 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.004 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:45.004 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925548+0000", 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925548+0000", 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925547+0000", 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:45.005 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925560+0000", 2024-09-16T14:48:45.006 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3689000000000001e-05 2024-09-16T14:48:45.006 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.006 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.006 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:45.006 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925581+0000", 2024-09-16T14:48:45.006 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1081e-05 2024-09-16T14:48:45.006 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.006 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.006 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:45.006 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925601+0000", 2024-09-16T14:48:45.006 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9471999999999999e-05 2024-09-16T14:48:45.006 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.006 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.007 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:45.007 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925693+0000", 2024-09-16T14:48:45.007 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1694000000000007e-05 2024-09-16T14:48:45.007 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.007 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.007 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:45.007 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925765+0000", 2024-09-16T14:48:45.007 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2787999999999995e-05 2024-09-16T14:48:45.007 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.007 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.007 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:45.007 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.929769+0000", 2024-09-16T14:48:45.007 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0040033639999999997 2024-09-16T14:48:45.007 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.008 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.008 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:45.008 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.929801+0000", 2024-09-16T14:48:45.008 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1755999999999999e-05 2024-09-16T14:48:45.008 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.008 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:45.008 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.008 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.008 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.008 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.bs1 37/27 ECSubWrite(tid=118, reqid=osd.3.0:116, at_version=0'0, trim_to=0'0, roll_forward_to=37'14))", 2024-09-16T14:48:45.008 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.931602+0000", 2024-09-16T14:48:45.008 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.0302052310000001, 2024-09-16T14:48:45.008 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0033775490000000001, 2024-09-16T14:48:45.008 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:45.009 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:45.009 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:45.009 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.009 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:45.009 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.931602+0000", 2024-09-16T14:48:45.009 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.009 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.009 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.009 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:45.009 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.931602+0000", 2024-09-16T14:48:45.010 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.010 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.010 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.010 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:45.010 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.931601+0000", 2024-09-16T14:48:45.010 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:45.010 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.010 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.010 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:45.010 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.931622+0000", 2024-09-16T14:48:45.010 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1126999999999999e-05 2024-09-16T14:48:45.010 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.010 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.010 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:45.011 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.931635+0000", 2024-09-16T14:48:45.011 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2938999999999999e-05 2024-09-16T14:48:45.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.011 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.011 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:45.011 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.931643+0000", 2024-09-16T14:48:45.011 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5650000000000002e-06 2024-09-16T14:48:45.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.011 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.011 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:45.011 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.931773+0000", 2024-09-16T14:48:45.011 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012994800000000001 2024-09-16T14:48:45.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.012 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.012 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:45.012 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.931783+0000", 2024-09-16T14:48:45.012 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0086e-05 2024-09-16T14:48:45.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.012 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.012 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:45.012 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934961+0000", 2024-09-16T14:48:45.012 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0031773230000000001 2024-09-16T14:48:45.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.012 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.012 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:45.012 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934979+0000", 2024-09-16T14:48:45.012 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8368000000000001e-05 2024-09-16T14:48:45.013 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.013 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:45.013 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.013 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.013 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.013 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 37/27 ECSubWrite(tid=119, reqid=osd.3.0:119, at_version=37'39, trim_to=0'0, roll_forward_to=36'37))", 2024-09-16T14:48:45.013 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.932976+0000", 2024-09-16T14:48:45.013 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.02883112, 2024-09-16T14:48:45.013 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0037841419999999999, 2024-09-16T14:48:45.013 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:45.013 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:45.013 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:45.013 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.014 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:45.014 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.932976+0000", 2024-09-16T14:48:45.014 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.014 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.014 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:45.014 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.932976+0000", 2024-09-16T14:48:45.014 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.014 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.014 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:45.014 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.932975+0000", 2024-09-16T14:48:45.014 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:45.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.015 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.015 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:45.015 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.933000+0000", 2024-09-16T14:48:45.015 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4811999999999999e-05 2024-09-16T14:48:45.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.015 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.015 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:45.015 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.933012+0000", 2024-09-16T14:48:45.015 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2442999999999999e-05 2024-09-16T14:48:45.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.015 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.015 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:45.015 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.933023+0000", 2024-09-16T14:48:45.015 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0361e-05 2024-09-16T14:48:45.016 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.016 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.016 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:45.016 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.933228+0000", 2024-09-16T14:48:45.016 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00020499299999999999 2024-09-16T14:48:45.016 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.016 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.016 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:45.016 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.933238+0000", 2024-09-16T14:48:45.016 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0115000000000001e-05 2024-09-16T14:48:45.016 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.016 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.016 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.936730+0000", 2024-09-16T14:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.003492584 2024-09-16T14:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.936760+0000", 2024-09-16T14:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9629e-05 2024-09-16T14:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s2 37/27 ECSubWrite(tid=17, reqid=osd.4.0:16, at_version=0'0, trim_to=0'0, roll_forward_to=37'14))", 2024-09-16T14:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.934051+0000", 2024-09-16T14:48:45.018 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.0277556800000001, 2024-09-16T14:48:45.018 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0053229810000000001, 2024-09-16T14:48:45.018 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:45.018 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:45.018 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:45.018 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.018 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:45.018 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934051+0000", 2024-09-16T14:48:45.018 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.018 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.018 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:45.018 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934051+0000", 2024-09-16T14:48:45.018 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.019 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.019 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:45.019 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934050+0000", 2024-09-16T14:48:45.019 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:45.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.019 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.019 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:45.019 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934066+0000", 2024-09-16T14:48:45.019 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5432000000000001e-05 2024-09-16T14:48:45.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.019 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.019 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:45.019 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934083+0000", 2024-09-16T14:48:45.020 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7770000000000001e-05 2024-09-16T14:48:45.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.020 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:45.020 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934094+0000", 2024-09-16T14:48:45.020 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0808000000000001e-05 2024-09-16T14:48:45.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.020 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:45.020 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934144+0000", 2024-09-16T14:48:45.020 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.0093999999999998e-05 2024-09-16T14:48:45.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.020 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:45.021 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934154+0000", 2024-09-16T14:48:45.021 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7669999999999998e-06 2024-09-16T14:48:45.021 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.021 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.021 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:45.021 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.939339+0000", 2024-09-16T14:48:45.021 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0051848780000000004 2024-09-16T14:48:45.021 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.021 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.021 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:45.021 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.939374+0000", 2024-09-16T14:48:45.021 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5142999999999999e-05 2024-09-16T14:48:45.021 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.021 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:45.022 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.022 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.022 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 37/27 ECSubWrite(tid=120, reqid=osd.3.0:120, at_version=37'41, trim_to=0'0, roll_forward_to=36'37))", 2024-09-16T14:48:45.022 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.934424+0000", 2024-09-16T14:48:45.022 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.027382952, 2024-09-16T14:48:45.022 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0049802249999999996, 2024-09-16T14:48:45.022 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:45.022 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:45.022 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:45.022 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.022 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:45.022 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934424+0000", 2024-09-16T14:48:45.022 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.023 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.023 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.023 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:45.023 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934424+0000", 2024-09-16T14:48:45.023 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.023 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.023 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.023 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:45.023 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934423+0000", 2024-09-16T14:48:45.023 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:45.023 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.023 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.023 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:45.023 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934446+0000", 2024-09-16T14:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3178000000000001e-05 2024-09-16T14:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934458+0000", 2024-09-16T14:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1524e-05 2024-09-16T14:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934466+0000", 2024-09-16T14:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.875e-06 2024-09-16T14:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.024 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:45.025 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934512+0000", 2024-09-16T14:48:45.025 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6482e-05 2024-09-16T14:48:45.025 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.025 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.025 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:45.025 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934522+0000", 2024-09-16T14:48:45.025 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9890000000000004e-06 2024-09-16T14:48:45.025 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.025 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.025 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:45.025 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.939379+0000", 2024-09-16T14:48:45.025 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0048565589999999999 2024-09-16T14:48:45.025 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.025 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.026 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:45.026 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.939404+0000", 2024-09-16T14:48:45.026 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5239999999999999e-05 2024-09-16T14:48:45.026 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.026 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:45.026 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.026 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.026 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 37/27 ECSubWrite(tid=106, reqid=osd.11.0:105, at_version=0'0, trim_to=0'0, roll_forward_to=37'11))", 2024-09-16T14:48:45.026 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.935039+0000", 2024-09-16T14:48:45.026 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.0267682229999999, 2024-09-16T14:48:45.026 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0043319910000000003, 2024-09-16T14:48:45.026 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:45.026 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:45.027 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:45.027 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.027 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:45.027 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935039+0000", 2024-09-16T14:48:45.027 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.027 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.027 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:45.027 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935039+0000", 2024-09-16T14:48:45.027 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.027 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.027 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:45.028 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935038+0000", 2024-09-16T14:48:45.028 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:45.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.028 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.028 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:45.028 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935049+0000", 2024-09-16T14:48:45.028 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1394e-05 2024-09-16T14:48:45.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.028 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.028 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:45.028 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935060+0000", 2024-09-16T14:48:45.028 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1097000000000001e-05 2024-09-16T14:48:45.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.028 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.029 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:45.029 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935068+0000", 2024-09-16T14:48:45.029 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7889999999999995e-06 2024-09-16T14:48:45.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.029 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.029 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:45.029 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935121+0000", 2024-09-16T14:48:45.029 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.3049e-05 2024-09-16T14:48:45.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.029 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.029 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:45.029 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935131+0000", 2024-09-16T14:48:45.029 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3579999999999992e-06 2024-09-16T14:48:45.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.030 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:45.030 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.939341+0000", 2024-09-16T14:48:45.030 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0042103920000000003 2024-09-16T14:48:45.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.030 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:45.030 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.939371+0000", 2024-09-16T14:48:45.030 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9580000000000001e-05 2024-09-16T14:48:45.030 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.030 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:45.030 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.031 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.0s6 37/27 ECSubWrite(tid=109, reqid=osd.11.0:109, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:45.031 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.940407+0000", 2024-09-16T14:48:45.031 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.0213993809999999, 2024-09-16T14:48:45.031 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0051074889999999998, 2024-09-16T14:48:45.031 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:45.031 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:45.031 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:45.031 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.031 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:45.031 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940407+0000", 2024-09-16T14:48:45.031 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.031 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.031 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.031 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:45.032 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940407+0000", 2024-09-16T14:48:45.032 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.032 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.032 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.032 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:45.032 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940406+0000", 2024-09-16T14:48:45.032 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:45.032 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.032 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.032 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:45.032 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940427+0000", 2024-09-16T14:48:45.032 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0570000000000001e-05 2024-09-16T14:48:45.032 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.032 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.033 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:45.033 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940445+0000", 2024-09-16T14:48:45.033 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.785e-05 2024-09-16T14:48:45.033 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.033 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.033 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:45.033 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940464+0000", 2024-09-16T14:48:45.033 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9456000000000001e-05 2024-09-16T14:48:45.033 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.033 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.033 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:45.033 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940588+0000", 2024-09-16T14:48:45.034 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012323 2024-09-16T14:48:45.034 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.034 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.034 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:45.034 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940597+0000", 2024-09-16T14:48:45.034 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8279999999999998e-06 2024-09-16T14:48:45.034 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.034 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.034 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:45.034 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945487+0000", 2024-09-16T14:48:45.034 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0048893570000000004 2024-09-16T14:48:45.034 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.034 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.034 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:45.035 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945515+0000", 2024-09-16T14:48:45.035 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8215999999999999e-05 2024-09-16T14:48:45.035 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.035 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:45.035 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.035 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.035 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 37/27 ECSubWrite(tid=124, reqid=osd.3.0:120, at_version=0'0, trim_to=0'0, roll_forward_to=37'41))", 2024-09-16T14:48:45.035 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.942753+0000", 2024-09-16T14:48:45.035 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.019053478, 2024-09-16T14:48:45.035 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0039558650000000002, 2024-09-16T14:48:45.035 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:45.035 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:45.035 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:45.036 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.036 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:45.036 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942753+0000", 2024-09-16T14:48:45.036 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.036 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.036 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.036 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:45.036 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942753+0000", 2024-09-16T14:48:45.036 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.036 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.036 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.036 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:45.036 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942752+0000", 2024-09-16T14:48:45.036 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:45.037 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.037 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.037 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:45.037 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942762+0000", 2024-09-16T14:48:45.037 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9150000000000008e-06 2024-09-16T14:48:45.037 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.037 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.037 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:45.037 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942777+0000", 2024-09-16T14:48:45.037 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.489e-05 2024-09-16T14:48:45.037 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.037 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.037 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:45.037 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942792+0000", 2024-09-16T14:48:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4252e-05 2024-09-16T14:48:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942848+0000", 2024-09-16T14:48:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6888000000000001e-05 2024-09-16T14:48:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942859+0000", 2024-09-16T14:48:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0579e-05 2024-09-16T14:48:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.038 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:45.039 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.946678+0000", 2024-09-16T14:48:45.039 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0038192320000000001 2024-09-16T14:48:45.039 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.039 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.039 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:45.039 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.946709+0000", 2024-09-16T14:48:45.039 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1013000000000003e-05 2024-09-16T14:48:45.039 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.039 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:45.039 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.039 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.039 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.039 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 37/27 ECSubWrite(tid=125, reqid=osd.3.0:125, at_version=37'43, trim_to=0'0, roll_forward_to=37'41))", 2024-09-16T14:48:45.039 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.943611+0000", 2024-09-16T14:48:45.040 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.0181953509999999, 2024-09-16T14:48:45.040 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0031125200000000001, 2024-09-16T14:48:45.040 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:45.040 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:45.040 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:45.040 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.040 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:45.040 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.943611+0000", 2024-09-16T14:48:45.040 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.040 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.040 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.040 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:45.040 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.943611+0000", 2024-09-16T14:48:45.041 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.041 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.041 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.041 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:45.041 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.943611+0000", 2024-09-16T14:48:45.041 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:45.041 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.041 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.041 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:45.041 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.943636+0000", 2024-09-16T14:48:45.041 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5536000000000001e-05 2024-09-16T14:48:45.041 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.041 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.041 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:45.042 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.943657+0000", 2024-09-16T14:48:45.042 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0373e-05 2024-09-16T14:48:45.042 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.042 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.042 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:45.042 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.943665+0000", 2024-09-16T14:48:45.042 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2949999999999994e-06 2024-09-16T14:48:45.042 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.042 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.042 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:45.042 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.943773+0000", 2024-09-16T14:48:45.042 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000107815 2024-09-16T14:48:45.042 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.043 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.043 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:45.043 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.943785+0000", 2024-09-16T14:48:45.043 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2211e-05 2024-09-16T14:48:45.043 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.043 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.043 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:45.043 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.946715+0000", 2024-09-16T14:48:45.043 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0029300429999999998 2024-09-16T14:48:45.043 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.043 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.043 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:45.043 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.946724+0000", 2024-09-16T14:48:45.043 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0820000000000005e-06 2024-09-16T14:48:45.044 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.044 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:45.044 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.044 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.044 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.044 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.0s6 37/27 ECSubWrite(tid=111, reqid=osd.11.0:109, at_version=0'0, trim_to=0'0, roll_forward_to=37'14))", 2024-09-16T14:48:45.044 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.949806+0000", 2024-09-16T14:48:45.044 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.0120012869999999, 2024-09-16T14:48:45.044 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0045257789999999997, 2024-09-16T14:48:45.044 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:45.044 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:45.044 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:45.044 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.045 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:45.045 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949806+0000", 2024-09-16T14:48:45.045 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.045 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.045 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.045 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:45.045 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949806+0000", 2024-09-16T14:48:45.045 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.045 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.045 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.045 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:45.045 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949805+0000", 2024-09-16T14:48:45.045 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:45.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.046 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.046 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:45.046 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949815+0000", 2024-09-16T14:48:45.046 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0012e-05 2024-09-16T14:48:45.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.046 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.046 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:45.046 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949827+0000", 2024-09-16T14:48:45.046 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2516e-05 2024-09-16T14:48:45.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.046 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.046 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:45.046 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949833+0000", 2024-09-16T14:48:45.047 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1229999999999997e-06 2024-09-16T14:48:45.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.047 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.047 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:45.047 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949877+0000", 2024-09-16T14:48:45.047 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.3316999999999999e-05 2024-09-16T14:48:45.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.047 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.047 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:45.047 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949905+0000", 2024-09-16T14:48:45.047 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8816999999999999e-05 2024-09-16T14:48:45.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.047 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.047 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.954305+0000", 2024-09-16T14:48:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0043995060000000001 2024-09-16T14:48:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.954331+0000", 2024-09-16T14:48:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6378999999999999e-05 2024-09-16T14:48:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 37/27 ECSubWrite(tid=127, reqid=osd.3.0:125, at_version=0'0, trim_to=0'0, roll_forward_to=37'43))", 2024-09-16T14:48:45.048 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.950029+0000", 2024-09-16T14:48:45.049 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.0117781809999999, 2024-09-16T14:48:45.049 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0068229909999999996, 2024-09-16T14:48:45.049 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:45.049 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:45.049 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:45.049 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.049 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:45.049 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950029+0000", 2024-09-16T14:48:45.049 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.049 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.049 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.049 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:45.049 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950029+0000", 2024-09-16T14:48:45.049 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:45.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.050 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.050 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:45.050 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950028+0000", 2024-09-16T14:48:45.050 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:45.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.050 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.050 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:45.050 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950040+0000", 2024-09-16T14:48:45.050 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2139e-05 2024-09-16T14:48:45.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.050 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.050 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:45.050 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950053+0000", 2024-09-16T14:48:45.051 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2471e-05 2024-09-16T14:48:45.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.051 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.051 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:45.051 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950060+0000", 2024-09-16T14:48:45.051 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2150000000000004e-06 2024-09-16T14:48:45.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.051 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.051 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:45.051 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950508+0000", 2024-09-16T14:48:45.051 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00044776899999999999 2024-09-16T14:48:45.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.051 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.051 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:45.052 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950517+0000", 2024-09-16T14:48:45.052 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2960000000000004e-06 2024-09-16T14:48:45.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.052 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:45.052 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.956820+0000", 2024-09-16T14:48:45.052 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0063035119999999998 2024-09-16T14:48:45.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:45.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:45.052 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:45.052 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.956852+0000", 2024-09-16T14:48:45.052 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1287e-05 2024-09-16T14:48:45.052 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.053 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:45.053 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.053 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:45.053 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:45.053 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:48:45.054 INFO:tasks.rados.rados.0.smithi040.stdout:130: writing smithi04040151-41 from 688128 to 1458176 tid 1 2024-09-16T14:48:45.054 INFO:tasks.rados.rados.0.smithi040.stdout:130: writing smithi04040151-41 from 1458176 to 1835008 tid 2 2024-09-16T14:48:45.054 INFO:tasks.rados.rados.0.smithi040.stdout:128: done (3 left) 2024-09-16T14:48:45.054 INFO:tasks.rados.rados.0.smithi040.stdout:131: snap_create 2024-09-16T14:48:45.054 INFO:tasks.rados.rados.0.smithi040.stdout:126: finishing copy_from racing read to smithi04040151-18 2024-09-16T14:48:45.055 INFO:tasks.rados.rados.0.smithi040.stdout:121: finishing write tid 4 to smithi04040151-19 2024-09-16T14:48:45.055 INFO:tasks.rados.rados.0.smithi040.stdout:121: oid 19 updating version 0 to 22 2024-09-16T14:48:45.055 INFO:tasks.rados.rados.0.smithi040.stdout:121: oid 19 updating version 22 to 23 2024-09-16T14:48:45.055 INFO:tasks.rados.rados.0.smithi040.stdout:121: oid 19 version 23 is already newer than 21 2024-09-16T14:48:45.055 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 19 v 23 (ObjNum 66 snap 4 seq_num 66) dirty exists 2024-09-16T14:48:45.055 INFO:tasks.rados.rados.0.smithi040.stdout:121: left oid 19 (ObjNum 66 snap 4 seq_num 66) 2024-09-16T14:48:45.055 INFO:tasks.rados.rados.0.smithi040.stdout:126: finishing copy_from to smithi04040151-18 2024-09-16T14:48:45.055 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 18 v 4 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-16T14:48:45.056 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:48:45.057 INFO:tasks.rados.rados.0.smithi040.stdout:130: finishing write tid 1 to smithi04040151-41 2024-09-16T14:48:45.057 INFO:tasks.rados.rados.0.smithi040.stdout:130: finishing write tid 2 to smithi04040151-41 2024-09-16T14:48:45.057 INFO:tasks.rados.rados.0.smithi040.stdout:130: finishing write tid 3 to smithi04040151-41 2024-09-16T14:48:45.057 INFO:tasks.rados.rados.0.smithi040.stdout:130: finishing write tid 4 to smithi04040151-41 2024-09-16T14:48:45.057 INFO:tasks.rados.rados.0.smithi040.stdout:130: oid 41 updating version 0 to 14 2024-09-16T14:48:45.057 INFO:tasks.rados.rados.0.smithi040.stdout:130: oid 41 version 14 is already newer than 13 2024-09-16T14:48:45.057 INFO:tasks.rados.rados.0.smithi040.stdout:130: oid 41 updating version 14 to 15 2024-09-16T14:48:45.057 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 41 v 15 (ObjNum 68 snap 4 seq_num 68) dirty exists 2024-09-16T14:48:45.057 INFO:tasks.rados.rados.0.smithi040.stdout:130: left oid 41 (ObjNum 68 snap 4 seq_num 68) 2024-09-16T14:48:45.058 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:45.053+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:45.155 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:45.154+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:45.203 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:48:45.203 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:48:45.203 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:48:45.203 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:48:45.212 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:48:45.256 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:45.255+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:45.300 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:48:45.319 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:48:45.319 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:48:45.319 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:48:45.319 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:48:45.319 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:48:45.329 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:48:45.370 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:45.369+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:45.424 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:48:45.424 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:48:45.441 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:48:45.441 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:48:45.458 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:48:45.458 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:48:45.461 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:45.459+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:45.475 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:48:45.475 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:48:45.479 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:45.479 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:45.488 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:48:45.492 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:48:45.493 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:48:45.509 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:48:45.509 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:48:45.509 INFO:teuthology.orchestra.run.smithi040.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T14:48:45.531 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:48:45.531 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:48:45.547 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:48:45.548 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:48:45.559 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:45.558+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:45.564 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:48:45.564 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:48:45.581 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:48:45.581 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:48:45.597 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:48:45.598 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:48:45.601 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:48:45.601 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:48:45.605 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:45.605 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:45.615 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:48:45.700 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:45.699+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:45.762 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:45.761+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:45.762 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:45.762 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:45.771 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:48:45.863 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:45.862+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:45.918 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:45.918 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:45.926 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:48:45.929 INFO:tasks.rados.rados.0.smithi040.stdout:121: done (3 left) 2024-09-16T14:48:45.929 INFO:tasks.rados.rados.0.smithi040.stdout:126: done (2 left) 2024-09-16T14:48:45.930 INFO:tasks.rados.rados.0.smithi040.stdout:130: done (1 left) 2024-09-16T14:48:45.930 INFO:tasks.rados.rados.0.smithi040.stdout:131: done (0 left) 2024-09-16T14:48:45.930 INFO:tasks.rados.rados.0.smithi040.stdout:132: read oid 5 snap -1 2024-09-16T14:48:45.930 INFO:tasks.rados.rados.0.smithi040.stdout:132: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-16T14:48:45.930 INFO:tasks.rados.rados.0.smithi040.stdout:133: copy_from oid 9 from oid 27 current snap is 5 2024-09-16T14:48:45.930 INFO:tasks.rados.rados.0.smithi040.stdout:134: snap_remove snap 4 2024-09-16T14:48:45.942 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:48:45.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:45.972+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:46.039 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:46.039 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:46.048 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:48:46.065 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:46.064+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:46.156 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:46.156 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:46.166 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:48:46.166 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:46.164+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:46.267 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:46.266+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:46.287 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:46.286+0000 7fcdec02c640 -1 osd.6 23 log_to_monitors true 2024-09-16T14:48:46.313 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:46.313 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:46.322 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:48:46.368 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:46.367+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:46.434 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:46.434 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:46.443 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:48:46.469 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:46.468+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:46.546 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:46.546 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:46.554 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:48:46.570 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:46.569+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:46.656 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:46.656 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:46.665 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:48:46.670 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:46.669+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:46.768 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:46.769 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:46.771 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:46.770+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:46.777 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:48:46.872 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:46.870+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:46.919 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:46.920 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:46.928 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:48:46.940 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is set 2024-09-16T14:48:46.941 INFO:tasks.rados.rados.0.smithi040.stdout:134: done (2 left) 2024-09-16T14:48:46.941 INFO:tasks.rados.rados.0.smithi040.stdout:135: read oid 24 snap -1 2024-09-16T14:48:46.941 INFO:tasks.rados.rados.0.smithi040.stdout:135: expect deleted 2024-09-16T14:48:46.941 INFO:tasks.rados.rados.0.smithi040.stdout:136: copy_from oid 15 from oid 12 current snap is 5 2024-09-16T14:48:46.942 INFO:tasks.rados.rados.0.smithi040.stdout:132: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-16T14:48:46.946 INFO:tasks.rados.rados.0.smithi040.stdout:133: finishing copy_from racing read to smithi04040151-9 2024-09-16T14:48:46.946 INFO:tasks.rados.rados.0.smithi040.stdout:133: finishing copy_from to smithi04040151-9 2024-09-16T14:48:46.946 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 9 v 4 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-16T14:48:46.946 INFO:tasks.rados.rados.0.smithi040.stdout:132: done (3 left) 2024-09-16T14:48:46.947 INFO:tasks.rados.rados.0.smithi040.stdout:133: done (2 left) 2024-09-16T14:48:46.947 INFO:tasks.rados.rados.0.smithi040.stdout:137: rmattr oid 13 current snap is 5 2024-09-16T14:48:46.947 INFO:tasks.rados.rados.0.smithi040.stdout:137: done (2 left) 2024-09-16T14:48:46.947 INFO:tasks.rados.rados.0.smithi040.stdout:138: delete oid 48 current snap is 5 2024-09-16T14:48:46.951 INFO:tasks.rados.rados.0.smithi040.stdout:136: finishing copy_from racing read to smithi04040151-15 2024-09-16T14:48:46.954 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:46.953+0000 7fcddfdc9640 -1 osd.6 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T14:48:46.973 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:46.972+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:47.020 INFO:tasks.rados.rados.0.smithi040.stdout:136: finishing copy_from to smithi04040151-15 2024-09-16T14:48:47.020 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 15 v 2 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-16T14:48:47.020 INFO:tasks.rados.rados.0.smithi040.stdout:135: done (2 left) 2024-09-16T14:48:47.020 INFO:tasks.rados.rados.0.smithi040.stdout:136: done (1 left) 2024-09-16T14:48:47.020 INFO:tasks.rados.rados.0.smithi040.stdout:138: done (0 left) 2024-09-16T14:48:47.020 INFO:tasks.rados.rados.0.smithi040.stdout:139: delete oid 7 current snap is 5 2024-09-16T14:48:47.025 INFO:tasks.rados.rados.0.smithi040.stdout:139: done (0 left) 2024-09-16T14:48:47.025 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 26 current snap is 5 2024-09-16T14:48:47.025 INFO:tasks.rados.rados.0.smithi040.stdout:140: seq_num 71 ranges {0=425984,425984=573440,999424=196608} 2024-09-16T14:48:47.027 INFO:tasks.rados.rados.0.smithi040.stdout:140: writing smithi04040151-26 from 0 to 425984 tid 1 2024-09-16T14:48:47.031 INFO:tasks.rados.rados.0.smithi040.stdout:140: writing smithi04040151-26 from 425984 to 999424 tid 2 2024-09-16T14:48:47.032 INFO:tasks.rados.rados.0.smithi040.stdout:140: writing smithi04040151-26 from 999424 to 1196032 tid 3 2024-09-16T14:48:47.033 INFO:tasks.rados.rados.0.smithi040.stdout:141: copy_from oid 32 from oid 15 current snap is 5 2024-09-16T14:48:47.033 INFO:tasks.rados.rados.0.smithi040.stdout:142: read oid 5 snap -1 2024-09-16T14:48:47.033 INFO:tasks.rados.rados.0.smithi040.stdout:142: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-16T14:48:47.033 INFO:tasks.rados.rados.0.smithi040.stdout:143: setattr oid 34 current snap is 5 2024-09-16T14:48:47.035 INFO:tasks.rados.rados.0.smithi040.stdout:140: finishing write tid 1 to smithi04040151-26 2024-09-16T14:48:47.035 INFO:tasks.rados.rados.0.smithi040.stdout:142: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-16T14:48:47.039 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:47.040 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:47.040 INFO:tasks.rados.rados.0.smithi040.stdout:142: done (3 left) 2024-09-16T14:48:47.040 INFO:tasks.rados.rados.0.smithi040.stdout:144: rmattr oid 45 current snap is 5 2024-09-16T14:48:47.040 INFO:tasks.rados.rados.0.smithi040.stdout:144: done (3 left) 2024-09-16T14:48:47.041 INFO:tasks.rados.rados.0.smithi040.stdout:145: snap_create 2024-09-16T14:48:47.041 INFO:tasks.rados.rados.0.smithi040.stdout:141: finishing copy_from racing read to smithi04040151-32 2024-09-16T14:48:47.042 INFO:tasks.rados.rados.0.smithi040.stdout:141: finishing copy_from to smithi04040151-32 2024-09-16T14:48:47.042 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 32 v 2 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-16T14:48:47.042 INFO:tasks.rados.rados.0.smithi040.stdout:140: finishing write tid 2 to smithi04040151-26 2024-09-16T14:48:47.043 INFO:tasks.rados.rados.0.smithi040.stdout:140: finishing write tid 3 to smithi04040151-26 2024-09-16T14:48:47.043 INFO:tasks.rados.rados.0.smithi040.stdout:140: finishing write tid 4 to smithi04040151-26 2024-09-16T14:48:47.046 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 34 v 52 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-16T14:48:47.047 INFO:tasks.rados.rados.0.smithi040.stdout:140: finishing write tid 5 to smithi04040151-26 2024-09-16T14:48:47.047 INFO:tasks.rados.rados.0.smithi040.stdout:140: oid 26 updating version 0 to 49 2024-09-16T14:48:47.047 INFO:tasks.rados.rados.0.smithi040.stdout:140: oid 26 updating version 49 to 51 2024-09-16T14:48:47.047 INFO:tasks.rados.rados.0.smithi040.stdout:140: oid 26 version 51 is already newer than 48 2024-09-16T14:48:47.047 INFO:tasks.rados.rados.0.smithi040.stdout:140: oid 26 version 51 is already newer than 50 2024-09-16T14:48:47.047 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 26 v 51 (ObjNum 71 snap 5 seq_num 71) dirty exists 2024-09-16T14:48:47.047 INFO:tasks.rados.rados.0.smithi040.stdout:140: left oid 26 (ObjNum 71 snap 5 seq_num 71) 2024-09-16T14:48:47.049 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:48:47.090 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:47.089+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:47.175 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:47.173+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:47.193 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:47.193 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:47.202 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:48:47.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:47.275+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:47.305 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:47.305 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:47.314 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:48:47.351 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:48:47.377 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:47.375+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:47.419 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:47.419 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:47.428 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:48:47.461 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:47.462 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:48:47.462 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:48:47.462 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:47.470 INFO:tasks.thrashosds.thrasher:test_bluestore_reshard completed 2024-09-16T14:48:47.511 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:47.510+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:48:47.544 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:47.544 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:47.551 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:48:47.579 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:47.578+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:47.603 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:48:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:47.655 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:47.664 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:48:47.709 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:47.708+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:47.739 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:48:47.740 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:48:47.757 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:48:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:48:47.767 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:47.767 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:47.772 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:48:47.772 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:48:47.776 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:48:47.781 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:47.780+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:47.789 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:48:47.789 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:48:47.806 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:48:47.806 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:48:47.823 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:48:47.823 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:48:47.823 INFO:teuthology.orchestra.run.smithi040.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T14:48:47.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:47.881+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:47.917 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:47.918 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:47.926 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:48:47.938 INFO:tasks.rados.rados.0.smithi040.stdout:140: done (3 left) 2024-09-16T14:48:47.938 INFO:tasks.rados.rados.0.smithi040.stdout:141: done (2 left) 2024-09-16T14:48:47.939 INFO:tasks.rados.rados.0.smithi040.stdout:143: done (1 left) 2024-09-16T14:48:47.939 INFO:tasks.rados.rados.0.smithi040.stdout:145: done (0 left) 2024-09-16T14:48:47.939 INFO:tasks.rados.rados.0.smithi040.stdout:146: read oid 22 snap -1 2024-09-16T14:48:47.939 INFO:tasks.rados.rados.0.smithi040.stdout:146: expect deleted 2024-09-16T14:48:47.939 INFO:tasks.rados.rados.0.smithi040.stdout:147: read oid 2 snap 5 2024-09-16T14:48:47.939 INFO:tasks.rados.rados.0.smithi040.stdout:147: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T14:48:47.939 INFO:tasks.rados.rados.0.smithi040.stdout:148: read oid 1 snap -1 2024-09-16T14:48:47.939 INFO:tasks.rados.rados.0.smithi040.stdout:148: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-16T14:48:47.939 INFO:tasks.rados.rados.0.smithi040.stdout:149: copy_from oid 40 from oid 5 current snap is 6 2024-09-16T14:48:47.939 INFO:tasks.rados.rados.0.smithi040.stdout:150: snap_create 2024-09-16T14:48:47.943 INFO:tasks.rados.rados.0.smithi040.stdout:147: expect (ObjNum 1 snap 0 seq_num 1) 2024-09-16T14:48:47.945 INFO:tasks.rados.rados.0.smithi040.stdout:149: finishing copy_from racing read to smithi04040151-40 2024-09-16T14:48:47.946 INFO:tasks.rados.rados.0.smithi040.stdout:148: expect (ObjNum 0 snap 0 seq_num 0) 2024-09-16T14:48:47.947 INFO:tasks.rados.rados.0.smithi040.stdout:149: finishing copy_from to smithi04040151-40 2024-09-16T14:48:47.947 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 40 v 6 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-16T14:48:47.982 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:47.982+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:48:48.032 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:48.032 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:48.041 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:48:48.042 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:48:48.043 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:48:48.060 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:48:48.060 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:48:48.076 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:48:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:48:48.085 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:48.084+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:48.093 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:48:48.093 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:48:48.109 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:48:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:48:48.112 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T14:48:48.113 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T14:48:48.147 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:48.147 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:48.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:48.147 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:48.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.147 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=79, reqid=osd.3.0:79, at_version=34'18, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:48:48.147 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:48.147 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.2371660550000003, 2024-09-16T14:48:48.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069050049999999997, 2024-09-16T14:48:48.147 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.148 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.148 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:48.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:48.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:48.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907863+0000", 2024-09-16T14:48:48.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5972000000000001e-05 2024-09-16T14:48:48.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907876+0000", 2024-09-16T14:48:48.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3230999999999999e-05 2024-09-16T14:48:48.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907900+0000", 2024-09-16T14:48:48.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3716000000000001e-05 2024-09-16T14:48:48.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907956+0000", 2024-09-16T14:48:48.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6307999999999997e-05 2024-09-16T14:48:48.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907965+0000", 2024-09-16T14:48:48.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1530000000000003e-06 2024-09-16T14:48:48.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914715+0000", 2024-09-16T14:48:48.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067495990000000002 2024-09-16T14:48:48.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914742+0000", 2024-09-16T14:48:48.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.728e-05 2024-09-16T14:48:48.153 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.153 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.153 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.154 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=80, reqid=osd.3.0:80, at_version=34'20, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:48:48.154 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:48:48.154 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.2357493850000001, 2024-09-16T14:48:48.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065560849999999997, 2024-09-16T14:48:48.154 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.154 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.154 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:48:48.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:48:48.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909253+0000", 2024-09-16T14:48:48.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909278+0000", 2024-09-16T14:48:48.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4905e-05 2024-09-16T14:48:48.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909295+0000", 2024-09-16T14:48:48.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6713000000000001e-05 2024-09-16T14:48:48.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.156 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.156 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909304+0000", 2024-09-16T14:48:48.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6739999999999992e-06 2024-09-16T14:48:48.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909367+0000", 2024-09-16T14:48:48.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2700999999999995e-05 2024-09-16T14:48:48.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909391+0000", 2024-09-16T14:48:48.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3614000000000001e-05 2024-09-16T14:48:48.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915784+0000", 2024-09-16T14:48:48.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063938479999999997 2024-09-16T14:48:48.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915810+0000", 2024-09-16T14:48:48.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5457999999999999e-05 2024-09-16T14:48:48.158 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.158 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.158 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.158 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:48:48.158 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:48.158 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.2351217959999996, 2024-09-16T14:48:48.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007542137, 2024-09-16T14:48:48.159 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.159 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.159 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:48.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:48.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:48.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909917+0000", 2024-09-16T14:48:48.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6513000000000001e-05 2024-09-16T14:48:48.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:48:48.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:48:48.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909941+0000", 2024-09-16T14:48:48.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1859999999999992e-06 2024-09-16T14:48:48.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910004+0000", 2024-09-16T14:48:48.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2706999999999994e-05 2024-09-16T14:48:48.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910012+0000", 2024-09-16T14:48:48.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.157e-06 2024-09-16T14:48:48.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917395+0000", 2024-09-16T14:48:48.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073827659999999998 2024-09-16T14:48:48.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.162 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.162 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917423+0000", 2024-09-16T14:48:48.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8568000000000002e-05 2024-09-16T14:48:48.162 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.163 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.163 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.163 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:48:48.163 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:48:48.163 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.2309595560000002, 2024-09-16T14:48:48.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065747590000000003, 2024-09-16T14:48:48.163 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.163 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.163 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:48:48.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:48:48.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914043+0000", 2024-09-16T14:48:48.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914071+0000", 2024-09-16T14:48:48.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7633000000000001e-05 2024-09-16T14:48:48.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914086+0000", 2024-09-16T14:48:48.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4973999999999999e-05 2024-09-16T14:48:48.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914095+0000", 2024-09-16T14:48:48.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2119999999999992e-06 2024-09-16T14:48:48.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915626+0000", 2024-09-16T14:48:48.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001530998 2024-09-16T14:48:48.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915635+0000", 2024-09-16T14:48:48.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0140000000000004e-06 2024-09-16T14:48:48.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920590+0000", 2024-09-16T14:48:48.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049547740000000003 2024-09-16T14:48:48.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920618+0000", 2024-09-16T14:48:48.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8767000000000001e-05 2024-09-16T14:48:48.167 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.167 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.167 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.167 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:48:48.167 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:48.168 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.2286370660000001, 2024-09-16T14:48:48.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006879947, 2024-09-16T14:48:48.168 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.168 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.168 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.168 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:48.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:48.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:48.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:48.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:48.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8702000000000002e-05 2024-09-16T14:48:48.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916394+0000", 2024-09-16T14:48:48.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6350000000000001e-06 2024-09-16T14:48:48.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916403+0000", 2024-09-16T14:48:48.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3950000000000001e-06 2024-09-16T14:48:48.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916459+0000", 2024-09-16T14:48:48.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5889e-05 2024-09-16T14:48:48.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916467+0000", 2024-09-16T14:48:48.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2770000000000005e-06 2024-09-16T14:48:48.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923224+0000", 2024-09-16T14:48:48.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067573099999999999 2024-09-16T14:48:48.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923246+0000", 2024-09-16T14:48:48.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1882000000000002e-05 2024-09-16T14:48:48.172 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.172 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.172 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.173 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=82, reqid=osd.3.0:80, at_version=0'0, trim_to=0'0, roll_forward_to=34'20))", 2024-09-16T14:48:48.173 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:48:48.173 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.227063201, 2024-09-16T14:48:48.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005378954, 2024-09-16T14:48:48.173 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.173 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.173 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:48:48.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:48:48.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917939+0000", 2024-09-16T14:48:48.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917952+0000", 2024-09-16T14:48:48.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2278e-05 2024-09-16T14:48:48.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917963+0000", 2024-09-16T14:48:48.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.102e-05 2024-09-16T14:48:48.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917970+0000", 2024-09-16T14:48:48.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9589999999999998e-06 2024-09-16T14:48:48.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.918179+0000", 2024-09-16T14:48:48.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020897 2024-09-16T14:48:48.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.918188+0000", 2024-09-16T14:48:48.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9600000000000006e-06 2024-09-16T14:48:48.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923288+0000", 2024-09-16T14:48:48.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051001830000000003 2024-09-16T14:48:48.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923319+0000", 2024-09-16T14:48:48.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1131999999999999e-05 2024-09-16T14:48:48.177 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.177 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.177 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.177 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es5 34/27 ECSubWrite(tid=75, reqid=client.4500.0:229, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-16T14:48:48.177 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.920854+0000", 2024-09-16T14:48:48.177 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.2241495479999998, 2024-09-16T14:48:48.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050284889999999997, 2024-09-16T14:48:48.177 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920854+0000", 2024-09-16T14:48:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920854+0000", 2024-09-16T14:48:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.178 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920853+0000", 2024-09-16T14:48:48.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920863+0000", 2024-09-16T14:48:48.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0560999999999999e-05 2024-09-16T14:48:48.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.179 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.179 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920875+0000", 2024-09-16T14:48:48.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1588e-05 2024-09-16T14:48:48.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920883+0000", 2024-09-16T14:48:48.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7889999999999995e-06 2024-09-16T14:48:48.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920933+0000", 2024-09-16T14:48:48.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0759e-05 2024-09-16T14:48:48.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920943+0000", 2024-09-16T14:48:48.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.978e-06 2024-09-16T14:48:48.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925863+0000", 2024-09-16T14:48:48.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049191779999999997 2024-09-16T14:48:48.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925882+0000", 2024-09-16T14:48:48.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9525e-05 2024-09-16T14:48:48.182 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.182 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.182 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.182 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 34/27 ECSubWrite(tid=169, reqid=osd.7.0:168, at_version=0'0, trim_to=0'0, roll_forward_to=34'8))", 2024-09-16T14:48:48.182 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.921043+0000", 2024-09-16T14:48:48.182 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.2239599229999998, 2024-09-16T14:48:48.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048257229999999996, 2024-09-16T14:48:48.182 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.182 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.182 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921043+0000", 2024-09-16T14:48:48.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921043+0000", 2024-09-16T14:48:48.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921042+0000", 2024-09-16T14:48:48.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921057+0000", 2024-09-16T14:48:48.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4270999999999999e-05 2024-09-16T14:48:48.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921073+0000", 2024-09-16T14:48:48.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6821999999999999e-05 2024-09-16T14:48:48.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921200+0000", 2024-09-16T14:48:48.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012666899999999999 2024-09-16T14:48:48.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921252+0000", 2024-09-16T14:48:48.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1892999999999999e-05 2024-09-16T14:48:48.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921260+0000", 2024-09-16T14:48:48.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0649999999999999e-06 2024-09-16T14:48:48.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925844+0000", 2024-09-16T14:48:48.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045839169999999999 2024-09-16T14:48:48.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925869+0000", 2024-09-16T14:48:48.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4909000000000002e-05 2024-09-16T14:48:48.186 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.187 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.187 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.187 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=37'32, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:48.187 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:48:48.187 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2240114520000001, 2024-09-16T14:48:48.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070991120000000003, 2024-09-16T14:48:48.187 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.188 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.188 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:48:48.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:48:48.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920991+0000", 2024-09-16T14:48:48.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921010+0000", 2024-09-16T14:48:48.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8529000000000002e-05 2024-09-16T14:48:48.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921018+0000", 2024-09-16T14:48:48.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4670000000000004e-06 2024-09-16T14:48:48.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921025+0000", 2024-09-16T14:48:48.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9330000000000004e-06 2024-09-16T14:48:48.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921076+0000", 2024-09-16T14:48:48.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0662999999999999e-05 2024-09-16T14:48:48.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921084+0000", 2024-09-16T14:48:48.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4780000000000008e-06 2024-09-16T14:48:48.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928073+0000", 2024-09-16T14:48:48.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069883940000000002 2024-09-16T14:48:48.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928091+0000", 2024-09-16T14:48:48.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7992999999999999e-05 2024-09-16T14:48:48.191 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.192 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.192 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.192 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s6 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:48.192 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:48:48.192 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2230481969999998, 2024-09-16T14:48:48.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059936520000000004, 2024-09-16T14:48:48.192 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.192 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.192 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:48:48.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:48:48.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:48:48.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921983+0000", 2024-09-16T14:48:48.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8493999999999999e-05 2024-09-16T14:48:48.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:48:48.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3110000000000006e-06 2024-09-16T14:48:48.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921999+0000", 2024-09-16T14:48:48.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5499999999999997e-06 2024-09-16T14:48:48.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922042+0000", 2024-09-16T14:48:48.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3387999999999999e-05 2024-09-16T14:48:48.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922053+0000", 2024-09-16T14:48:48.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0244999999999999e-05 2024-09-16T14:48:48.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.927921+0000", 2024-09-16T14:48:48.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0058679159999999999 2024-09-16T14:48:48.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.927949+0000", 2024-09-16T14:48:48.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8038000000000002e-05 2024-09-16T14:48:48.196 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.196 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.196 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.197 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:48.197 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:48:48.197 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2230121470000004, 2024-09-16T14:48:48.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070100509999999998, 2024-09-16T14:48:48.197 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.197 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.197 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:48:48.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:48:48.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921990+0000", 2024-09-16T14:48:48.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922012+0000", 2024-09-16T14:48:48.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1681000000000001e-05 2024-09-16T14:48:48.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922025+0000", 2024-09-16T14:48:48.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3101999999999999e-05 2024-09-16T14:48:48.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922033+0000", 2024-09-16T14:48:48.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1899999999999995e-06 2024-09-16T14:48:48.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922727+0000", 2024-09-16T14:48:48.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00069429699999999997 2024-09-16T14:48:48.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922737+0000", 2024-09-16T14:48:48.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.285e-06 2024-09-16T14:48:48.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928986+0000", 2024-09-16T14:48:48.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062492679999999997 2024-09-16T14:48:48.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929001+0000", 2024-09-16T14:48:48.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5058999999999999e-05 2024-09-16T14:48:48.202 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.202 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.202 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.202 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 37/27 ECSubWrite(tid=189, reqid=osd.7.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'12))", 2024-09-16T14:48:48.203 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922523+0000", 2024-09-16T14:48:48.203 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2224802830000003, 2024-09-16T14:48:48.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055324470000000002, 2024-09-16T14:48:48.203 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.203 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.203 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922523+0000", 2024-09-16T14:48:48.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922523+0000", 2024-09-16T14:48:48.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922522+0000", 2024-09-16T14:48:48.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922531+0000", 2024-09-16T14:48:48.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6079999999999996e-06 2024-09-16T14:48:48.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922540+0000", 2024-09-16T14:48:48.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-16T14:48:48.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922548+0000", 2024-09-16T14:48:48.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4089999999999997e-06 2024-09-16T14:48:48.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922589+0000", 2024-09-16T14:48:48.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0853999999999999e-05 2024-09-16T14:48:48.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922597+0000", 2024-09-16T14:48:48.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3270000000000001e-06 2024-09-16T14:48:48.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928016+0000", 2024-09-16T14:48:48.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054186269999999996 2024-09-16T14:48:48.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928055+0000", 2024-09-16T14:48:48.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9749999999999997e-05 2024-09-16T14:48:48.207 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.207 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.207 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.207 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.ds2 37/27 ECSubWrite(tid=190, reqid=osd.7.0:188, at_version=0'0, trim_to=0'0, roll_forward_to=37'20))", 2024-09-16T14:48:48.207 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:48:48.207 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2222536919999998, 2024-09-16T14:48:48.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062183510000000004, 2024-09-16T14:48:48.207 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.207 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.207 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:48:48.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:48:48.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:48:48.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922758+0000", 2024-09-16T14:48:48.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6209999999999993e-06 2024-09-16T14:48:48.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922767+0000", 2024-09-16T14:48:48.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4159999999999993e-06 2024-09-16T14:48:48.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922780+0000", 2024-09-16T14:48:48.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2992e-05 2024-09-16T14:48:48.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922834+0000", 2024-09-16T14:48:48.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3835999999999998e-05 2024-09-16T14:48:48.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922911+0000", 2024-09-16T14:48:48.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7534999999999995e-05 2024-09-16T14:48:48.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928945+0000", 2024-09-16T14:48:48.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060339210000000002 2024-09-16T14:48:48.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928968+0000", 2024-09-16T14:48:48.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2450999999999999e-05 2024-09-16T14:48:48.211 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.211 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.211 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.211 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s6 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:48:48.211 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923698+0000", 2024-09-16T14:48:48.212 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2213054379999999, 2024-09-16T14:48:48.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055372479999999998, 2024-09-16T14:48:48.212 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.212 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.212 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923698+0000", 2024-09-16T14:48:48.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923698+0000", 2024-09-16T14:48:48.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923697+0000", 2024-09-16T14:48:48.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923708+0000", 2024-09-16T14:48:48.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1524e-05 2024-09-16T14:48:48.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923720+0000", 2024-09-16T14:48:48.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1620000000000001e-05 2024-09-16T14:48:48.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923732+0000", 2024-09-16T14:48:48.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1711e-05 2024-09-16T14:48:48.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923796+0000", 2024-09-16T14:48:48.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4673999999999998e-05 2024-09-16T14:48:48.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923805+0000", 2024-09-16T14:48:48.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3240000000000003e-06 2024-09-16T14:48:48.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929203+0000", 2024-09-16T14:48:48.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005398351 2024-09-16T14:48:48.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929235+0000", 2024-09-16T14:48:48.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1902000000000001e-05 2024-09-16T14:48:48.216 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.216 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.216 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.216 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:48:48.216 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:48:48.216 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2198030820000003, 2024-09-16T14:48:48.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079457759999999999, 2024-09-16T14:48:48.217 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.217 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.217 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:48:48.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:48:48.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:48:48.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925222+0000", 2024-09-16T14:48:48.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1871000000000001e-05 2024-09-16T14:48:48.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925238+0000", 2024-09-16T14:48:48.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6719999999999999e-05 2024-09-16T14:48:48.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925244+0000", 2024-09-16T14:48:48.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3630000000000002e-06 2024-09-16T14:48:48.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925291+0000", 2024-09-16T14:48:48.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6944000000000003e-05 2024-09-16T14:48:48.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925299+0000", 2024-09-16T14:48:48.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:48:48.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933129+0000", 2024-09-16T14:48:48.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078297330000000002 2024-09-16T14:48:48.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933146+0000", 2024-09-16T14:48:48.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.664e-05 2024-09-16T14:48:48.220 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.221 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.221 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.221 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es5 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:48:48.221 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:48:48.221 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2197514470000002, 2024-09-16T14:48:48.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078358790000000005, 2024-09-16T14:48:48.221 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.221 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.221 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:48:48.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:48:48.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925251+0000", 2024-09-16T14:48:48.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925265+0000", 2024-09-16T14:48:48.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4078e-05 2024-09-16T14:48:48.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925273+0000", 2024-09-16T14:48:48.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1069999999999999e-06 2024-09-16T14:48:48.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925279+0000", 2024-09-16T14:48:48.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4409999999999998e-06 2024-09-16T14:48:48.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925330+0000", 2024-09-16T14:48:48.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1208999999999997e-05 2024-09-16T14:48:48.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925339+0000", 2024-09-16T14:48:48.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8109999999999992e-06 2024-09-16T14:48:48.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933064+0000", 2024-09-16T14:48:48.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077252149999999997 2024-09-16T14:48:48.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933088+0000", 2024-09-16T14:48:48.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3317999999999999e-05 2024-09-16T14:48:48.225 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.225 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.225 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.225 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 37/27 ECSubWrite(tid=34, reqid=osd.1.0:33, at_version=0'0, trim_to=0'0, roll_forward_to=37'24))", 2024-09-16T14:48:48.225 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.931207+0000", 2024-09-16T14:48:48.226 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2137966110000002, 2024-09-16T14:48:48.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046407749999999998, 2024-09-16T14:48:48.226 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.226 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.226 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.931207+0000", 2024-09-16T14:48:48.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.931207+0000", 2024-09-16T14:48:48.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.931206+0000", 2024-09-16T14:48:48.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.931218+0000", 2024-09-16T14:48:48.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1958e-05 2024-09-16T14:48:48.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.931229+0000", 2024-09-16T14:48:48.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1309e-05 2024-09-16T14:48:48.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.931235+0000", 2024-09-16T14:48:48.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4520000000000002e-06 2024-09-16T14:48:48.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.931283+0000", 2024-09-16T14:48:48.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7355999999999999e-05 2024-09-16T14:48:48.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.931289+0000", 2024-09-16T14:48:48.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3430000000000001e-06 2024-09-16T14:48:48.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.935820+0000", 2024-09-16T14:48:48.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045307200000000002 2024-09-16T14:48:48.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.935847+0000", 2024-09-16T14:48:48.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7483000000000001e-05 2024-09-16T14:48:48.230 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.230 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.230 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.230 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=123, reqid=osd.3.0:123, at_version=37'38, trim_to=0'0, roll_forward_to=37'34))", 2024-09-16T14:48:48.230 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.936249+0000", 2024-09-16T14:48:48.230 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2087544579999996, 2024-09-16T14:48:48.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050599219999999997, 2024-09-16T14:48:48.230 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.231 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.231 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936249+0000", 2024-09-16T14:48:48.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936249+0000", 2024-09-16T14:48:48.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936248+0000", 2024-09-16T14:48:48.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936265+0000", 2024-09-16T14:48:48.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7010000000000001e-05 2024-09-16T14:48:48.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936274+0000", 2024-09-16T14:48:48.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9840000000000007e-06 2024-09-16T14:48:48.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936281+0000", 2024-09-16T14:48:48.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8109999999999997e-06 2024-09-16T14:48:48.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936317+0000", 2024-09-16T14:48:48.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5923000000000001e-05 2024-09-16T14:48:48.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936322+0000", 2024-09-16T14:48:48.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1390000000000003e-06 2024-09-16T14:48:48.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941287+0000", 2024-09-16T14:48:48.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049647529999999997 2024-09-16T14:48:48.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941309+0000", 2024-09-16T14:48:48.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1767e-05 2024-09-16T14:48:48.234 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.234 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.235 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.235 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 37/27 ECSubWrite(tid=124, reqid=osd.3.0:120, at_version=0'0, trim_to=0'0, roll_forward_to=37'41))", 2024-09-16T14:48:48.235 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.942813+0000", 2024-09-16T14:48:48.235 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2021902850000004, 2024-09-16T14:48:48.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050535570000000002, 2024-09-16T14:48:48.235 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.235 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.235 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942813+0000", 2024-09-16T14:48:48.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942813+0000", 2024-09-16T14:48:48.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942812+0000", 2024-09-16T14:48:48.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942833+0000", 2024-09-16T14:48:48.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0823999999999999e-05 2024-09-16T14:48:48.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942848+0000", 2024-09-16T14:48:48.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5577999999999999e-05 2024-09-16T14:48:48.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942857+0000", 2024-09-16T14:48:48.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0010000000000007e-06 2024-09-16T14:48:48.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942901+0000", 2024-09-16T14:48:48.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3832000000000003e-05 2024-09-16T14:48:48.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942911+0000", 2024-09-16T14:48:48.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6039999999999999e-06 2024-09-16T14:48:48.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.947844+0000", 2024-09-16T14:48:48.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049332129999999997 2024-09-16T14:48:48.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.947866+0000", 2024-09-16T14:48:48.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2393999999999999e-05 2024-09-16T14:48:48.239 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.239 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.239 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.239 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 37/27 ECSubWrite(tid=125, reqid=osd.3.0:125, at_version=37'43, trim_to=0'0, roll_forward_to=37'41))", 2024-09-16T14:48:48.239 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.943728+0000", 2024-09-16T14:48:48.239 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2012753619999996, 2024-09-16T14:48:48.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005567626, 2024-09-16T14:48:48.240 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.240 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.240 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943728+0000", 2024-09-16T14:48:48.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943728+0000", 2024-09-16T14:48:48.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943727+0000", 2024-09-16T14:48:48.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943749+0000", 2024-09-16T14:48:48.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1634999999999999e-05 2024-09-16T14:48:48.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943760+0000", 2024-09-16T14:48:48.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1727e-05 2024-09-16T14:48:48.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943768+0000", 2024-09-16T14:48:48.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4529999999999996e-06 2024-09-16T14:48:48.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943814+0000", 2024-09-16T14:48:48.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6346e-05 2024-09-16T14:48:48.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943823+0000", 2024-09-16T14:48:48.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3230000000000004e-06 2024-09-16T14:48:48.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949278+0000", 2024-09-16T14:48:48.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054548649999999997 2024-09-16T14:48:48.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949295+0000", 2024-09-16T14:48:48.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7079e-05 2024-09-16T14:48:48.243 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.243 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.243 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.244 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.244 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.244 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:48.245 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:48:48.246 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:48.244+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:48.345 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:48.344+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:48.387 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:48.388 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:48.388 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:48.388 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:48.397 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:48:48.446 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:48.445+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:48.500 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:48.500 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:48.500 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:48.500 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:48.500 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:48.508 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:48:48.550 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:48.548+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:48.649 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:48.647+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:48.652 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:48.652 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:48.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:48.653 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:48.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.653 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 32/27 ECSubWrite(tid=21, reqid=client.4500.0:214, at_version=32'12, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:48:48.653 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:38.899969+0000", 2024-09-16T14:48:48.653 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.750719879, 2024-09-16T14:48:48.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040995110000000001, 2024-09-16T14:48:48.653 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.653 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.653 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899969+0000", 2024-09-16T14:48:48.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899969+0000", 2024-09-16T14:48:48.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.899968+0000", 2024-09-16T14:48:48.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:48.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900282+0000", 2024-09-16T14:48:48.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00031321000000000002 2024-09-16T14:48:48.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900293+0000", 2024-09-16T14:48:48.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1582e-05 2024-09-16T14:48:48.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900302+0000", 2024-09-16T14:48:48.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0170000000000002e-06 2024-09-16T14:48:48.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900359+0000", 2024-09-16T14:48:48.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6750999999999999e-05 2024-09-16T14:48:48.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.900369+0000", 2024-09-16T14:48:48.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9299999999999998e-06 2024-09-16T14:48:48.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904046+0000", 2024-09-16T14:48:48.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036775240000000002 2024-09-16T14:48:48.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:38.904068+0000", 2024-09-16T14:48:48.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1710999999999999e-05 2024-09-16T14:48:48.659 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.659 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.659 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.659 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 34/27 ECSubWrite(tid=80, reqid=osd.3.0:80, at_version=34'20, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:48:48.659 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:48:48.659 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.7412735569999995, 2024-09-16T14:48:48.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055013639999999999, 2024-09-16T14:48:48.659 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.659 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.660 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:48:48.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:48:48.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909414+0000", 2024-09-16T14:48:48.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909442+0000", 2024-09-16T14:48:48.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7662999999999999e-05 2024-09-16T14:48:48.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909452+0000", 2024-09-16T14:48:48.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0773e-05 2024-09-16T14:48:48.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909463+0000", 2024-09-16T14:48:48.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0144e-05 2024-09-16T14:48:48.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909522+0000", 2024-09-16T14:48:48.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9651000000000002e-05 2024-09-16T14:48:48.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909531+0000", 2024-09-16T14:48:48.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8529999999999998e-06 2024-09-16T14:48:48.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914874+0000", 2024-09-16T14:48:48.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053432829999999999 2024-09-16T14:48:48.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914916+0000", 2024-09-16T14:48:48.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1928000000000002e-05 2024-09-16T14:48:48.664 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.664 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.664 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.664 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:48:48.664 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:48:48.664 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.7412047470000003, 2024-09-16T14:48:48.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079855010000000008, 2024-09-16T14:48:48.664 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:48:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:48:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909483+0000", 2024-09-16T14:48:48.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909521+0000", 2024-09-16T14:48:48.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7185999999999999e-05 2024-09-16T14:48:48.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909543+0000", 2024-09-16T14:48:48.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2802999999999998e-05 2024-09-16T14:48:48.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909551+0000", 2024-09-16T14:48:48.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5039999999999996e-06 2024-09-16T14:48:48.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909927+0000", 2024-09-16T14:48:48.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037637400000000001 2024-09-16T14:48:48.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:48:48.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2040000000000002e-06 2024-09-16T14:48:48.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917442+0000", 2024-09-16T14:48:48.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007508659 2024-09-16T14:48:48.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917469+0000", 2024-09-16T14:48:48.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7197999999999999e-05 2024-09-16T14:48:48.669 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.669 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.669 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.669 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:48:48.669 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:48.669 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.7371324010000002, 2024-09-16T14:48:48.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067701159999999996, 2024-09-16T14:48:48.669 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.669 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.670 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:48.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:48.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:48.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913579+0000", 2024-09-16T14:48:48.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2829000000000001e-05 2024-09-16T14:48:48.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913591+0000", 2024-09-16T14:48:48.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2264e-05 2024-09-16T14:48:48.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913600+0000", 2024-09-16T14:48:48.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9959999999999999e-06 2024-09-16T14:48:48.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913648+0000", 2024-09-16T14:48:48.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7759000000000001e-05 2024-09-16T14:48:48.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913657+0000", 2024-09-16T14:48:48.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6709999999999994e-06 2024-09-16T14:48:48.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920296+0000", 2024-09-16T14:48:48.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066385120000000001 2024-09-16T14:48:48.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920326+0000", 2024-09-16T14:48:48.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0405000000000001e-05 2024-09-16T14:48:48.673 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.674 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.674 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.674 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 34/27 ECSubWrite(tid=74, reqid=client.4500.0:229, at_version=34'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:48:48.674 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:48.674 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.7363164429999998, 2024-09-16T14:48:48.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060056090000000003, 2024-09-16T14:48:48.675 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.675 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.675 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:48.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:48.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:48.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914468+0000", 2024-09-16T14:48:48.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6398999999999996e-05 2024-09-16T14:48:48.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914483+0000", 2024-09-16T14:48:48.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5238e-05 2024-09-16T14:48:48.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914494+0000", 2024-09-16T14:48:48.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0192e-05 2024-09-16T14:48:48.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914622+0000", 2024-09-16T14:48:48.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012855599999999999 2024-09-16T14:48:48.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914635+0000", 2024-09-16T14:48:48.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2510000000000001e-05 2024-09-16T14:48:48.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920339+0000", 2024-09-16T14:48:48.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057041549999999998 2024-09-16T14:48:48.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920378+0000", 2024-09-16T14:48:48.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8918999999999998e-05 2024-09-16T14:48:48.680 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.680 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.680 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.680 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:48:48.680 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:48.681 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.7343036669999998, 2024-09-16T14:48:48.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069011319999999999, 2024-09-16T14:48:48.681 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.681 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.681 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:48.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:48.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:48.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:48.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916410+0000", 2024-09-16T14:48:48.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5343000000000001e-05 2024-09-16T14:48:48.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916418+0000", 2024-09-16T14:48:48.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2360000000000004e-06 2024-09-16T14:48:48.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916428+0000", 2024-09-16T14:48:48.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6779999999999995e-06 2024-09-16T14:48:48.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916492+0000", 2024-09-16T14:48:48.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4219000000000003e-05 2024-09-16T14:48:48.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916502+0000", 2024-09-16T14:48:48.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6690000000000001e-06 2024-09-16T14:48:48.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923254+0000", 2024-09-16T14:48:48.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067518070000000003 2024-09-16T14:48:48.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923286+0000", 2024-09-16T14:48:48.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2367999999999999e-05 2024-09-16T14:48:48.687 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.687 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.687 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.688 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 34/27 ECSubWrite(tid=75, reqid=client.4500.0:229, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-16T14:48:48.688 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.920873+0000", 2024-09-16T14:48:48.688 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.7298152529999999, 2024-09-16T14:48:48.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049917640000000001, 2024-09-16T14:48:48.688 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.688 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.688 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920873+0000", 2024-09-16T14:48:48.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920873+0000", 2024-09-16T14:48:48.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920872+0000", 2024-09-16T14:48:48.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920884+0000", 2024-09-16T14:48:48.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1501999999999999e-05 2024-09-16T14:48:48.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920898+0000", 2024-09-16T14:48:48.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4391e-05 2024-09-16T14:48:48.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920916+0000", 2024-09-16T14:48:48.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8057e-05 2024-09-16T14:48:48.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920965+0000", 2024-09-16T14:48:48.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8962000000000003e-05 2024-09-16T14:48:48.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920975+0000", 2024-09-16T14:48:48.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3989999999999993e-06 2024-09-16T14:48:48.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925839+0000", 2024-09-16T14:48:48.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048646330000000002 2024-09-16T14:48:48.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925865+0000", 2024-09-16T14:48:48.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5703000000000001e-05 2024-09-16T14:48:48.695 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.695 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.695 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.695 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/27 ECSubWrite(tid=169, reqid=osd.7.0:168, at_version=0'0, trim_to=0'0, roll_forward_to=34'8))", 2024-09-16T14:48:48.696 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.921046+0000", 2024-09-16T14:48:48.696 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 7.7296425280000003, 2024-09-16T14:48:48.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048734829999999996, 2024-09-16T14:48:48.696 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.696 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.696 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921046+0000", 2024-09-16T14:48:48.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921046+0000", 2024-09-16T14:48:48.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921045+0000", 2024-09-16T14:48:48.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921056+0000", 2024-09-16T14:48:48.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0733e-05 2024-09-16T14:48:48.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921067+0000", 2024-09-16T14:48:48.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1048e-05 2024-09-16T14:48:48.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921076+0000", 2024-09-16T14:48:48.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:48:48.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921135+0000", 2024-09-16T14:48:48.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8681000000000003e-05 2024-09-16T14:48:48.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921144+0000", 2024-09-16T14:48:48.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9399999999999997e-06 2024-09-16T14:48:48.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925903+0000", 2024-09-16T14:48:48.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047588860000000004 2024-09-16T14:48:48.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925920+0000", 2024-09-16T14:48:48.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6147999999999999e-05 2024-09-16T14:48:48.700 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.701 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.701 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.701 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s7 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:48.701 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:48:48.701 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.7289992830000003, 2024-09-16T14:48:48.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006972692, 2024-09-16T14:48:48.701 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.701 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.701 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:48:48.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:48:48.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:48:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921709+0000", 2024-09-16T14:48:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0411000000000002e-05 2024-09-16T14:48:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921719+0000", 2024-09-16T14:48:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2650000000000002e-06 2024-09-16T14:48:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921726+0000", 2024-09-16T14:48:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8770000000000004e-06 2024-09-16T14:48:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921773+0000", 2024-09-16T14:48:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7435999999999998e-05 2024-09-16T14:48:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921782+0000", 2024-09-16T14:48:48.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6039999999999993e-06 2024-09-16T14:48:48.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928635+0000", 2024-09-16T14:48:48.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068533129999999998 2024-09-16T14:48:48.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928662+0000", 2024-09-16T14:48:48.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7089000000000001e-05 2024-09-16T14:48:48.706 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.706 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.706 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.706 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:48.706 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:48:48.706 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.7285027949999998, 2024-09-16T14:48:48.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064783530000000001, 2024-09-16T14:48:48.706 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.706 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.707 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:48:48.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:48:48.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922185+0000", 2024-09-16T14:48:48.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922205+0000", 2024-09-16T14:48:48.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9871e-05 2024-09-16T14:48:48.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922219+0000", 2024-09-16T14:48:48.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4266e-05 2024-09-16T14:48:48.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922229+0000", 2024-09-16T14:48:48.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5619999999999993e-06 2024-09-16T14:48:48.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922270+0000", 2024-09-16T14:48:48.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0917000000000001e-05 2024-09-16T14:48:48.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922278+0000", 2024-09-16T14:48:48.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3890000000000005e-06 2024-09-16T14:48:48.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928630+0000", 2024-09-16T14:48:48.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063524439999999996 2024-09-16T14:48:48.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928664+0000", 2024-09-16T14:48:48.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3651e-05 2024-09-16T14:48:48.710 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.710 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.710 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.711 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s5 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:48:48.711 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:48:48.711 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.7270543910000002, 2024-09-16T14:48:48.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069615850000000002, 2024-09-16T14:48:48.711 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.711 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.711 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:48:48.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:48:48.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923633+0000", 2024-09-16T14:48:48.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923644+0000", 2024-09-16T14:48:48.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.08e-05 2024-09-16T14:48:48.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923657+0000", 2024-09-16T14:48:48.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2836999999999999e-05 2024-09-16T14:48:48.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923664+0000", 2024-09-16T14:48:48.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1470000000000004e-06 2024-09-16T14:48:48.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923885+0000", 2024-09-16T14:48:48.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022155800000000001 2024-09-16T14:48:48.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923894+0000", 2024-09-16T14:48:48.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9260000000000006e-06 2024-09-16T14:48:48.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930571+0000", 2024-09-16T14:48:48.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066770420000000002 2024-09-16T14:48:48.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930596+0000", 2024-09-16T14:48:48.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4383e-05 2024-09-16T14:48:48.715 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.715 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.715 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.716 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as3 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:48:48.716 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:48:48.716 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.7263081549999999, 2024-09-16T14:48:48.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062471519999999997, 2024-09-16T14:48:48.716 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.716 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.716 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:48:48.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:48:48.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:48:48.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924405+0000", 2024-09-16T14:48:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5147000000000002e-05 2024-09-16T14:48:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924414+0000", 2024-09-16T14:48:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4810000000000006e-06 2024-09-16T14:48:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924421+0000", 2024-09-16T14:48:48.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0899999999999999e-06 2024-09-16T14:48:48.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924468+0000", 2024-09-16T14:48:48.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.727e-05 2024-09-16T14:48:48.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924477+0000", 2024-09-16T14:48:48.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7360000000000007e-06 2024-09-16T14:48:48.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930601+0000", 2024-09-16T14:48:48.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061242529999999996 2024-09-16T14:48:48.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930628+0000", 2024-09-16T14:48:48.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6517e-05 2024-09-16T14:48:48.720 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.720 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.720 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.720 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:48:48.721 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:48:48.721 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.7260579509999996, 2024-09-16T14:48:48.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006019328, 2024-09-16T14:48:48.721 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.721 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.721 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:48:48.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:48:48.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924630+0000", 2024-09-16T14:48:48.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924650+0000", 2024-09-16T14:48:48.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0024e-05 2024-09-16T14:48:48.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924660+0000", 2024-09-16T14:48:48.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9219999999999992e-06 2024-09-16T14:48:48.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924668+0000", 2024-09-16T14:48:48.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6820000000000003e-06 2024-09-16T14:48:48.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924711+0000", 2024-09-16T14:48:48.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3009000000000001e-05 2024-09-16T14:48:48.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924720+0000", 2024-09-16T14:48:48.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9290000000000004e-06 2024-09-16T14:48:48.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930634+0000", 2024-09-16T14:48:48.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059144810000000001 2024-09-16T14:48:48.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930650+0000", 2024-09-16T14:48:48.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5688999999999998e-05 2024-09-16T14:48:48.725 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.725 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.725 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.725 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.5s7 37/27 ECSubWrite(tid=48, reqid=osd.5.0:47, at_version=0'0, trim_to=0'0, roll_forward_to=37'11))", 2024-09-16T14:48:48.725 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925599+0000", 2024-09-16T14:48:48.725 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.7250893640000005, 2024-09-16T14:48:48.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050916670000000002, 2024-09-16T14:48:48.726 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.726 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.726 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925599+0000", 2024-09-16T14:48:48.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925599+0000", 2024-09-16T14:48:48.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925598+0000", 2024-09-16T14:48:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925608+0000", 2024-09-16T14:48:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0034000000000001e-05 2024-09-16T14:48:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925619+0000", 2024-09-16T14:48:48.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0575e-05 2024-09-16T14:48:48.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925624+0000", 2024-09-16T14:48:48.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3909999999999997e-06 2024-09-16T14:48:48.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925664+0000", 2024-09-16T14:48:48.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9379999999999999e-05 2024-09-16T14:48:48.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925670+0000", 2024-09-16T14:48:48.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6619999999999999e-06 2024-09-16T14:48:48.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930669+0000", 2024-09-16T14:48:48.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049987199999999999 2024-09-16T14:48:48.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930691+0000", 2024-09-16T14:48:48.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1695000000000002e-05 2024-09-16T14:48:48.731 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.731 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.731 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.732 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 37/27 ECSubWrite(tid=119, reqid=osd.3.0:119, at_version=37'39, trim_to=0'0, roll_forward_to=36'37))", 2024-09-16T14:48:48.732 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.933227+0000", 2024-09-16T14:48:48.732 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.7174611449999997, 2024-09-16T14:48:48.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041478280000000001, 2024-09-16T14:48:48.732 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.732 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.732 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933227+0000", 2024-09-16T14:48:48.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933227+0000", 2024-09-16T14:48:48.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933226+0000", 2024-09-16T14:48:48.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933247+0000", 2024-09-16T14:48:48.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1259000000000001e-05 2024-09-16T14:48:48.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933264+0000", 2024-09-16T14:48:48.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6362e-05 2024-09-16T14:48:48.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933273+0000", 2024-09-16T14:48:48.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4150000000000005e-06 2024-09-16T14:48:48.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933318+0000", 2024-09-16T14:48:48.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999998e-05 2024-09-16T14:48:48.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933327+0000", 2024-09-16T14:48:48.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0080000000000008e-06 2024-09-16T14:48:48.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.937345+0000", 2024-09-16T14:48:48.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040184189999999996 2024-09-16T14:48:48.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.937375+0000", 2024-09-16T14:48:48.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9745000000000001e-05 2024-09-16T14:48:48.737 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.738 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.738 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.738 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 37/27 ECSubWrite(tid=123, reqid=osd.3.0:123, at_version=37'38, trim_to=0'0, roll_forward_to=37'34))", 2024-09-16T14:48:48.738 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.935961+0000", 2024-09-16T14:48:48.738 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.7147273819999995, 2024-09-16T14:48:48.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052566519999999997, 2024-09-16T14:48:48.738 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.738 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.738 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.935961+0000", 2024-09-16T14:48:48.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.935961+0000", 2024-09-16T14:48:48.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.935961+0000", 2024-09-16T14:48:48.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936022+0000", 2024-09-16T14:48:48.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1205999999999996e-05 2024-09-16T14:48:48.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936037+0000", 2024-09-16T14:48:48.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5478e-05 2024-09-16T14:48:48.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936045+0000", 2024-09-16T14:48:48.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7060000000000004e-06 2024-09-16T14:48:48.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936604+0000", 2024-09-16T14:48:48.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055949199999999998 2024-09-16T14:48:48.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936612+0000", 2024-09-16T14:48:48.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:48:48.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941192+0000", 2024-09-16T14:48:48.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045797370000000004 2024-09-16T14:48:48.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941218+0000", 2024-09-16T14:48:48.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5826e-05 2024-09-16T14:48:48.742 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.742 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.742 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.743 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.8s6 37/27 ECSubWrite(tid=110, reqid=osd.11.0:110, at_version=37'35, trim_to=0'0, roll_forward_to=35'31))", 2024-09-16T14:48:48.743 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.941403+0000", 2024-09-16T14:48:48.743 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.7092855819999997, 2024-09-16T14:48:48.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053034789999999998, 2024-09-16T14:48:48.743 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.743 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.743 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941403+0000", 2024-09-16T14:48:48.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941403+0000", 2024-09-16T14:48:48.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941402+0000", 2024-09-16T14:48:48.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941417+0000", 2024-09-16T14:48:48.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4578e-05 2024-09-16T14:48:48.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941431+0000", 2024-09-16T14:48:48.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3729e-05 2024-09-16T14:48:48.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941439+0000", 2024-09-16T14:48:48.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0199999999999994e-06 2024-09-16T14:48:48.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941487+0000", 2024-09-16T14:48:48.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8936e-05 2024-09-16T14:48:48.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941496+0000", 2024-09-16T14:48:48.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5320000000000006e-06 2024-09-16T14:48:48.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.946682+0000", 2024-09-16T14:48:48.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051858779999999997 2024-09-16T14:48:48.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.946706+0000", 2024-09-16T14:48:48.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4541000000000001e-05 2024-09-16T14:48:48.747 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.747 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.747 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.747 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 37/27 ECSubWrite(tid=125, reqid=osd.3.0:125, at_version=37'43, trim_to=0'0, roll_forward_to=37'41))", 2024-09-16T14:48:48.747 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.943746+0000", 2024-09-16T14:48:48.747 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.7069420549999998, 2024-09-16T14:48:48.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054472219999999998, 2024-09-16T14:48:48.747 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.748 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.748 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943746+0000", 2024-09-16T14:48:48.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943746+0000", 2024-09-16T14:48:48.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943746+0000", 2024-09-16T14:48:48.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943767+0000", 2024-09-16T14:48:48.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1325e-05 2024-09-16T14:48:48.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943779+0000", 2024-09-16T14:48:48.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1865000000000001e-05 2024-09-16T14:48:48.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943787+0000", 2024-09-16T14:48:48.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7449999999999995e-06 2024-09-16T14:48:48.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943828+0000", 2024-09-16T14:48:48.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0985000000000001e-05 2024-09-16T14:48:48.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943836+0000", 2024-09-16T14:48:48.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5459999999999992e-06 2024-09-16T14:48:48.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949139+0000", 2024-09-16T14:48:48.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053031220000000004 2024-09-16T14:48:48.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949194+0000", 2024-09-16T14:48:48.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4360000000000001e-05 2024-09-16T14:48:48.752 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.752 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.752 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.752 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 38/27 ECSubWrite(tid=36, reqid=client.4500.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=38'25))", 2024-09-16T14:48:48.752 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.942295+0000", 2024-09-16T14:48:48.752 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.7083937310000001, 2024-09-16T14:48:48.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040603790000000002, 2024-09-16T14:48:48.753 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.753 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.753 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942295+0000", 2024-09-16T14:48:48.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942295+0000", 2024-09-16T14:48:48.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942294+0000", 2024-09-16T14:48:48.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:48.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942304+0000", 2024-09-16T14:48:48.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0149999999999999e-05 2024-09-16T14:48:48.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942316+0000", 2024-09-16T14:48:48.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2215000000000001e-05 2024-09-16T14:48:48.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942324+0000", 2024-09-16T14:48:48.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0899999999999999e-06 2024-09-16T14:48:48.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942358+0000", 2024-09-16T14:48:48.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4158000000000001e-05 2024-09-16T14:48:48.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942365+0000", 2024-09-16T14:48:48.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8750000000000002e-06 2024-09-16T14:48:48.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.946331+0000", 2024-09-16T14:48:48.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0039660470000000003 2024-09-16T14:48:48.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.946355+0000", 2024-09-16T14:48:48.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4532000000000002e-05 2024-09-16T14:48:48.756 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.756 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.757 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.757 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 38/27 ECSubWrite(tid=116, reqid=client.4500.0:300, at_version=0'0, trim_to=0'0, roll_forward_to=38'14))", 2024-09-16T14:48:48.757 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.943674+0000", 2024-09-16T14:48:48.757 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.7070143230000001, 2024-09-16T14:48:48.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045685220000000002, 2024-09-16T14:48:48.757 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:48.757 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:48.757 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:48.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:48.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943674+0000", 2024-09-16T14:48:48.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:48.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943674+0000", 2024-09-16T14:48:48.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:48.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:48.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943673+0000", 2024-09-16T14:48:48.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:48.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:48.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943684+0000", 2024-09-16T14:48:48.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0599e-05 2024-09-16T14:48:48.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:48.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943699+0000", 2024-09-16T14:48:48.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5235000000000001e-05 2024-09-16T14:48:48.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:48.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943708+0000", 2024-09-16T14:48:48.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2439999999999993e-06 2024-09-16T14:48:48.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:48.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943756+0000", 2024-09-16T14:48:48.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8350000000000003e-05 2024-09-16T14:48:48.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:48.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943765+0000", 2024-09-16T14:48:48.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5119999999999997e-06 2024-09-16T14:48:48.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:48.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.948215+0000", 2024-09-16T14:48:48.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044499429999999996 2024-09-16T14:48:48.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:48.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:48.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:48.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.948243+0000", 2024-09-16T14:48:48.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7396999999999999e-05 2024-09-16T14:48:48.761 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.761 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.761 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.761 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:48.761 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:48.761 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:48.763 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:48:48.764 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:48.762+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:48.865 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:48.864+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:48.906 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:48.907 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:48.907 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:48.916 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:48:48.940 INFO:tasks.rados.rados.0.smithi040.stdout:146: done (4 left) 2024-09-16T14:48:48.940 INFO:tasks.rados.rados.0.smithi040.stdout:147: done (3 left) 2024-09-16T14:48:48.940 INFO:tasks.rados.rados.0.smithi040.stdout:148: done (2 left) 2024-09-16T14:48:48.940 INFO:tasks.rados.rados.0.smithi040.stdout:149: done (1 left) 2024-09-16T14:48:48.941 INFO:tasks.rados.rados.0.smithi040.stdout:150: done (0 left) 2024-09-16T14:48:48.941 INFO:tasks.rados.rados.0.smithi040.stdout:151: copy_from oid 28 from oid 41 current snap is 7 2024-09-16T14:48:48.941 INFO:tasks.rados.rados.0.smithi040.stdout:152: delete oid 16 current snap is 7 2024-09-16T14:48:48.943 INFO:tasks.rados.rados.0.smithi040.stdout:152: done (1 left) 2024-09-16T14:48:48.944 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 37 current snap is 7 2024-09-16T14:48:48.944 INFO:tasks.rados.rados.0.smithi040.stdout:153: seq_num 73 ranges {1458176=458752,1916928=475136,2392064=294912} 2024-09-16T14:48:48.954 INFO:tasks.rados.rados.0.smithi040.stdout:153: writing smithi04040151-37 from 1458176 to 1916928 tid 1 2024-09-16T14:48:48.957 INFO:tasks.rados.rados.0.smithi040.stdout:153: writing smithi04040151-37 from 1916928 to 2392064 tid 2 2024-09-16T14:48:48.960 INFO:tasks.rados.rados.0.smithi040.stdout:153: writing smithi04040151-37 from 2392064 to 2686976 tid 3 2024-09-16T14:48:48.960 INFO:tasks.rados.rados.0.smithi040.stdout:154: rollback oid 31 current snap is 7 2024-09-16T14:48:48.961 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 31 to 5 2024-09-16T14:48:48.961 INFO:tasks.rados.rados.0.smithi040.stdout:151: finishing copy_from racing read to smithi04040151-28 2024-09-16T14:48:48.961 INFO:tasks.rados.rados.0.smithi040.stdout:151: finishing copy_from to smithi04040151-28 2024-09-16T14:48:48.961 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 28 v 15 (ObjNum 68 snap 4 seq_num 68) dirty exists 2024-09-16T14:48:48.961 INFO:tasks.rados.rados.0.smithi040.stdout:151: done (2 left) 2024-09-16T14:48:48.961 INFO:tasks.rados.rados.0.smithi040.stdout:155: snap_remove snap 6 2024-09-16T14:48:48.963 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:48:48.965 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:48.963+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:49.026 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:49.026 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:49.026 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:49.026 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:49.026 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:49.035 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:48:49.075 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:49.074+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:49.166 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:49.164+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:49.179 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:49.179 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:49.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:49.179 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:49.179 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.179 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:21 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'11, mlcod=19'8)", 2024-09-16T14:48:49.179 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.389034+0000", 2024-09-16T14:48:49.179 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.786863059000002, 2024-09-16T14:48:49.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066701460000000001, 2024-09-16T14:48:49.180 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.180 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:49.180 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389034+0000", 2024-09-16T14:48:49.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.180 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.180 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389034+0000", 2024-09-16T14:48:49.180 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389033+0000", 2024-09-16T14:48:49.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:49.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389110+0000", 2024-09-16T14:48:49.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6965000000000006e-05 2024-09-16T14:48:49.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389122+0000", 2024-09-16T14:48:49.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.166e-05 2024-09-16T14:48:49.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389129+0000", 2024-09-16T14:48:49.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8120000000000002e-06 2024-09-16T14:48:49.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389166+0000", 2024-09-16T14:48:49.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6986999999999997e-05 2024-09-16T14:48:49.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:49.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389186+0000", 2024-09-16T14:48:49.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0043000000000001e-05 2024-09-16T14:48:49.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:49.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395613+0000", 2024-09-16T14:48:49.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064270109999999998 2024-09-16T14:48:49.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395704+0000", 2024-09-16T14:48:49.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1595000000000006e-05 2024-09-16T14:48:49.184 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.184 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.184 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.184 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_repop(client.4225.0:22 1.0 e19/18 1:30306672:devicehealth::main.db.0000000000000000:head v 19'12, mlcod=19'8)", 2024-09-16T14:48:49.184 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:22.389513+0000", 2024-09-16T14:48:49.184 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 26.786384668, 2024-09-16T14:48:49.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062632759999999999, 2024-09-16T14:48:49.184 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.184 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:49.184 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389513+0000", 2024-09-16T14:48:49.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389513+0000", 2024-09-16T14:48:49.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389512+0000", 2024-09-16T14:48:49.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:49.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389520+0000", 2024-09-16T14:48:49.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2609999999999993e-06 2024-09-16T14:48:49.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389533+0000", 2024-09-16T14:48:49.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2860999999999999e-05 2024-09-16T14:48:49.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389539+0000", 2024-09-16T14:48:49.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2369999999999997e-06 2024-09-16T14:48:49.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389568+0000", 2024-09-16T14:48:49.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9411999999999999e-05 2024-09-16T14:48:49.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:49.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.389584+0000", 2024-09-16T14:48:49.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5891000000000001e-05 2024-09-16T14:48:49.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:49.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395712+0000", 2024-09-16T14:48:49.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061272360000000003 2024-09-16T14:48:49.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:22.395776+0000", 2024-09-16T14:48:49.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4290999999999998e-05 2024-09-16T14:48:49.189 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.189 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.189 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.189 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.189 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.189 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:229 3.es0 3:7ecfd044:::smithi04040151-35:head [copy-from ver 4] snapc 4=[4,2] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:48:49.190 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:48:49.190 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.2717275210000007, 2024-09-16T14:48:49.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016506461, 2024-09-16T14:48:49.190 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.190 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:49.190 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:49.190 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:49.190 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:49.190 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 229 2024-09-16T14:48:49.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.190 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:48:49.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:48:49.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904168+0000", 2024-09-16T14:48:49.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:48:49.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904187+0000", 2024-09-16T14:48:49.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9449999999999998e-05 2024-09-16T14:48:49.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904206+0000", 2024-09-16T14:48:49.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8698e-05 2024-09-16T14:48:49.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.192 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.192 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904215+0000", 2024-09-16T14:48:49.192 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:48:49.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904276+0000", 2024-09-16T14:48:49.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1130999999999995e-05 2024-09-16T14:48:49.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:48:49.193 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904284+0000", 2024-09-16T14:48:49.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0609999999999996e-06 2024-09-16T14:48:49.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.193 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906272+0000", 2024-09-16T14:48:49.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019884210000000002 2024-09-16T14:48:49.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.194 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:49.194 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906391+0000", 2024-09-16T14:48:49.194 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000118843 2024-09-16T14:48:49.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.912595+0000", 2024-09-16T14:48:49.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062038190000000002 2024-09-16T14:48:49.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:49.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917584+0000", 2024-09-16T14:48:49.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049889069999999999 2024-09-16T14:48:49.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.195 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:49.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920592+0000", 2024-09-16T14:48:49.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030081639999999998 2024-09-16T14:48:49.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920676+0000", 2024-09-16T14:48:49.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3974999999999995e-05 2024-09-16T14:48:49.196 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.196 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.196 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.196 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:230 3.es0 3:7ecfd044:::smithi04040151-35:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:48:49.196 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:48:49.197 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.2716660379999993, 2024-09-16T14:48:49.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0096834569999999995, 2024-09-16T14:48:49.197 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.197 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:49.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:49.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:49.197 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:49.197 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 230 2024-09-16T14:48:49.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.197 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:48:49.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:48:49.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:48:49.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:49.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904235+0000", 2024-09-16T14:48:49.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4059999999999998e-06 2024-09-16T14:48:49.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904244+0000", 2024-09-16T14:48:49.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-16T14:48:49.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904250+0000", 2024-09-16T14:48:49.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0719999999999996e-06 2024-09-16T14:48:49.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904338+0000", 2024-09-16T14:48:49.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7923000000000005e-05 2024-09-16T14:48:49.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:48:49.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904344+0000", 2024-09-16T14:48:49.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8669999999999999e-06 2024-09-16T14:48:49.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906575+0000", 2024-09-16T14:48:49.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002230994 2024-09-16T14:48:49.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:48:49.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906632+0000", 2024-09-16T14:48:49.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7357e-05 2024-09-16T14:48:49.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913716+0000", 2024-09-16T14:48:49.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070835070000000002 2024-09-16T14:48:49.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:49.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913822+0000", 2024-09-16T14:48:49.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010566299999999999 2024-09-16T14:48:49.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913915+0000", 2024-09-16T14:48:49.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3018999999999998e-05 2024-09-16T14:48:49.202 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.202 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.202 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.203 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs1 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:48:49.203 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.913964+0000", 2024-09-16T14:48:49.203 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.2619331860000003, 2024-09-16T14:48:49.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063214940000000004, 2024-09-16T14:48:49.203 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.203 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:49.203 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913964+0000", 2024-09-16T14:48:49.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913964+0000", 2024-09-16T14:48:49.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913964+0000", 2024-09-16T14:48:49.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:49.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913996+0000", 2024-09-16T14:48:49.204 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2218000000000002e-05 2024-09-16T14:48:49.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914007+0000", 2024-09-16T14:48:49.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1216999999999999e-05 2024-09-16T14:48:49.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.205 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914018+0000", 2024-09-16T14:48:49.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0902e-05 2024-09-16T14:48:49.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.205 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.205 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914075+0000", 2024-09-16T14:48:49.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6864999999999999e-05 2024-09-16T14:48:49.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914084+0000", 2024-09-16T14:48:49.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1540000000000008e-06 2024-09-16T14:48:49.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.206 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:49.206 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920256+0000", 2024-09-16T14:48:49.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061714339999999999 2024-09-16T14:48:49.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.207 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.207 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920286+0000", 2024-09-16T14:48:49.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9910999999999999e-05 2024-09-16T14:48:49.207 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.207 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.207 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.207 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.207 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s1 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:48:49.207 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916482+0000", 2024-09-16T14:48:49.207 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.2594152679999997, 2024-09-16T14:48:49.207 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069103619999999998, 2024-09-16T14:48:49.207 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916482+0000", 2024-09-16T14:48:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916482+0000", 2024-09-16T14:48:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916482+0000", 2024-09-16T14:48:49.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:49.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916501+0000", 2024-09-16T14:48:49.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9494999999999999e-05 2024-09-16T14:48:49.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916511+0000", 2024-09-16T14:48:49.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4269999999999997e-06 2024-09-16T14:48:49.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916518+0000", 2024-09-16T14:48:49.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0310000000000001e-06 2024-09-16T14:48:49.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916572+0000", 2024-09-16T14:48:49.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4691000000000002e-05 2024-09-16T14:48:49.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916583+0000", 2024-09-16T14:48:49.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.039e-05 2024-09-16T14:48:49.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:49.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923344+0000", 2024-09-16T14:48:49.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067608720000000002 2024-09-16T14:48:49.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923392+0000", 2024-09-16T14:48:49.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8779999999999997e-05 2024-09-16T14:48:49.211 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.211 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.211 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.212 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:238 3.as0 3:53e83d93:::smithi04040151-22:4 [read 0~360448,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:48:49.212 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.940260+0000", 2024-09-16T14:48:49.212 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.2356376910000009, 2024-09-16T14:48:49.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068824810000000002, 2024-09-16T14:48:49.212 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.212 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:49.212 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:49.212 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:49.212 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:49.212 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 238 2024-09-16T14:48:49.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.212 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940260+0000", 2024-09-16T14:48:49.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940260+0000", 2024-09-16T14:48:49.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940259+0000", 2024-09-16T14:48:49.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:49.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940261+0000", 2024-09-16T14:48:49.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.717e-06 2024-09-16T14:48:49.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940264+0000", 2024-09-16T14:48:49.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2349999999999999e-06 2024-09-16T14:48:49.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.940267+0000", 2024-09-16T14:48:49.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8540000000000001e-06 2024-09-16T14:48:49.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.946049+0000", 2024-09-16T14:48:49.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057822469999999999 2024-09-16T14:48:49.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:49.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.946135+0000", 2024-09-16T14:48:49.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5871000000000005e-05 2024-09-16T14:48:49.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.947142+0000", 2024-09-16T14:48:49.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010067699999999999 2024-09-16T14:48:49.216 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.216 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.216 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.216 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs4 37/27 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:49.216 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:48:49.216 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.2551292270000003, 2024-09-16T14:48:49.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0098104279999999995, 2024-09-16T14:48:49.217 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.217 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:49.217 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:48:49.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:48:49.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:48:49.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:49.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.218 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.218 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920785+0000", 2024-09-16T14:48:49.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7218e-05 2024-09-16T14:48:49.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920793+0000", 2024-09-16T14:48:49.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9300000000000003e-06 2024-09-16T14:48:49.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920800+0000", 2024-09-16T14:48:49.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1860000000000003e-06 2024-09-16T14:48:49.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923692+0000", 2024-09-16T14:48:49.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028921329999999999 2024-09-16T14:48:49.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923701+0000", 2024-09-16T14:48:49.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6109999999999994e-06 2024-09-16T14:48:49.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:49.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930554+0000", 2024-09-16T14:48:49.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068534019999999998 2024-09-16T14:48:49.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930578+0000", 2024-09-16T14:48:49.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4232000000000001e-05 2024-09-16T14:48:49.222 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.223 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.223 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.223 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s4 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:49.223 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923896+0000", 2024-09-16T14:48:49.223 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.2520011369999997, 2024-09-16T14:48:49.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006707055, 2024-09-16T14:48:49.223 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.223 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:49.223 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923896+0000", 2024-09-16T14:48:49.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923896+0000", 2024-09-16T14:48:49.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923896+0000", 2024-09-16T14:48:49.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:49.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923920+0000", 2024-09-16T14:48:49.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3713000000000001e-05 2024-09-16T14:48:49.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923928+0000", 2024-09-16T14:48:49.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6810000000000004e-06 2024-09-16T14:48:49.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923936+0000", 2024-09-16T14:48:49.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.537e-06 2024-09-16T14:48:49.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923983+0000", 2024-09-16T14:48:49.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7686999999999999e-05 2024-09-16T14:48:49.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923993+0000", 2024-09-16T14:48:49.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3619999999999995e-06 2024-09-16T14:48:49.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:49.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930584+0000", 2024-09-16T14:48:49.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065914190000000003 2024-09-16T14:48:49.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930603+0000", 2024-09-16T14:48:49.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8936999999999998e-05 2024-09-16T14:48:49.229 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.229 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.229 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.229 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s1 37/27 ECSubWrite(tid=119, reqid=osd.3.0:119, at_version=37'39, trim_to=0'0, roll_forward_to=36'37))", 2024-09-16T14:48:49.230 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.933173+0000", 2024-09-16T14:48:49.230 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.2427245469999999, 2024-09-16T14:48:49.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006508443, 2024-09-16T14:48:49.230 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.230 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:49.230 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933173+0000", 2024-09-16T14:48:49.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.230 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.230 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933173+0000", 2024-09-16T14:48:49.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933172+0000", 2024-09-16T14:48:49.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:49.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933191+0000", 2024-09-16T14:48:49.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8766999999999998e-05 2024-09-16T14:48:49.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933202+0000", 2024-09-16T14:48:49.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1337e-05 2024-09-16T14:48:49.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933210+0000", 2024-09-16T14:48:49.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1049999999999995e-06 2024-09-16T14:48:49.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933258+0000", 2024-09-16T14:48:49.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7753000000000002e-05 2024-09-16T14:48:49.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933268+0000", 2024-09-16T14:48:49.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5009999999999993e-06 2024-09-16T14:48:49.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:49.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.939661+0000", 2024-09-16T14:48:49.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063929349999999998 2024-09-16T14:48:49.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.939681+0000", 2024-09-16T14:48:49.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0618000000000001e-05 2024-09-16T14:48:49.234 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.234 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.234 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.235 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s1 37/27 ECSubWrite(tid=120, reqid=osd.3.0:120, at_version=37'41, trim_to=0'0, roll_forward_to=36'37))", 2024-09-16T14:48:49.235 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.934770+0000", 2024-09-16T14:48:49.235 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.241126768, 2024-09-16T14:48:49.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0076309480000000002, 2024-09-16T14:48:49.235 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.235 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:49.235 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934770+0000", 2024-09-16T14:48:49.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934770+0000", 2024-09-16T14:48:49.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934770+0000", 2024-09-16T14:48:49.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:49.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934786+0000", 2024-09-16T14:48:49.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6401000000000001e-05 2024-09-16T14:48:49.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934796+0000", 2024-09-16T14:48:49.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0267e-05 2024-09-16T14:48:49.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934804+0000", 2024-09-16T14:48:49.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1040000000000007e-06 2024-09-16T14:48:49.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934848+0000", 2024-09-16T14:48:49.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4070000000000003e-05 2024-09-16T14:48:49.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934858+0000", 2024-09-16T14:48:49.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7079999999999992e-06 2024-09-16T14:48:49.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:49.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942379+0000", 2024-09-16T14:48:49.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0075206379999999996 2024-09-16T14:48:49.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942401+0000", 2024-09-16T14:48:49.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2597999999999999e-05 2024-09-16T14:48:49.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.242 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.242 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:298 3.es0 3:7ecfd044:::smithi04040151-35:head [append 0~720896] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:48:49.242 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:48:49.242 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2470637059999996, 2024-09-16T14:48:49.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014187808, 2024-09-16T14:48:49.242 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.242 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:49.242 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:49.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:49.243 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:49.243 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 298 2024-09-16T14:48:49.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.243 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:48:49.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:48:49.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928832+0000", 2024-09-16T14:48:49.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:49.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929746+0000", 2024-09-16T14:48:49.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00091434199999999995 2024-09-16T14:48:49.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929760+0000", 2024-09-16T14:48:49.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3519e-05 2024-09-16T14:48:49.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929767+0000", 2024-09-16T14:48:49.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3180000000000001e-06 2024-09-16T14:48:49.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929849+0000", 2024-09-16T14:48:49.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2020000000000004e-05 2024-09-16T14:48:49.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:48:49.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929854+0000", 2024-09-16T14:48:49.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1209999999999998e-06 2024-09-16T14:48:49.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931714+0000", 2024-09-16T14:48:49.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001859989 2024-09-16T14:48:49.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:49.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931845+0000", 2024-09-16T14:48:49.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130901 2024-09-16T14:48:49.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933089+0000", 2024-09-16T14:48:49.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001243535 2024-09-16T14:48:49.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:49.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936370+0000", 2024-09-16T14:48:49.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032817660000000002 2024-09-16T14:48:49.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:49.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942945+0000", 2024-09-16T14:48:49.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065741560000000003 2024-09-16T14:48:49.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943021+0000", 2024-09-16T14:48:49.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6734000000000001e-05 2024-09-16T14:48:49.249 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.250 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.250 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.250 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:299 3.es0 3:7ecfd044:::smithi04040151-35:head [append 0~376832] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:48:49.250 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:48:49.250 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2447739059999998, 2024-09-16T14:48:49.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012149627, 2024-09-16T14:48:49.250 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.250 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:49.250 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:49.250 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:49.250 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:49.250 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 299 2024-09-16T14:48:49.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.251 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:48:49.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:48:49.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931122+0000", 2024-09-16T14:48:49.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:49.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931722+0000", 2024-09-16T14:48:49.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059944100000000004 2024-09-16T14:48:49.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931732+0000", 2024-09-16T14:48:49.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9369999999999999e-06 2024-09-16T14:48:49.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931739+0000", 2024-09-16T14:48:49.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2819999999999999e-06 2024-09-16T14:48:49.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933835+0000", 2024-09-16T14:48:49.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020961140000000001 2024-09-16T14:48:49.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:49.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933929+0000", 2024-09-16T14:48:49.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3683000000000004e-05 2024-09-16T14:48:49.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.934917+0000", 2024-09-16T14:48:49.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00098803200000000002 2024-09-16T14:48:49.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:49.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.939558+0000", 2024-09-16T14:48:49.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00464175 2024-09-16T14:48:49.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:49.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943205+0000", 2024-09-16T14:48:49.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036465820000000002 2024-09-16T14:48:49.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943273+0000", 2024-09-16T14:48:49.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7917999999999996e-05 2024-09-16T14:48:49.256 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.256 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.256 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.256 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:300 3.es0 3:7ecfd044:::smithi04040151-35:head [setxattr _header (56) in=7b] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:48:49.257 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:48:49.257 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2441478119999996, 2024-09-16T14:48:49.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011740911, 2024-09-16T14:48:49.257 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.257 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:49.257 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:49.257 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:49.257 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:49.257 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 300 2024-09-16T14:48:49.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.257 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:48:49.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:48:49.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:48:49.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:49.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931753+0000", 2024-09-16T14:48:49.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0380000000000001e-06 2024-09-16T14:48:49.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931759+0000", 2024-09-16T14:48:49.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3430000000000001e-06 2024-09-16T14:48:49.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931765+0000", 2024-09-16T14:48:49.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-16T14:48:49.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.935623+0000", 2024-09-16T14:48:49.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038583049999999998 2024-09-16T14:48:49.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:49.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.935717+0000", 2024-09-16T14:48:49.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4587999999999996e-05 2024-09-16T14:48:49.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936150+0000", 2024-09-16T14:48:49.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043270599999999999 2024-09-16T14:48:49.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:49.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.939576+0000", 2024-09-16T14:48:49.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034257620000000002 2024-09-16T14:48:49.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:49.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943399+0000", 2024-09-16T14:48:49.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003823145 2024-09-16T14:48:49.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943490+0000", 2024-09-16T14:48:49.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1241000000000003e-05 2024-09-16T14:48:49.262 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.262 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.262 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.262 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:301 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:48:49.263 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:48:49.263 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.2441265120000002, 2024-09-16T14:48:49.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014215955000000001, 2024-09-16T14:48:49.263 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.263 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:49.263 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:49.263 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:49.263 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:49.263 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 301 2024-09-16T14:48:49.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.263 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:48:49.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:48:49.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931770+0000", 2024-09-16T14:48:49.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:49.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931775+0000", 2024-09-16T14:48:49.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4530000000000004e-06 2024-09-16T14:48:49.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931782+0000", 2024-09-16T14:48:49.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0550000000000002e-06 2024-09-16T14:48:49.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931787+0000", 2024-09-16T14:48:49.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2310000000000004e-06 2024-09-16T14:48:49.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936401+0000", 2024-09-16T14:48:49.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046143419999999996 2024-09-16T14:48:49.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:48:49.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936482+0000", 2024-09-16T14:48:49.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0407000000000005e-05 2024-09-16T14:48:49.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943851+0000", 2024-09-16T14:48:49.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073695000000000002 2024-09-16T14:48:49.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:49.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943951+0000", 2024-09-16T14:48:49.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9327000000000006e-05 2024-09-16T14:48:49.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.945987+0000", 2024-09-16T14:48:49.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002036057 2024-09-16T14:48:49.269 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.269 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.269 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.269 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:368 3.8s0 3:1f96bdfc:::smithi04040151-37:head [append 0~458752] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:48:49.269 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.954633+0000", 2024-09-16T14:48:49.269 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.22126447299999999, 2024-09-16T14:48:49.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090058240000000008, 2024-09-16T14:48:49.270 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.270 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:49.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:49.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:49.270 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:49.270 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 368 2024-09-16T14:48:49.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.270 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.954633+0000", 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.954633+0000", 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.954631+0000", 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.956022+0000", 2024-09-16T14:48:49.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001390591 2024-09-16T14:48:49.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.956034+0000", 2024-09-16T14:48:49.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2432999999999999e-05 2024-09-16T14:48:49.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.956041+0000", 2024-09-16T14:48:49.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8950000000000001e-06 2024-09-16T14:48:49.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.956093+0000", 2024-09-16T14:48:49.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1641999999999998e-05 2024-09-16T14:48:49.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:49.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.956194+0000", 2024-09-16T14:48:49.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100797 2024-09-16T14:48:49.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957200+0000", 2024-09-16T14:48:49.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001006847 2024-09-16T14:48:49.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:49.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960279+0000", 2024-09-16T14:48:49.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030784900000000001 2024-09-16T14:48:49.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:49.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.963564+0000", 2024-09-16T14:48:49.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003285217 2024-09-16T14:48:49.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.963639+0000", 2024-09-16T14:48:49.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4412000000000005e-05 2024-09-16T14:48:49.277 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.278 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.278 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.278 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:369 3.8s0 3:1f96bdfc:::smithi04040151-37:head [append 0~475136] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:48:49.278 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:48:49.278 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.21818451899999999, 2024-09-16T14:48:49.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010979078999999999, 2024-09-16T14:48:49.279 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.279 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:49.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:49.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:49.279 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:49.279 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 369 2024-09-16T14:48:49.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.280 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:48:49.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:48:49.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957711+0000", 2024-09-16T14:48:49.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:48:49.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958323+0000", 2024-09-16T14:48:49.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000612439 2024-09-16T14:48:49.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958339+0000", 2024-09-16T14:48:49.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5985999999999999e-05 2024-09-16T14:48:49.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958348+0000", 2024-09-16T14:48:49.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8699999999999998e-06 2024-09-16T14:48:49.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958393+0000", 2024-09-16T14:48:49.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4981000000000002e-05 2024-09-16T14:48:49.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:49.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958478+0000", 2024-09-16T14:48:49.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4845999999999994e-05 2024-09-16T14:48:49.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.959659+0000", 2024-09-16T14:48:49.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011808999999999999 2024-09-16T14:48:49.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:49.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962687+0000", 2024-09-16T14:48:49.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030278330000000002 2024-09-16T14:48:49.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:49.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968651+0000", 2024-09-16T14:48:49.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005964764 2024-09-16T14:48:49.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968692+0000", 2024-09-16T14:48:49.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0374999999999999e-05 2024-09-16T14:48:49.287 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.287 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.287 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.287 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:370 3.8s0 3:1f96bdfc:::smithi04040151-37:head [append 0~294912] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:48:49.287 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.959879+0000", 2024-09-16T14:48:49.287 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.216017768, 2024-09-16T14:48:49.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089453780000000004, 2024-09-16T14:48:49.288 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.288 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:49.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:49.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:49.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:49.288 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 370 2024-09-16T14:48:49.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.288 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.959879+0000", 2024-09-16T14:48:49.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.959879+0000", 2024-09-16T14:48:49.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.959878+0000", 2024-09-16T14:48:49.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:49.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960381+0000", 2024-09-16T14:48:49.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000502871 2024-09-16T14:48:49.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960395+0000", 2024-09-16T14:48:49.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3997e-05 2024-09-16T14:48:49.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960403+0000", 2024-09-16T14:48:49.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8960000000000003e-06 2024-09-16T14:48:49.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960433+0000", 2024-09-16T14:48:49.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0239999999999998e-05 2024-09-16T14:48:49.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:49.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960514+0000", 2024-09-16T14:48:49.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0451000000000006e-05 2024-09-16T14:48:49.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.961193+0000", 2024-09-16T14:48:49.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000679804 2024-09-16T14:48:49.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:49.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.963712+0000", 2024-09-16T14:48:49.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0025186549999999999 2024-09-16T14:48:49.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:49.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968794+0000", 2024-09-16T14:48:49.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050814529999999997 2024-09-16T14:48:49.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968825+0000", 2024-09-16T14:48:49.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1300999999999997e-05 2024-09-16T14:48:49.293 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.293 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.293 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.293 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:371 3.8s0 3:1f96bdfc:::smithi04040151-37:head [setxattr _header (56) in=7b] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:48:49.293 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.960414+0000", 2024-09-16T14:48:49.294 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.215483378, 2024-09-16T14:48:49.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088134779999999996, 2024-09-16T14:48:49.294 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.294 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:49.294 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:49.294 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:49.294 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:49.294 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 371 2024-09-16T14:48:49.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.294 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960414+0000", 2024-09-16T14:48:49.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960414+0000", 2024-09-16T14:48:49.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960413+0000", 2024-09-16T14:48:49.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:49.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960417+0000", 2024-09-16T14:48:49.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0330000000000002e-06 2024-09-16T14:48:49.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960424+0000", 2024-09-16T14:48:49.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0829999999999998e-06 2024-09-16T14:48:49.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960434+0000", 2024-09-16T14:48:49.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7019999999999996e-06 2024-09-16T14:48:49.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.961938+0000", 2024-09-16T14:48:49.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0015041340000000001 2024-09-16T14:48:49.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:49.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962037+0000", 2024-09-16T14:48:49.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9324e-05 2024-09-16T14:48:49.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962471+0000", 2024-09-16T14:48:49.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000433798 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.964825+0000", 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023542699999999999 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.969185+0000", 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0043595029999999998 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.969227+0000", 2024-09-16T14:48:49.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2366000000000001e-05 2024-09-16T14:48:49.299 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.299 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.299 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.299 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:372 3.8s0 3:1f96bdfc:::smithi04040151-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:48:49.299 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:48:49.299 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.215456183, 2024-09-16T14:48:49.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010116534999999999, 2024-09-16T14:48:49.299 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:49.299 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:49.299 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:49.299 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:49.299 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:49.300 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 372 2024-09-16T14:48:49.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.300 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:49.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:49.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:48:49.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:49.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:48:49.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:49.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:49.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:48:49.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:49.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:49.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960445+0000", 2024-09-16T14:48:49.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0579999999999999e-06 2024-09-16T14:48:49.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:49.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960452+0000", 2024-09-16T14:48:49.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5360000000000004e-06 2024-09-16T14:48:49.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960458+0000", 2024-09-16T14:48:49.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3260000000000003e-06 2024-09-16T14:48:49.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962902+0000", 2024-09-16T14:48:49.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002444313 2024-09-16T14:48:49.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:48:49.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962973+0000", 2024-09-16T14:48:49.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1179000000000004e-05 2024-09-16T14:48:49.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:49.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.969538+0000", 2024-09-16T14:48:49.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006565169 2024-09-16T14:48:49.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:49.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.969635+0000", 2024-09-16T14:48:49.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6427000000000004e-05 2024-09-16T14:48:49.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:49.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:49.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:49.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.970558+0000", 2024-09-16T14:48:49.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00092299600000000004 2024-09-16T14:48:49.306 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.307 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.307 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.307 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:49.307 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:49.307 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:49.310 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:48:49.310 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:49.308+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:49.410 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:49.409+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:49.425 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:49.425 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:48:49.425 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:48:49.425 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:49.434 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:48:49.519 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:49.518+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:49.577 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:49.577 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:48:49.577 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:48:49.577 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:48:49.577 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:49.586 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:48:49.612 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:49.611+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:49.712 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:49.711+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:49.730 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:49.730 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:48:49.730 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:48:49.730 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [] 2024-09-16T14:48:49.730 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:49.738 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:48:49.813 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:49.812+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:49.848 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is unset 2024-09-16T14:48:49.849 INFO:tasks.rados.rados.0.smithi040.stdout:155: done (2 left) 2024-09-16T14:48:49.849 INFO:tasks.rados.rados.0.smithi040.stdout:156: snap_create 2024-09-16T14:48:49.849 INFO:tasks.rados.rados.0.smithi040.stdout:153: finishing write tid 1 to smithi04040151-37 2024-09-16T14:48:49.849 INFO:tasks.rados.rados.0.smithi040.stdout:154: finishing rollback tid 0 to smithi04040151-31 2024-09-16T14:48:49.849 INFO:tasks.rados.rados.0.smithi040.stdout:153: finishing write tid 2 to smithi04040151-37 2024-09-16T14:48:49.849 INFO:tasks.rados.rados.0.smithi040.stdout:153: finishing write tid 3 to smithi04040151-37 2024-09-16T14:48:49.849 INFO:tasks.rados.rados.0.smithi040.stdout:154: finishing rollback tid 1 to smithi04040151-31 2024-09-16T14:48:49.849 INFO:tasks.rados.rados.0.smithi040.stdout:153: finishing write tid 4 to smithi04040151-37 2024-09-16T14:48:49.850 INFO:tasks.rados.rados.0.smithi040.stdout:154: finishing rollback tid 2 to smithi04040151-31 2024-09-16T14:48:49.850 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 31 v 56 (ObjNum 55 snap 1 seq_num 55) dirty exists 2024-09-16T14:48:49.850 INFO:tasks.rados.rados.0.smithi040.stdout:153: finishing write tid 5 to smithi04040151-37 2024-09-16T14:48:49.850 INFO:tasks.rados.rados.0.smithi040.stdout:153: oid 37 updating version 0 to 38 2024-09-16T14:48:49.850 INFO:tasks.rados.rados.0.smithi040.stdout:153: oid 37 version 38 is already newer than 37 2024-09-16T14:48:49.850 INFO:tasks.rados.rados.0.smithi040.stdout:153: oid 37 updating version 38 to 39 2024-09-16T14:48:49.850 INFO:tasks.rados.rados.0.smithi040.stdout:153: oid 37 updating version 39 to 40 2024-09-16T14:48:49.850 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 37 v 40 (ObjNum 73 snap 7 seq_num 73) dirty exists 2024-09-16T14:48:49.850 INFO:tasks.rados.rados.0.smithi040.stdout:153: left oid 37 (ObjNum 73 snap 7 seq_num 73) 2024-09-16T14:48:49.863 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:48:49.863 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:48:49.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:49.863 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 43/27 ECSubWrite(tid=148, reqid=osd.3.0:148, at_version=43'58, trim_to=0'0, roll_forward_to=42'56))", 2024-09-16T14:48:49.863 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:49.859010+0000", 2024-09-16T14:48:49.863 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.0032451730000000001, 2024-09-16T14:48:49.863 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0032711799999999998, 2024-09-16T14:48:49.863 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:49.864 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:49.864 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:49.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:49.864 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:49.864 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859010+0000", 2024-09-16T14:48:49.864 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:49.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:49.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:49.864 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:49.864 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859010+0000", 2024-09-16T14:48:49.864 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:49.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:49.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:49.865 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:49.865 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859009+0000", 2024-09-16T14:48:49.865 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:49.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:49.865 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:49.865 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:49.865 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859032+0000", 2024-09-16T14:48:49.865 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.355e-05 2024-09-16T14:48:49.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:49.865 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:49.865 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:49.865 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859043+0000", 2024-09-16T14:48:49.865 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0919e-05 2024-09-16T14:48:49.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:49.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:49.866 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:49.866 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859051+0000", 2024-09-16T14:48:49.866 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8590000000000005e-06 2024-09-16T14:48:49.866 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:49.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:49.866 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:49.866 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859118+0000", 2024-09-16T14:48:49.866 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.7018000000000001e-05 2024-09-16T14:48:49.866 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:49.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:49.866 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:49.866 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859129+0000", 2024-09-16T14:48:49.867 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.093e-05 2024-09-16T14:48:49.867 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:49.867 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:49.867 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:49.867 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:49.867 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-16T14:48:49.867 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 1 2024-09-16T14:48:49.867 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:48:49.874 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:48:49.914 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:49.914+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:50.016 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:50.015+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:50.021 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:48:50.021 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:48:50.021 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:48:50.021 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:48:50.021 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:48:50.030 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:48:50.114 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:48:50.117 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:50.116+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:50.139 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:48:50.140 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:48:50.140 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:48:50.140 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:48:50.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.140 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:48:50.140 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.924050+0000", 2024-09-16T14:48:50.140 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 6.2136186010000003, 2024-09-16T14:48:50.140 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0050080109999999997, 2024-09-16T14:48:50.140 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.140 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.141 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.141 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.141 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924050+0000", 2024-09-16T14:48:50.141 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.141 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.141 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924050+0000", 2024-09-16T14:48:50.141 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.141 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.142 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924049+0000", 2024-09-16T14:48:50.142 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:50.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.142 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.142 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924073+0000", 2024-09-16T14:48:50.142 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3495000000000001e-05 2024-09-16T14:48:50.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.142 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.142 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924084+0000", 2024-09-16T14:48:50.142 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0821e-05 2024-09-16T14:48:50.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.143 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.143 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924092+0000", 2024-09-16T14:48:50.143 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4749999999999993e-06 2024-09-16T14:48:50.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.143 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.143 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924140+0000", 2024-09-16T14:48:50.143 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7488999999999999e-05 2024-09-16T14:48:50.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.143 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.143 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924149+0000", 2024-09-16T14:48:50.143 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6020000000000006e-06 2024-09-16T14:48:50.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.144 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.144 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.929014+0000", 2024-09-16T14:48:50.144 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0048650120000000002 2024-09-16T14:48:50.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.144 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.144 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.929058+0000", 2024-09-16T14:48:50.144 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.3593000000000001e-05 2024-09-16T14:48:50.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.144 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.145 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s3 37/27 ECSubWrite(tid=48, reqid=osd.5.0:47, at_version=0'0, trim_to=0'0, roll_forward_to=37'11))", 2024-09-16T14:48:50.145 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.925548+0000", 2024-09-16T14:48:50.145 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 6.2121205140000004, 2024-09-16T14:48:50.145 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.004252507, 2024-09-16T14:48:50.145 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.145 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.145 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.145 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.145 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925548+0000", 2024-09-16T14:48:50.145 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.146 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.146 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925548+0000", 2024-09-16T14:48:50.146 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.146 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.146 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925547+0000", 2024-09-16T14:48:50.146 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:50.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.146 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.146 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925560+0000", 2024-09-16T14:48:50.147 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3689000000000001e-05 2024-09-16T14:48:50.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.147 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.147 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925581+0000", 2024-09-16T14:48:50.147 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1081e-05 2024-09-16T14:48:50.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.147 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.147 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925601+0000", 2024-09-16T14:48:50.147 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9471999999999999e-05 2024-09-16T14:48:50.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.147 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.148 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925693+0000", 2024-09-16T14:48:50.148 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1694000000000007e-05 2024-09-16T14:48:50.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.148 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.148 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.925765+0000", 2024-09-16T14:48:50.148 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2787999999999995e-05 2024-09-16T14:48:50.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.148 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.148 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.929769+0000", 2024-09-16T14:48:50.148 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0040033639999999997 2024-09-16T14:48:50.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.149 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.149 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.929801+0000", 2024-09-16T14:48:50.149 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1755999999999999e-05 2024-09-16T14:48:50.149 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.149 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.149 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.149 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 37/27 ECSubWrite(tid=119, reqid=osd.3.0:119, at_version=37'39, trim_to=0'0, roll_forward_to=36'37))", 2024-09-16T14:48:50.149 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.932976+0000", 2024-09-16T14:48:50.149 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 6.20469282, 2024-09-16T14:48:50.149 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0037841419999999999, 2024-09-16T14:48:50.149 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.150 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.150 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.150 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.150 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.932976+0000", 2024-09-16T14:48:50.150 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.150 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.150 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.932976+0000", 2024-09-16T14:48:50.151 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.151 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.151 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.932975+0000", 2024-09-16T14:48:50.151 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:50.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.151 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.151 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.933000+0000", 2024-09-16T14:48:50.151 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4811999999999999e-05 2024-09-16T14:48:50.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.933012+0000", 2024-09-16T14:48:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2442999999999999e-05 2024-09-16T14:48:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.933023+0000", 2024-09-16T14:48:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0361e-05 2024-09-16T14:48:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.933228+0000", 2024-09-16T14:48:50.152 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00020499299999999999 2024-09-16T14:48:50.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.153 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.153 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.933238+0000", 2024-09-16T14:48:50.153 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0115000000000001e-05 2024-09-16T14:48:50.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.153 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.153 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.936730+0000", 2024-09-16T14:48:50.153 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.003492584 2024-09-16T14:48:50.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.154 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.154 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.936760+0000", 2024-09-16T14:48:50.154 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9629e-05 2024-09-16T14:48:50.154 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.154 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.154 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.154 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s2 37/27 ECSubWrite(tid=17, reqid=osd.4.0:16, at_version=0'0, trim_to=0'0, roll_forward_to=37'14))", 2024-09-16T14:48:50.155 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.934051+0000", 2024-09-16T14:48:50.155 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 6.2036173799999998, 2024-09-16T14:48:50.155 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0053229810000000001, 2024-09-16T14:48:50.155 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.155 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.155 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.155 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.156 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934051+0000", 2024-09-16T14:48:50.156 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.156 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.156 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.156 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934051+0000", 2024-09-16T14:48:50.156 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.156 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.156 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.156 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934050+0000", 2024-09-16T14:48:50.156 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:50.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.157 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.157 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934066+0000", 2024-09-16T14:48:50.157 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5432000000000001e-05 2024-09-16T14:48:50.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.157 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.157 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934083+0000", 2024-09-16T14:48:50.158 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7770000000000001e-05 2024-09-16T14:48:50.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.158 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.158 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934094+0000", 2024-09-16T14:48:50.158 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0808000000000001e-05 2024-09-16T14:48:50.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.158 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.158 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934144+0000", 2024-09-16T14:48:50.158 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.0093999999999998e-05 2024-09-16T14:48:50.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.158 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.159 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934154+0000", 2024-09-16T14:48:50.159 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7669999999999998e-06 2024-09-16T14:48:50.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.159 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.159 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.159 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.939339+0000", 2024-09-16T14:48:50.159 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0051848780000000004 2024-09-16T14:48:50.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.159 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.159 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.159 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.939374+0000", 2024-09-16T14:48:50.159 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5142999999999999e-05 2024-09-16T14:48:50.159 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.160 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.160 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.160 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.160 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 37/27 ECSubWrite(tid=120, reqid=osd.3.0:120, at_version=37'41, trim_to=0'0, roll_forward_to=36'37))", 2024-09-16T14:48:50.160 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.934424+0000", 2024-09-16T14:48:50.160 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 6.2032446520000004, 2024-09-16T14:48:50.160 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0049802249999999996, 2024-09-16T14:48:50.160 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.160 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.160 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.160 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.160 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.160 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934424+0000", 2024-09-16T14:48:50.161 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.161 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.161 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934424+0000", 2024-09-16T14:48:50.161 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.161 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.161 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934423+0000", 2024-09-16T14:48:50.161 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:50.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.162 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.162 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.162 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934446+0000", 2024-09-16T14:48:50.162 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3178000000000001e-05 2024-09-16T14:48:50.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.162 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.162 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.162 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934458+0000", 2024-09-16T14:48:50.162 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1524e-05 2024-09-16T14:48:50.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.163 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.163 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934466+0000", 2024-09-16T14:48:50.163 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.875e-06 2024-09-16T14:48:50.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.163 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.163 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934512+0000", 2024-09-16T14:48:50.163 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6482e-05 2024-09-16T14:48:50.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.163 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.164 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.934522+0000", 2024-09-16T14:48:50.164 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9890000000000004e-06 2024-09-16T14:48:50.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.164 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.164 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.939379+0000", 2024-09-16T14:48:50.164 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0048565589999999999 2024-09-16T14:48:50.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.164 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.165 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.939404+0000", 2024-09-16T14:48:50.165 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5239999999999999e-05 2024-09-16T14:48:50.165 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.165 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.165 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.165 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.165 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 37/27 ECSubWrite(tid=106, reqid=osd.11.0:105, at_version=0'0, trim_to=0'0, roll_forward_to=37'11))", 2024-09-16T14:48:50.165 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.935039+0000", 2024-09-16T14:48:50.165 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 6.2026299229999999, 2024-09-16T14:48:50.166 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0043319910000000003, 2024-09-16T14:48:50.166 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.166 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.166 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.166 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.166 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.166 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935039+0000", 2024-09-16T14:48:50.166 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.166 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.166 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.167 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935039+0000", 2024-09-16T14:48:50.167 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.167 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.167 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935038+0000", 2024-09-16T14:48:50.167 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:50.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.167 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.168 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935049+0000", 2024-09-16T14:48:50.168 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1394e-05 2024-09-16T14:48:50.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.168 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.168 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935060+0000", 2024-09-16T14:48:50.168 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1097000000000001e-05 2024-09-16T14:48:50.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.169 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.169 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935068+0000", 2024-09-16T14:48:50.169 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7889999999999995e-06 2024-09-16T14:48:50.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.169 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.169 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.169 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935121+0000", 2024-09-16T14:48:50.169 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.3049e-05 2024-09-16T14:48:50.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.169 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.169 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.170 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935131+0000", 2024-09-16T14:48:50.170 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3579999999999992e-06 2024-09-16T14:48:50.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.170 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.170 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.939341+0000", 2024-09-16T14:48:50.170 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0042103920000000003 2024-09-16T14:48:50.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.170 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.170 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.939371+0000", 2024-09-16T14:48:50.170 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9580000000000001e-05 2024-09-16T14:48:50.170 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.170 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.171 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.171 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.171 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.171 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.0s6 37/27 ECSubWrite(tid=109, reqid=osd.11.0:109, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:50.171 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.940407+0000", 2024-09-16T14:48:50.171 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 6.1972610809999997, 2024-09-16T14:48:50.171 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0051074889999999998, 2024-09-16T14:48:50.171 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.171 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.171 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.171 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.171 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940407+0000", 2024-09-16T14:48:50.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.172 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940407+0000", 2024-09-16T14:48:50.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.172 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940406+0000", 2024-09-16T14:48:50.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:50.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.173 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.173 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940427+0000", 2024-09-16T14:48:50.173 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0570000000000001e-05 2024-09-16T14:48:50.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.173 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.173 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940445+0000", 2024-09-16T14:48:50.173 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.785e-05 2024-09-16T14:48:50.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.174 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.174 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940464+0000", 2024-09-16T14:48:50.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9456000000000001e-05 2024-09-16T14:48:50.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.174 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.174 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.174 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940588+0000", 2024-09-16T14:48:50.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012323 2024-09-16T14:48:50.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.174 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.174 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.174 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940597+0000", 2024-09-16T14:48:50.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8279999999999998e-06 2024-09-16T14:48:50.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.175 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.175 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945487+0000", 2024-09-16T14:48:50.175 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0048893570000000004 2024-09-16T14:48:50.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.175 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.175 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945515+0000", 2024-09-16T14:48:50.175 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8215999999999999e-05 2024-09-16T14:48:50.175 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.175 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.175 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.176 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 37/27 ECSubWrite(tid=124, reqid=osd.3.0:120, at_version=0'0, trim_to=0'0, roll_forward_to=37'41))", 2024-09-16T14:48:50.176 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.942753+0000", 2024-09-16T14:48:50.176 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 6.1949151779999996, 2024-09-16T14:48:50.176 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0039558650000000002, 2024-09-16T14:48:50.176 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.176 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.176 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.176 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.176 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.176 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942753+0000", 2024-09-16T14:48:50.176 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.176 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.177 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.177 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942753+0000", 2024-09-16T14:48:50.177 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.177 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.177 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942752+0000", 2024-09-16T14:48:50.177 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:50.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.177 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.177 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942762+0000", 2024-09-16T14:48:50.177 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9150000000000008e-06 2024-09-16T14:48:50.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.178 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.178 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.178 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942777+0000", 2024-09-16T14:48:50.178 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.489e-05 2024-09-16T14:48:50.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.178 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.178 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.178 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942792+0000", 2024-09-16T14:48:50.178 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4252e-05 2024-09-16T14:48:50.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.178 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.178 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.178 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942848+0000", 2024-09-16T14:48:50.178 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6888000000000001e-05 2024-09-16T14:48:50.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.179 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.179 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942859+0000", 2024-09-16T14:48:50.179 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0579e-05 2024-09-16T14:48:50.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.179 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.179 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.946678+0000", 2024-09-16T14:48:50.179 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0038192320000000001 2024-09-16T14:48:50.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.179 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.179 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.946709+0000", 2024-09-16T14:48:50.180 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1013000000000003e-05 2024-09-16T14:48:50.180 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.180 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.180 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.180 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.180 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.0s6 37/27 ECSubWrite(tid=111, reqid=osd.11.0:109, at_version=0'0, trim_to=0'0, roll_forward_to=37'14))", 2024-09-16T14:48:50.180 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.949806+0000", 2024-09-16T14:48:50.180 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 6.1878629869999999, 2024-09-16T14:48:50.180 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0045257789999999997, 2024-09-16T14:48:50.180 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.180 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.180 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.181 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.181 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949806+0000", 2024-09-16T14:48:50.181 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.181 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.181 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949806+0000", 2024-09-16T14:48:50.181 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.182 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.182 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949805+0000", 2024-09-16T14:48:50.182 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:50.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.182 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.182 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949815+0000", 2024-09-16T14:48:50.182 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0012e-05 2024-09-16T14:48:50.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.182 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.183 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949827+0000", 2024-09-16T14:48:50.183 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2516e-05 2024-09-16T14:48:50.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.183 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.183 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.183 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949833+0000", 2024-09-16T14:48:50.183 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1229999999999997e-06 2024-09-16T14:48:50.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.183 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.183 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.183 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949877+0000", 2024-09-16T14:48:50.183 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.3316999999999999e-05 2024-09-16T14:48:50.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.184 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.184 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949905+0000", 2024-09-16T14:48:50.184 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8816999999999999e-05 2024-09-16T14:48:50.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.184 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.184 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.954305+0000", 2024-09-16T14:48:50.184 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0043995060000000001 2024-09-16T14:48:50.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.184 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.184 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.954331+0000", 2024-09-16T14:48:50.184 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6378999999999999e-05 2024-09-16T14:48:50.185 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.185 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.185 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.185 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 37/27 ECSubWrite(tid=127, reqid=osd.3.0:125, at_version=0'0, trim_to=0'0, roll_forward_to=37'43))", 2024-09-16T14:48:50.185 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.950029+0000", 2024-09-16T14:48:50.185 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 6.187639881, 2024-09-16T14:48:50.185 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0068229909999999996, 2024-09-16T14:48:50.185 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.185 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.185 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.185 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.186 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950029+0000", 2024-09-16T14:48:50.186 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.186 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.186 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950029+0000", 2024-09-16T14:48:50.186 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.186 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.186 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950028+0000", 2024-09-16T14:48:50.186 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:50.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.187 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.187 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950040+0000", 2024-09-16T14:48:50.187 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2139e-05 2024-09-16T14:48:50.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.187 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.187 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950053+0000", 2024-09-16T14:48:50.187 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2471e-05 2024-09-16T14:48:50.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.187 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.187 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950060+0000", 2024-09-16T14:48:50.187 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2150000000000004e-06 2024-09-16T14:48:50.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.188 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.188 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950508+0000", 2024-09-16T14:48:50.188 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00044776899999999999 2024-09-16T14:48:50.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.188 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.188 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.950517+0000", 2024-09-16T14:48:50.188 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2960000000000004e-06 2024-09-16T14:48:50.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.188 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.188 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.956820+0000", 2024-09-16T14:48:50.189 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0063035119999999998 2024-09-16T14:48:50.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.189 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.189 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.956852+0000", 2024-09-16T14:48:50.189 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1287e-05 2024-09-16T14:48:50.189 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.189 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.189 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.189 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 38/27 ECSubWrite(tid=113, reqid=client.4500.0:298, at_version=38'12, trim_to=0'0, roll_forward_to=37'11))", 2024-09-16T14:48:50.189 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.933004+0000", 2024-09-16T14:48:50.190 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.2046642399999996, 2024-09-16T14:48:50.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071165250000000003, 2024-09-16T14:48:50.190 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.190 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.190 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.190 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.190 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933004+0000", 2024-09-16T14:48:50.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.190 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.190 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933004+0000", 2024-09-16T14:48:50.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.191 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.191 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933004+0000", 2024-09-16T14:48:50.191 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:50.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.191 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.191 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933424+0000", 2024-09-16T14:48:50.191 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00041965200000000001 2024-09-16T14:48:50.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.191 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.191 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933466+0000", 2024-09-16T14:48:50.192 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2030999999999998e-05 2024-09-16T14:48:50.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.192 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.192 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933474+0000", 2024-09-16T14:48:50.192 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6149999999999997e-06 2024-09-16T14:48:50.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.193 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.193 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933531+0000", 2024-09-16T14:48:50.193 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6857000000000001e-05 2024-09-16T14:48:50.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.193 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.193 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933542+0000", 2024-09-16T14:48:50.193 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.136e-05 2024-09-16T14:48:50.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.194 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.194 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940101+0000", 2024-09-16T14:48:50.194 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0065581220000000004 2024-09-16T14:48:50.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.194 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.194 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940121+0000", 2024-09-16T14:48:50.194 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0290000000000001e-05 2024-09-16T14:48:50.194 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.195 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.195 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.195 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 38/27 ECSubWrite(tid=35, reqid=client.4500.0:302, at_version=38'25, trim_to=0'0, roll_forward_to=37'24))", 2024-09-16T14:48:50.195 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.934315+0000", 2024-09-16T14:48:50.195 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.2033532210000004, 2024-09-16T14:48:50.195 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0054571439999999997, 2024-09-16T14:48:50.195 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.196 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.196 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.196 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.196 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.934315+0000", 2024-09-16T14:48:50.196 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.196 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.196 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.196 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.934315+0000", 2024-09-16T14:48:50.196 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.197 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.197 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.197 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.197 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.934315+0000", 2024-09-16T14:48:50.197 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:50.197 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.197 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.197 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.197 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.934376+0000", 2024-09-16T14:48:50.197 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0514999999999999e-05 2024-09-16T14:48:50.197 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.198 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.198 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.198 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.934397+0000", 2024-09-16T14:48:50.198 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1029000000000001e-05 2024-09-16T14:48:50.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.198 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.198 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.198 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.934405+0000", 2024-09-16T14:48:50.198 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.337e-06 2024-09-16T14:48:50.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.198 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.198 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.198 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.934487+0000", 2024-09-16T14:48:50.199 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2515000000000005e-05 2024-09-16T14:48:50.199 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.199 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.199 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.199 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.934497+0000", 2024-09-16T14:48:50.199 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6339999999999996e-06 2024-09-16T14:48:50.199 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.199 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.199 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.200 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.939746+0000", 2024-09-16T14:48:50.200 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0052494380000000004 2024-09-16T14:48:50.200 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.200 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.200 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.200 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.939772+0000", 2024-09-16T14:48:50.200 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6010999999999999e-05 2024-09-16T14:48:50.200 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.200 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.200 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.200 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.201 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.201 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 38/27 ECSubWrite(tid=114, reqid=client.4500.0:299, at_version=38'13, trim_to=0'0, roll_forward_to=37'11))", 2024-09-16T14:48:50.201 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.935359+0000", 2024-09-16T14:48:50.201 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.2023095780000004, 2024-09-16T14:48:50.201 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.004785381, 2024-09-16T14:48:50.201 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.201 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.201 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.201 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.201 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.201 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.935359+0000", 2024-09-16T14:48:50.201 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.202 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.202 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.202 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.202 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.935359+0000", 2024-09-16T14:48:50.202 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.202 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.202 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.202 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.202 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.935358+0000", 2024-09-16T14:48:50.203 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:50.203 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.203 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.203 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.203 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.935464+0000", 2024-09-16T14:48:50.203 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000105475 2024-09-16T14:48:50.203 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.203 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.203 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.203 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.935481+0000", 2024-09-16T14:48:50.204 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7289e-05 2024-09-16T14:48:50.204 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.204 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.204 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.204 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.935489+0000", 2024-09-16T14:48:50.204 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2879999999999993e-06 2024-09-16T14:48:50.204 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.204 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.204 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.204 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.935611+0000", 2024-09-16T14:48:50.205 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000121688 2024-09-16T14:48:50.205 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.205 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.205 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.205 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.935621+0000", 2024-09-16T14:48:50.205 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0122999999999999e-05 2024-09-16T14:48:50.205 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.205 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.205 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.205 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940131+0000", 2024-09-16T14:48:50.205 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0045100280000000001 2024-09-16T14:48:50.205 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.205 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.206 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.206 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940144+0000", 2024-09-16T14:48:50.206 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3338e-05 2024-09-16T14:48:50.206 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.206 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.206 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.206 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.206 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.206 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 38/27 ECSubWrite(tid=115, reqid=client.4500.0:300, at_version=38'14, trim_to=0'0, roll_forward_to=37'11))", 2024-09-16T14:48:50.206 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.936173+0000", 2024-09-16T14:48:50.206 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.2014952699999997, 2024-09-16T14:48:50.206 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0040208290000000001, 2024-09-16T14:48:50.207 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.207 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.207 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.207 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.207 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.207 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.936173+0000", 2024-09-16T14:48:50.207 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.207 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.207 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.207 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.207 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.936173+0000", 2024-09-16T14:48:50.207 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.207 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.207 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.208 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.208 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.936173+0000", 2024-09-16T14:48:50.208 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:50.208 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.208 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.208 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.208 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.936187+0000", 2024-09-16T14:48:50.208 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4748e-05 2024-09-16T14:48:50.208 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.208 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.208 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.208 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.936198+0000", 2024-09-16T14:48:50.208 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0141e-05 2024-09-16T14:48:50.209 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.209 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.209 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.209 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.936205+0000", 2024-09-16T14:48:50.209 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3490000000000003e-06 2024-09-16T14:48:50.209 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.209 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.209 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.209 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.937311+0000", 2024-09-16T14:48:50.209 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0011062439999999999 2024-09-16T14:48:50.209 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.209 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.209 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.209 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.937321+0000", 2024-09-16T14:48:50.210 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5009999999999993e-06 2024-09-16T14:48:50.210 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.210 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.210 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.210 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940178+0000", 2024-09-16T14:48:50.210 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.002856927 2024-09-16T14:48:50.210 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.210 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.210 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.210 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940194+0000", 2024-09-16T14:48:50.210 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6455000000000001e-05 2024-09-16T14:48:50.210 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.210 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.210 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:316 3.6s0 3:66a618e5:::smithi04040151-18:head [copy-from ver 4] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:48:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.943055+0000", 2024-09-16T14:48:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.1946132900000004, 2024-09-16T14:48:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090504580000000008, 2024-09-16T14:48:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:48:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:48:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 316 2024-09-16T14:48:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.211 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.212 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.212 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.212 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943055+0000", 2024-09-16T14:48:50.212 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.212 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.212 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.212 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.212 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943055+0000", 2024-09-16T14:48:50.212 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.212 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.212 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.212 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.212 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943053+0000", 2024-09-16T14:48:50.213 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:48:50.213 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.213 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.213 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.213 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943063+0000", 2024-09-16T14:48:50.213 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5380000000000008e-06 2024-09-16T14:48:50.213 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.213 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.213 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.213 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943076+0000", 2024-09-16T14:48:50.213 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2908000000000001e-05 2024-09-16T14:48:50.213 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.213 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.214 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.214 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943085+0000", 2024-09-16T14:48:50.214 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7660000000000004e-06 2024-09-16T14:48:50.214 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.214 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.214 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.214 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943140+0000", 2024-09-16T14:48:50.214 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5606999999999998e-05 2024-09-16T14:48:50.214 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.214 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.214 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:48:50.214 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943241+0000", 2024-09-16T14:48:50.214 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010045800000000001 2024-09-16T14:48:50.214 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.215 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.215 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.215 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.949553+0000", 2024-09-16T14:48:50.215 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0063119270000000002 2024-09-16T14:48:50.215 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.215 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.215 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.215 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.951796+0000", 2024-09-16T14:48:50.215 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0022432279999999999 2024-09-16T14:48:50.215 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.215 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.215 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:48:50.215 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.952069+0000", 2024-09-16T14:48:50.216 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00027283199999999998 2024-09-16T14:48:50.216 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.216 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.216 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.216 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.952106+0000", 2024-09-16T14:48:50.216 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.7150999999999997e-05 2024-09-16T14:48:50.216 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.216 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.216 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.216 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.216 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.216 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:317 3.6s0 3:66a618e5:::smithi04040151-18:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:48:50.216 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.943425+0000", 2024-09-16T14:48:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.1942436580000004, 2024-09-16T14:48:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.007553377, 2024-09-16T14:48:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:48:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:48:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:48:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 317 2024-09-16T14:48:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943425+0000", 2024-09-16T14:48:50.217 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.218 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.218 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.218 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.218 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943425+0000", 2024-09-16T14:48:50.218 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.218 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.218 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.218 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.218 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943424+0000", 2024-09-16T14:48:50.218 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:50.218 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.218 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.218 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.219 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943429+0000", 2024-09-16T14:48:50.219 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.3270000000000002e-06 2024-09-16T14:48:50.219 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.219 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.219 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.219 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943446+0000", 2024-09-16T14:48:50.219 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7323e-05 2024-09-16T14:48:50.219 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.219 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.219 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.219 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943454+0000", 2024-09-16T14:48:50.219 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.695e-06 2024-09-16T14:48:50.219 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.220 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.220 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.220 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943534+0000", 2024-09-16T14:48:50.220 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0641000000000003e-05 2024-09-16T14:48:50.220 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.220 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.220 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:48:50.220 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.943621+0000", 2024-09-16T14:48:50.220 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6472999999999997e-05 2024-09-16T14:48:50.220 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.220 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.220 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.221 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.950774+0000", 2024-09-16T14:48:50.221 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071538210000000003 2024-09-16T14:48:50.221 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.221 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.221 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:48:50.221 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.950875+0000", 2024-09-16T14:48:50.221 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010006299999999999 2024-09-16T14:48:50.221 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.221 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.221 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.221 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.950978+0000", 2024-09-16T14:48:50.221 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000103705 2024-09-16T14:48:50.221 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.222 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.222 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.222 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.222 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.222 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 42/27 ECSubWrite(tid=146, reqid=client.4500.0:375, at_version=42'56, trim_to=0'0, roll_forward_to=40'52))", 2024-09-16T14:48:50.222 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.964370+0000", 2024-09-16T14:48:50.222 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.173298865, 2024-09-16T14:48:50.222 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0051849110000000004, 2024-09-16T14:48:50.222 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.222 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.222 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.222 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.222 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.223 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.964370+0000", 2024-09-16T14:48:50.223 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.223 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.223 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.223 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.223 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.964370+0000", 2024-09-16T14:48:50.223 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.223 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.223 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.223 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.223 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.964369+0000", 2024-09-16T14:48:50.223 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:50.223 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.223 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.224 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.224 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.964390+0000", 2024-09-16T14:48:50.224 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1812000000000001e-05 2024-09-16T14:48:50.224 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.224 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.224 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.224 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.964408+0000", 2024-09-16T14:48:50.224 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8168e-05 2024-09-16T14:48:50.224 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.224 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.224 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.224 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.964418+0000", 2024-09-16T14:48:50.224 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.183e-06 2024-09-16T14:48:50.225 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.225 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.225 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.225 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.964486+0000", 2024-09-16T14:48:50.225 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8215999999999999e-05 2024-09-16T14:48:50.225 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.225 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.225 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.225 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.964497+0000", 2024-09-16T14:48:50.225 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0868e-05 2024-09-16T14:48:50.225 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.225 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.226 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.226 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.969520+0000", 2024-09-16T14:48:50.226 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0050237110000000001 2024-09-16T14:48:50.226 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.226 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.226 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.226 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.969555+0000", 2024-09-16T14:48:50.226 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.4119999999999999e-05 2024-09-16T14:48:50.226 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.226 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.226 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.226 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.226 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.227 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 43/27 ECSubWrite(tid=42, reqid=osd.1.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=43'30))", 2024-09-16T14:48:50.227 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:49.851899+0000", 2024-09-16T14:48:50.227 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.285769883, 2024-09-16T14:48:50.227 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0070248460000000004, 2024-09-16T14:48:50.227 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.227 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.227 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.227 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.228 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.228 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851899+0000", 2024-09-16T14:48:50.228 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.228 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.228 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.228 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.228 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851899+0000", 2024-09-16T14:48:50.228 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.228 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.228 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.228 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.228 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851898+0000", 2024-09-16T14:48:50.228 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:50.229 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.229 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.229 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.229 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851910+0000", 2024-09-16T14:48:50.229 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2028000000000001e-05 2024-09-16T14:48:50.229 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.229 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.229 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.229 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851922+0000", 2024-09-16T14:48:50.229 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2435999999999999e-05 2024-09-16T14:48:50.229 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.229 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.230 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.230 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851931+0000", 2024-09-16T14:48:50.230 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5539999999999998e-06 2024-09-16T14:48:50.230 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.230 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.230 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.230 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851988+0000", 2024-09-16T14:48:50.230 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6620999999999999e-05 2024-09-16T14:48:50.230 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.230 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.230 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.231 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851998+0000", 2024-09-16T14:48:50.231 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0548e-05 2024-09-16T14:48:50.231 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.231 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.231 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.231 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.858895+0000", 2024-09-16T14:48:50.231 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0068972950000000003 2024-09-16T14:48:50.231 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.232 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.232 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.232 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.858924+0000", 2024-09-16T14:48:50.232 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8082e-05 2024-09-16T14:48:50.232 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.232 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.232 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.232 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.232 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.232 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 43/27 ECSubWrite(tid=148, reqid=osd.3.0:148, at_version=43'58, trim_to=0'0, roll_forward_to=42'56))", 2024-09-16T14:48:50.233 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:49.859010+0000", 2024-09-16T14:48:50.233 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.27865895000000002, 2024-09-16T14:48:50.233 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044472469999999997, 2024-09-16T14:48:50.233 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.233 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.233 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.233 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.233 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.233 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859010+0000", 2024-09-16T14:48:50.233 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.233 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.233 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.233 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.234 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859010+0000", 2024-09-16T14:48:50.234 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.234 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.234 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.234 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.234 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859009+0000", 2024-09-16T14:48:50.234 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:50.234 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.234 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.234 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.235 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859032+0000", 2024-09-16T14:48:50.235 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.355e-05 2024-09-16T14:48:50.235 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.235 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.235 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.235 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859043+0000", 2024-09-16T14:48:50.235 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0919e-05 2024-09-16T14:48:50.235 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.235 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.235 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.235 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859051+0000", 2024-09-16T14:48:50.235 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8590000000000005e-06 2024-09-16T14:48:50.235 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.236 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.236 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.236 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859118+0000", 2024-09-16T14:48:50.236 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.7018000000000001e-05 2024-09-16T14:48:50.236 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.236 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.236 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.236 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.859129+0000", 2024-09-16T14:48:50.236 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.093e-05 2024-09-16T14:48:50.236 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.236 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.236 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.236 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.863427+0000", 2024-09-16T14:48:50.237 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0042978649999999997 2024-09-16T14:48:50.237 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.237 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.237 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.237 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.863457+0000", 2024-09-16T14:48:50.237 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9767000000000001e-05 2024-09-16T14:48:50.237 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.237 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.237 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.237 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.237 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.237 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 43/27 ECSubWrite(tid=149, reqid=osd.3.0:148, at_version=0'0, trim_to=0'0, roll_forward_to=43'58))", 2024-09-16T14:48:50.237 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:49.863802+0000", 2024-09-16T14:48:50.237 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.27386703000000001, 2024-09-16T14:48:50.238 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0047847469999999998, 2024-09-16T14:48:50.238 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:48:50.238 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:48:50.238 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:48:50.238 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.238 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:48:50.238 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.863802+0000", 2024-09-16T14:48:50.238 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.239 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.239 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:48:50.239 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.863802+0000", 2024-09-16T14:48:50.239 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:48:50.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.239 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.239 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:48:50.239 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.863801+0000", 2024-09-16T14:48:50.239 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:48:50.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.240 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.240 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:48:50.240 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.863814+0000", 2024-09-16T14:48:50.240 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2835e-05 2024-09-16T14:48:50.240 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.240 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.240 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:48:50.240 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.863831+0000", 2024-09-16T14:48:50.240 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7370000000000001e-05 2024-09-16T14:48:50.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.241 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.241 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:48:50.241 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.863840+0000", 2024-09-16T14:48:50.241 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.329e-06 2024-09-16T14:48:50.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.241 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.241 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:48:50.241 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.863892+0000", 2024-09-16T14:48:50.241 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1592999999999999e-05 2024-09-16T14:48:50.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.242 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.242 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:48:50.242 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.863902+0000", 2024-09-16T14:48:50.242 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.029e-05 2024-09-16T14:48:50.242 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.242 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.242 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:48:50.242 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.868546+0000", 2024-09-16T14:48:50.242 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0046436289999999998 2024-09-16T14:48:50.242 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:48:50.243 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:48:50.243 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:48:50.243 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.868586+0000", 2024-09-16T14:48:50.243 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.0538999999999999e-05 2024-09-16T14:48:50.243 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.243 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.243 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.243 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:48:50.243 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:48:50.243 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:48:50.248 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:48:50.248 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:48:50.249 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:48:50.249 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:50.244+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:50.253 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:48:50.253 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:48:50.269 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:48:50.269 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:48:50.285 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:48:50.286 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:48:50.302 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:48:50.302 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:48:50.318 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:48:50.318 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:48:50.335 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:48:50.335 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-16T14:48:50.345 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:50.344+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:50.351 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:48:50.351 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:48:50.361 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:50.361 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:50.368 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:48:50.368 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:48:50.370 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:48:50.385 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:48:50.385 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:48:50.402 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:48:50.402 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:48:50.419 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:48:50.419 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:48:50.471 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:50.470+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:50.491 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:50.491 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:50.500 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:48:50.548 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:50.547+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:51.059 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:51.059 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:51.060 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:48:51.060 INFO:tasks.rados.rados.0.smithi040.stdout:153: done (2 left) 2024-09-16T14:48:51.060 INFO:tasks.rados.rados.0.smithi040.stdout:154: done (1 left) 2024-09-16T14:48:51.060 INFO:tasks.rados.rados.0.smithi040.stdout:156: done (0 left) 2024-09-16T14:48:51.061 INFO:tasks.rados.rados.0.smithi040.stdout:157: snap_create 2024-09-16T14:48:51.061 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:51.058+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:48:51.160 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:51.159+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:51.173 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:51.173 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:51.182 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:48:51.261 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:51.260+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:51.289 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:51.289 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:51.297 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:48:51.362 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:51.361+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:51.404 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:51.404 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:51.413 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:48:51.463 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:51.462+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:51.520 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:51.520 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:51.528 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:48:51.571 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:51.568+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:51.665 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:51.664+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:51.675 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:51.676 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:51.684 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:48:51.766 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:51.765+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:51.790 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:51.791 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:51.799 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:48:51.850 INFO:tasks.rados.rados.0.smithi040.stdout:157: done (0 left) 2024-09-16T14:48:51.850 INFO:tasks.rados.rados.0.smithi040.stdout:158: snap_remove snap 7 2024-09-16T14:48:51.867 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:51.866+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:51.869 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:48:51.914 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:51.915 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:51.925 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:48:51.968 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:51.967+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:48:52.028 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:52.028 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:52.037 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:48:52.069 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:52.068+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:52.170 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:52.169+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:52.179 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:52.179 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:52.188 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:48:52.272 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:52.271+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:52.330 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:52.331 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:52.339 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:48:52.373 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:52.372+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:52.431 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:48:52.441 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:52.442 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:52.450 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:48:52.471 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 6] 2024-09-16T14:48:52.471 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:48:52.471 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-16T14:48:52.471 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T14:48:52.472 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T14:48:52.472 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-16T14:48:52.475 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:52.474+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:52.555 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:52.556 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:52.556 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:48:52.556 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:48:52.564 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:48:52.572 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:48:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:48:52.576 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:52.575+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:52.588 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:48:52.589 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:48:52.598 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:52.598 INFO:teuthology.orchestra.run.smithi111.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T14:48:52.598 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:52.605 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:48:52.605 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:48:52.669 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:52.669 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:52.678 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:48:52.678 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:52.677+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:52.779 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:52.778+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:52.820 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:52.820 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:52.822 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:48:52.823 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:48:52.829 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:48:52.839 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:48:52.839 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:48:52.856 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is unset 2024-09-16T14:48:52.857 INFO:tasks.rados.rados.0.smithi040.stdout:158: done (0 left) 2024-09-16T14:48:52.857 INFO:tasks.rados.rados.0.smithi040.stdout:159: read oid 48 snap -1 2024-09-16T14:48:52.857 INFO:tasks.rados.rados.0.smithi040.stdout:159: expect deleted 2024-09-16T14:48:52.857 INFO:tasks.rados.rados.0.smithi040.stdout:160: snap_create 2024-09-16T14:48:52.857 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:48:52.858 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:48:52.880 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:52.879+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:52.880 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:48:52.881 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:48:52.898 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:48:52.898 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:48:52.916 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:48:52.916 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:48:52.931 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:48:52.931 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:48:52.948 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:48:52.948 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:48:52.975 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:52.975 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:48:52.975 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:48:52.975 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:48:52.976 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.976 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:52.976 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:48:52.976 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 30.833348962999999, 2024-09-16T14:48:52.976 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.223181782, 2024-09-16T14:48:52.976 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:52.976 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:52.976 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:52.976 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:52.976 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:52.976 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 1 2024-09-16T14:48:52.976 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.977 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:52.977 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.977 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:52.977 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:48:52.977 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:52.977 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.977 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.977 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:52.977 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:48:52.977 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:52.977 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.977 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.977 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:52.977 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139495+0000", 2024-09-16T14:48:52.978 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:52.978 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.978 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.978 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:52.978 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139504+0000", 2024-09-16T14:48:52.978 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6119999999999999e-06 2024-09-16T14:48:52.978 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.978 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.978 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:52.978 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139514+0000", 2024-09-16T14:48:52.978 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:48:52.978 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.978 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.978 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:52.979 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139533+0000", 2024-09-16T14:48:52.979 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9573e-05 2024-09-16T14:48:52.979 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.979 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.979 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:52.979 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139585+0000", 2024-09-16T14:48:52.979 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2170999999999999e-05 2024-09-16T14:48:52.979 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.979 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.979 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:48:52.979 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139594+0000", 2024-09-16T14:48:52.979 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.884e-06 2024-09-16T14:48:52.979 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.979 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.980 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:52.980 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359233+0000", 2024-09-16T14:48:52.980 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.21963890899999999 2024-09-16T14:48:52.980 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.980 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.980 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:52.980 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359437+0000", 2024-09-16T14:48:52.980 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00020374999999999999 2024-09-16T14:48:52.980 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.980 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.980 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:52.980 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359611+0000", 2024-09-16T14:48:52.980 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017445899999999999 2024-09-16T14:48:52.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.981 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.981 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:52.981 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.360506+0000", 2024-09-16T14:48:52.981 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00089459299999999995 2024-09-16T14:48:52.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.981 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.981 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:52.981 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362594+0000", 2024-09-16T14:48:52.981 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002087776 2024-09-16T14:48:52.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.981 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.981 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:52.982 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362636+0000", 2024-09-16T14:48:52.982 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1933e-05 2024-09-16T14:48:52.982 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.982 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.982 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:52.982 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362679+0000", 2024-09-16T14:48:52.982 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2877999999999998e-05 2024-09-16T14:48:52.982 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:52.982 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:52.982 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:52.982 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.982 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.982 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:52.983 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:48:52.983 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 30.584637556000001, 2024-09-16T14:48:52.983 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077691490000000004, 2024-09-16T14:48:52.983 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:52.983 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:52.983 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:52.983 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:52.983 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:52.983 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 21 2024-09-16T14:48:52.983 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.983 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:52.983 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.983 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:52.984 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:48:52.984 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:52.984 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.984 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.984 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:52.984 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:48:52.984 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:52.984 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.984 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.984 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:52.984 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:48:52.984 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:52.985 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.985 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.985 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:52.985 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388278+0000", 2024-09-16T14:48:52.985 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0809000000000006e-05 2024-09-16T14:48:52.985 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.985 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.985 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:52.985 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388286+0000", 2024-09-16T14:48:52.985 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.204e-06 2024-09-16T14:48:52.985 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.985 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.985 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:52.986 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388291+0000", 2024-09-16T14:48:52.986 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9349999999999999e-06 2024-09-16T14:48:52.986 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.986 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.986 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:52.986 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388713+0000", 2024-09-16T14:48:52.986 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00042189899999999999 2024-09-16T14:48:52.986 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.986 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.986 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:52.986 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388766+0000", 2024-09-16T14:48:52.986 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2750999999999997e-05 2024-09-16T14:48:52.986 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.987 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.987 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:52.987 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388853+0000", 2024-09-16T14:48:52.987 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6635e-05 2024-09-16T14:48:52.987 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.987 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.987 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:52.987 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.390635+0000", 2024-09-16T14:48:52.987 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0017817950000000001 2024-09-16T14:48:52.987 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.987 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.987 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:52.987 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.395898+0000", 2024-09-16T14:48:52.988 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0052630510000000004 2024-09-16T14:48:52.988 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.988 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.988 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:52.988 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.395941+0000", 2024-09-16T14:48:52.988 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3248999999999999e-05 2024-09-16T14:48:52.988 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.988 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.988 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:52.988 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.395977+0000", 2024-09-16T14:48:52.988 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6501000000000002e-05 2024-09-16T14:48:52.989 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:52.989 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:52.989 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:52.989 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.989 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.989 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:52.989 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:48:52.992 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 30.584546547999999, 2024-09-16T14:48:52.992 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077998870000000001, 2024-09-16T14:48:52.992 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:52.992 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:52.992 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:52.992 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:52.992 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:52.992 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 22 2024-09-16T14:48:52.992 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.993 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:52.993 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.993 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:52.993 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:48:52.993 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:52.993 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.993 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.993 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:52.993 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:48:52.993 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:52.993 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.993 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.994 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:52.994 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:48:52.994 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:52.994 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.994 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.994 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:52.994 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388311+0000", 2024-09-16T14:48:52.994 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2853e-05 2024-09-16T14:48:52.994 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.994 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.994 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:52.994 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388318+0000", 2024-09-16T14:48:52.994 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7549999999999997e-06 2024-09-16T14:48:52.995 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.995 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.995 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:52.995 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388322+0000", 2024-09-16T14:48:52.995 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5889999999999999e-06 2024-09-16T14:48:52.995 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.995 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.995 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:52.995 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.389160+0000", 2024-09-16T14:48:52.995 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000837877 2024-09-16T14:48:52.995 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.995 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.995 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:52.995 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.389230+0000", 2024-09-16T14:48:52.996 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0551999999999998e-05 2024-09-16T14:48:52.996 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.996 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.996 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:52.996 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.389371+0000", 2024-09-16T14:48:52.996 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014102099999999999 2024-09-16T14:48:52.996 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.996 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.996 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:52.996 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.390881+0000", 2024-09-16T14:48:52.996 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00151004 2024-09-16T14:48:52.996 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.996 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.997 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:52.997 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.396022+0000", 2024-09-16T14:48:52.997 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005140716 2024-09-16T14:48:52.997 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.997 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.997 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:52.997 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.396074+0000", 2024-09-16T14:48:52.998 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2089999999999998e-05 2024-09-16T14:48:52.998 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.998 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.998 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:52.998 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.396099+0000", 2024-09-16T14:48:52.998 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5038e-05 2024-09-16T14:48:52.998 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:52.998 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:52.998 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:52.998 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:52.998 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:52.999 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:52.999 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.406628+0000", 2024-09-16T14:48:52.999 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 30.566217871999999, 2024-09-16T14:48:52.999 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0071159200000000004, 2024-09-16T14:48:52.999 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:52.999 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:52.999 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:52.999 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.000 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.000 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 34 2024-09-16T14:48:53.000 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.000 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.000 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.000 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.000 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406628+0000", 2024-09-16T14:48:53.000 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.000 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.000 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.000 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.000 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406628+0000", 2024-09-16T14:48:53.001 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.001 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.001 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.001 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.001 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406628+0000", 2024-09-16T14:48:53.001 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:48:53.001 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.001 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.001 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.001 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406630+0000", 2024-09-16T14:48:53.001 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.266e-06 2024-09-16T14:48:53.001 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.001 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.002 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.002 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406636+0000", 2024-09-16T14:48:53.002 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.7810000000000002e-06 2024-09-16T14:48:53.002 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.002 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.002 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.002 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406639+0000", 2024-09-16T14:48:53.002 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8449999999999999e-06 2024-09-16T14:48:53.002 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.003 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.003 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.003 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.408361+0000", 2024-09-16T14:48:53.003 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001722847 2024-09-16T14:48:53.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.003 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.003 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.003 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.408406+0000", 2024-09-16T14:48:53.003 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4984999999999997e-05 2024-09-16T14:48:53.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.003 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.004 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.004 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.408486+0000", 2024-09-16T14:48:53.004 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0028e-05 2024-09-16T14:48:53.004 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.004 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.004 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.004 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.410024+0000", 2024-09-16T14:48:53.004 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015372750000000001 2024-09-16T14:48:53.004 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.004 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.004 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.004 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413696+0000", 2024-09-16T14:48:53.004 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036724959999999999 2024-09-16T14:48:53.005 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.005 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.005 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.005 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413724+0000", 2024-09-16T14:48:53.005 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7563e-05 2024-09-16T14:48:53.005 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.005 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.005 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.005 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413744+0000", 2024-09-16T14:48:53.005 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0071000000000001e-05 2024-09-16T14:48:53.005 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.005 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.006 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.006 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.006 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:53.006 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.406644+0000", 2024-09-16T14:48:53.006 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 30.566201679999999, 2024-09-16T14:48:53.006 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0071766360000000001, 2024-09-16T14:48:53.006 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.006 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.006 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.006 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.006 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.006 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 35 2024-09-16T14:48:53.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.007 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.007 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.007 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.007 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406644+0000", 2024-09-16T14:48:53.007 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.007 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.007 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.007 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.007 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406644+0000", 2024-09-16T14:48:53.007 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.007 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.007 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.007 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.007 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406644+0000", 2024-09-16T14:48:53.008 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.008 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.008 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.008 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.008 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406674+0000", 2024-09-16T14:48:53.008 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0513000000000001e-05 2024-09-16T14:48:53.008 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.008 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.008 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.008 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406679+0000", 2024-09-16T14:48:53.009 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6550000000000003e-06 2024-09-16T14:48:53.009 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.009 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.009 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.009 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406683+0000", 2024-09-16T14:48:53.009 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.7500000000000001e-06 2024-09-16T14:48:53.009 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.009 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.009 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.009 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.408720+0000", 2024-09-16T14:48:53.009 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0020375860000000001 2024-09-16T14:48:53.010 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.010 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.010 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.010 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.408800+0000", 2024-09-16T14:48:53.010 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9350000000000004e-05 2024-09-16T14:48:53.010 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.010 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.010 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.010 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.408938+0000", 2024-09-16T14:48:53.010 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00013873999999999999 2024-09-16T14:48:53.010 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.010 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.011 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.011 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.410027+0000", 2024-09-16T14:48:53.011 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001088969 2024-09-16T14:48:53.011 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.011 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.011 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.011 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413779+0000", 2024-09-16T14:48:53.011 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003751561 2024-09-16T14:48:53.011 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.011 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.011 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.011 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413803+0000", 2024-09-16T14:48:53.011 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3907e-05 2024-09-16T14:48:53.011 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.012 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.012 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.012 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413821+0000", 2024-09-16T14:48:53.012 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7944999999999998e-05 2024-09-16T14:48:53.012 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.012 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.012 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.012 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.012 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.012 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:53.012 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:48:53.013 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 30.566159672000001, 2024-09-16T14:48:53.013 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0072029349999999997, 2024-09-16T14:48:53.013 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.013 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.013 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.013 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.013 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.013 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 36 2024-09-16T14:48:53.013 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.013 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.013 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.013 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.013 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:48:53.014 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.014 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.014 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.014 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.014 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:48:53.014 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.014 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.014 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.014 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.014 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:48:53.014 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.014 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.014 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.015 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.015 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406695+0000", 2024-09-16T14:48:53.015 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1530000000000003e-06 2024-09-16T14:48:53.015 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.015 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.015 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.015 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406700+0000", 2024-09-16T14:48:53.015 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7079999999999996e-06 2024-09-16T14:48:53.015 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.015 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.015 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.015 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406702+0000", 2024-09-16T14:48:53.016 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7410000000000001e-06 2024-09-16T14:48:53.016 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.016 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.016 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.016 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409398+0000", 2024-09-16T14:48:53.016 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0026957420000000001 2024-09-16T14:48:53.016 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.016 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.016 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.016 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409441+0000", 2024-09-16T14:48:53.016 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3071999999999997e-05 2024-09-16T14:48:53.016 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.017 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.017 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.017 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409512+0000", 2024-09-16T14:48:53.017 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0871999999999995e-05 2024-09-16T14:48:53.017 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.017 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.017 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.017 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.410156+0000", 2024-09-16T14:48:53.017 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00064395800000000003 2024-09-16T14:48:53.017 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.017 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.017 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.017 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413850+0000", 2024-09-16T14:48:53.018 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036938959999999999 2024-09-16T14:48:53.018 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.018 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.018 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.018 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413872+0000", 2024-09-16T14:48:53.018 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2195e-05 2024-09-16T14:48:53.018 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.018 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.018 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.018 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413889+0000", 2024-09-16T14:48:53.018 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6948000000000001e-05 2024-09-16T14:48:53.018 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.019 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.019 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.019 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.019 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.019 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:53.019 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.406706+0000", 2024-09-16T14:48:53.019 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 30.566140094000001, 2024-09-16T14:48:53.019 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0072520049999999997, 2024-09-16T14:48:53.019 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.019 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.019 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.019 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.020 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.020 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 37 2024-09-16T14:48:53.020 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.020 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.020 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.020 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.020 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406706+0000", 2024-09-16T14:48:53.020 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.020 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.020 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.020 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.020 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406706+0000", 2024-09-16T14:48:53.020 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.021 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.021 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.021 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.021 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406705+0000", 2024-09-16T14:48:53.021 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.021 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.021 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.021 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.021 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406708+0000", 2024-09-16T14:48:53.021 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.57e-06 2024-09-16T14:48:53.021 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.021 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.021 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.022 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406712+0000", 2024-09-16T14:48:53.022 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.425e-06 2024-09-16T14:48:53.022 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.022 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.022 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.022 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406715+0000", 2024-09-16T14:48:53.022 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7700000000000002e-06 2024-09-16T14:48:53.022 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.022 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.022 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.022 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409721+0000", 2024-09-16T14:48:53.022 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003006327 2024-09-16T14:48:53.022 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.023 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.023 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.023 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409770+0000", 2024-09-16T14:48:53.023 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8022000000000002e-05 2024-09-16T14:48:53.023 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.023 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.023 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.023 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409853+0000", 2024-09-16T14:48:53.023 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3604999999999997e-05 2024-09-16T14:48:53.023 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.023 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.023 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.023 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.410391+0000", 2024-09-16T14:48:53.024 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00053785400000000004 2024-09-16T14:48:53.024 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.024 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.024 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.024 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413911+0000", 2024-09-16T14:48:53.024 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0035205240000000001 2024-09-16T14:48:53.024 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.024 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.024 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.024 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413939+0000", 2024-09-16T14:48:53.024 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7032000000000001e-05 2024-09-16T14:48:53.024 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.025 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.025 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.025 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413958+0000", 2024-09-16T14:48:53.025 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9147000000000002e-05 2024-09-16T14:48:53.025 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.025 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.025 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.025 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.025 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.025 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:53.025 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.620547+0000", 2024-09-16T14:48:53.025 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 30.352298463, 2024-09-16T14:48:53.025 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0071149270000000001, 2024-09-16T14:48:53.026 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.026 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.026 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.026 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.026 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.026 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 115 2024-09-16T14:48:53.026 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.026 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.026 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.026 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.026 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620547+0000", 2024-09-16T14:48:53.026 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.026 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620547+0000", 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620547+0000", 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620616+0000", 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9244000000000003e-05 2024-09-16T14:48:53.027 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.028 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.028 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.028 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620624+0000", 2024-09-16T14:48:53.028 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0979999999999994e-06 2024-09-16T14:48:53.028 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.028 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.028 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.028 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620631+0000", 2024-09-16T14:48:53.028 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.81e-06 2024-09-16T14:48:53.028 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.028 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.028 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.028 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625202+0000", 2024-09-16T14:48:53.028 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045710350000000002 2024-09-16T14:48:53.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.029 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.029 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.029 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625255+0000", 2024-09-16T14:48:53.029 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3303000000000001e-05 2024-09-16T14:48:53.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.029 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.029 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.029 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625330+0000", 2024-09-16T14:48:53.029 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4486000000000005e-05 2024-09-16T14:48:53.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.029 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.029 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627422+0000", 2024-09-16T14:48:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0020923259999999998 2024-09-16T14:48:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627596+0000", 2024-09-16T14:48:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017343400000000001 2024-09-16T14:48:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627626+0000", 2024-09-16T14:48:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0094999999999999e-05 2024-09-16T14:48:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.031 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.031 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627662+0000", 2024-09-16T14:48:53.031 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6603000000000002e-05 2024-09-16T14:48:53.031 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.031 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.031 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.031 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.031 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.031 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:53.031 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.620638+0000", 2024-09-16T14:48:53.031 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 30.352207991, 2024-09-16T14:48:53.031 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0078265469999999997, 2024-09-16T14:48:53.031 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.032 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.032 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.032 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.032 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.032 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 116 2024-09-16T14:48:53.032 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.032 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.032 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.032 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.032 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620638+0000", 2024-09-16T14:48:53.032 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.032 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.032 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.033 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.033 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620638+0000", 2024-09-16T14:48:53.033 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.033 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.033 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.033 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.033 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620637+0000", 2024-09-16T14:48:53.033 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.033 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.033 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.033 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.033 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620641+0000", 2024-09-16T14:48:53.033 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9360000000000001e-06 2024-09-16T14:48:53.033 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.034 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.034 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.034 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620647+0000", 2024-09-16T14:48:53.034 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3339999999999998e-06 2024-09-16T14:48:53.034 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.034 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.034 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.034 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620652+0000", 2024-09-16T14:48:53.034 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0950000000000003e-06 2024-09-16T14:48:53.034 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.034 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.034 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.034 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625724+0000", 2024-09-16T14:48:53.034 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050719370000000003 2024-09-16T14:48:53.035 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.035 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.035 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.035 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625768+0000", 2024-09-16T14:48:53.035 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4039000000000003e-05 2024-09-16T14:48:53.035 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.035 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.035 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.035 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625836+0000", 2024-09-16T14:48:53.035 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7181999999999995e-05 2024-09-16T14:48:53.035 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.035 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.035 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.035 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627424+0000", 2024-09-16T14:48:53.036 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015887119999999999 2024-09-16T14:48:53.036 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.036 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.036 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.036 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628421+0000", 2024-09-16T14:48:53.036 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00099648699999999989 2024-09-16T14:48:53.036 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.036 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.036 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.036 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628446+0000", 2024-09-16T14:48:53.036 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5307999999999999e-05 2024-09-16T14:48:53.036 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.036 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.037 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.037 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628464+0000", 2024-09-16T14:48:53.037 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8170999999999999e-05 2024-09-16T14:48:53.037 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.037 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.037 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.037 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.037 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.037 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:53.037 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:48:53.038 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 30.352188266999999, 2024-09-16T14:48:53.038 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0079223479999999992, 2024-09-16T14:48:53.038 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.038 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.038 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.038 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.038 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.038 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 117 2024-09-16T14:48:53.038 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.038 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.038 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.038 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.039 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:48:53.039 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.039 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.039 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.039 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.039 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:48:53.039 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.039 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.039 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.039 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.040 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:48:53.040 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.040 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.040 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.040 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.040 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620662+0000", 2024-09-16T14:48:53.040 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7609999999999998e-06 2024-09-16T14:48:53.040 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.040 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.040 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.041 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620668+0000", 2024-09-16T14:48:53.041 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.84e-06 2024-09-16T14:48:53.041 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.041 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.041 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.041 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620673+0000", 2024-09-16T14:48:53.042 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.468e-06 2024-09-16T14:48:53.042 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.042 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.042 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.042 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.626022+0000", 2024-09-16T14:48:53.042 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053484680000000003 2024-09-16T14:48:53.042 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.042 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.042 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.043 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.626071+0000", 2024-09-16T14:48:53.043 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9854000000000001e-05 2024-09-16T14:48:53.043 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.043 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.043 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.043 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.626145+0000", 2024-09-16T14:48:53.043 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.3159999999999997e-05 2024-09-16T14:48:53.043 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.043 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.044 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.044 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627426+0000", 2024-09-16T14:48:53.044 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001281847 2024-09-16T14:48:53.044 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.044 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.044 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.044 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628501+0000", 2024-09-16T14:48:53.044 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0010748019999999999 2024-09-16T14:48:53.044 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.044 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.044 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.044 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628541+0000", 2024-09-16T14:48:53.044 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9598999999999998e-05 2024-09-16T14:48:53.045 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.045 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.045 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.045 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628580+0000", 2024-09-16T14:48:53.045 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8977999999999998e-05 2024-09-16T14:48:53.045 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.045 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.045 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.045 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.045 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.045 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:53.045 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:48:53.045 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.580921373999999, 2024-09-16T14:48:53.046 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0081240189999999997, 2024-09-16T14:48:53.046 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.046 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.046 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.046 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.046 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.046 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 158 2024-09-16T14:48:53.046 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.046 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.046 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.046 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.046 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:48:53.046 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.046 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.047 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.047 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.047 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:48:53.047 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.047 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.047 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.047 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.047 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:48:53.047 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.047 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.047 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.047 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.048 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392007+0000", 2024-09-16T14:48:53.048 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3108000000000005e-05 2024-09-16T14:48:53.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.048 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.048 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.048 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392014+0000", 2024-09-16T14:48:53.048 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6170000000000003e-06 2024-09-16T14:48:53.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.048 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.048 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.048 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392017+0000", 2024-09-16T14:48:53.048 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3589999999999999e-06 2024-09-16T14:48:53.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.049 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.049 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.049 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394165+0000", 2024-09-16T14:48:53.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021481270000000001 2024-09-16T14:48:53.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.049 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.049 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.049 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394238+0000", 2024-09-16T14:48:53.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2366999999999993e-05 2024-09-16T14:48:53.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.049 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.049 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.049 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394347+0000", 2024-09-16T14:48:53.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000109907 2024-09-16T14:48:53.050 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.050 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.050 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.050 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396066+0000", 2024-09-16T14:48:53.050 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001718731 2024-09-16T14:48:53.050 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.050 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.050 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.050 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.399984+0000", 2024-09-16T14:48:53.050 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0039181850000000002 2024-09-16T14:48:53.050 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.050 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.050 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.050 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400027+0000", 2024-09-16T14:48:53.051 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2184000000000001e-05 2024-09-16T14:48:53.051 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.051 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.051 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.051 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400048+0000", 2024-09-16T14:48:53.051 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1897999999999999e-05 2024-09-16T14:48:53.051 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.051 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.051 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.051 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.051 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.051 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:53.051 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:48:53.051 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.580824580000002, 2024-09-16T14:48:53.052 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0081462830000000007, 2024-09-16T14:48:53.052 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.052 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.052 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.052 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.052 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.052 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 159 2024-09-16T14:48:53.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.052 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.052 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.052 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.052 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:48:53.052 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.053 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.053 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.053 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:48:53.053 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.053 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.053 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.053 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.053 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:48:53.053 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.053 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.053 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.053 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.053 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392023+0000", 2024-09-16T14:48:53.053 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:48:53.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.054 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.054 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.054 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392035+0000", 2024-09-16T14:48:53.054 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2207e-05 2024-09-16T14:48:53.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.054 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.054 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.054 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392040+0000", 2024-09-16T14:48:53.054 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7450000000000003e-06 2024-09-16T14:48:53.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.054 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.054 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.054 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394756+0000", 2024-09-16T14:48:53.055 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0027154010000000001 2024-09-16T14:48:53.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.055 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.055 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.055 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394799+0000", 2024-09-16T14:48:53.055 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3921999999999997e-05 2024-09-16T14:48:53.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.055 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.055 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.055 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394867+0000", 2024-09-16T14:48:53.055 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7770999999999999e-05 2024-09-16T14:48:53.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.055 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.056 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.056 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396922+0000", 2024-09-16T14:48:53.056 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0020549819999999999 2024-09-16T14:48:53.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.056 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.056 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.056 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400094+0000", 2024-09-16T14:48:53.056 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031720979999999999 2024-09-16T14:48:53.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.056 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.056 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.056 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400134+0000", 2024-09-16T14:48:53.056 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9449999999999997e-05 2024-09-16T14:48:53.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.057 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.057 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.057 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400167+0000", 2024-09-16T14:48:53.057 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3711000000000003e-05 2024-09-16T14:48:53.057 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.057 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.057 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.057 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.057 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.057 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:160 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:53.057 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:48:53.057 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.580802346999999, 2024-09-16T14:48:53.057 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082045659999999999, 2024-09-16T14:48:53.057 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.058 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.058 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.058 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.058 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.058 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 160 2024-09-16T14:48:53.058 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.058 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.058 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.058 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.058 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:48:53.058 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.058 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.058 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.058 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.059 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:48:53.059 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.059 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.059 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.059 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:48:53.059 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.059 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.059 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.059 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392045+0000", 2024-09-16T14:48:53.059 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.283e-06 2024-09-16T14:48:53.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.059 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.060 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.060 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392050+0000", 2024-09-16T14:48:53.060 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1099999999999996e-06 2024-09-16T14:48:53.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.060 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.060 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.060 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392052+0000", 2024-09-16T14:48:53.060 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4660000000000002e-06 2024-09-16T14:48:53.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.060 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.060 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.060 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395234+0000", 2024-09-16T14:48:53.060 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031823260000000001 2024-09-16T14:48:53.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.061 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.061 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.061 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395283+0000", 2024-09-16T14:48:53.061 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8347000000000003e-05 2024-09-16T14:48:53.061 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.061 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.061 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.061 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395364+0000", 2024-09-16T14:48:53.061 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0853999999999995e-05 2024-09-16T14:48:53.061 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.061 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.061 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.061 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396928+0000", 2024-09-16T14:48:53.061 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015642239999999999 2024-09-16T14:48:53.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.062 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.062 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.062 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400202+0000", 2024-09-16T14:48:53.062 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032747359999999999 2024-09-16T14:48:53.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.062 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.062 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.062 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400227+0000", 2024-09-16T14:48:53.062 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4079000000000001e-05 2024-09-16T14:48:53.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.062 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.063 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.063 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400248+0000", 2024-09-16T14:48:53.063 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1444e-05 2024-09-16T14:48:53.063 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.063 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.063 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.063 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.063 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.063 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:161 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:53.063 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:48:53.063 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.580790912000001, 2024-09-16T14:48:53.063 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082675720000000008, 2024-09-16T14:48:53.063 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.063 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.064 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.064 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.064 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.064 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 161 2024-09-16T14:48:53.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.064 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.064 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.064 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.064 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:48:53.064 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.065 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.065 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.065 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:48:53.065 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.065 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.065 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.065 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:48:53.065 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:48:53.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.065 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.065 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.066 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392122+0000", 2024-09-16T14:48:53.066 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7698000000000001e-05 2024-09-16T14:48:53.066 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.066 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.066 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.066 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392136+0000", 2024-09-16T14:48:53.066 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3464e-05 2024-09-16T14:48:53.066 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.066 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.066 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.066 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392140+0000", 2024-09-16T14:48:53.067 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0790000000000002e-06 2024-09-16T14:48:53.067 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.067 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.067 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.067 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395579+0000", 2024-09-16T14:48:53.067 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034393219999999999 2024-09-16T14:48:53.067 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.067 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.067 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.067 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395624+0000", 2024-09-16T14:48:53.067 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4946e-05 2024-09-16T14:48:53.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.068 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.068 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.068 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395692+0000", 2024-09-16T14:48:53.068 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7843999999999996e-05 2024-09-16T14:48:53.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.068 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.068 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.068 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397834+0000", 2024-09-16T14:48:53.068 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021421529999999999 2024-09-16T14:48:53.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.068 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.068 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.069 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400281+0000", 2024-09-16T14:48:53.069 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002447238 2024-09-16T14:48:53.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.069 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.069 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.069 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400305+0000", 2024-09-16T14:48:53.069 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3428999999999999e-05 2024-09-16T14:48:53.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.069 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.069 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.070 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400322+0000", 2024-09-16T14:48:53.070 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7632e-05 2024-09-16T14:48:53.070 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.070 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.070 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.070 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.070 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.070 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:53.070 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:48:53.070 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.580701633, 2024-09-16T14:48:53.070 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.008260814, 2024-09-16T14:48:53.071 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.071 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.071 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.071 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.071 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.071 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 162 2024-09-16T14:48:53.071 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.071 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.071 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.071 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.071 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:48:53.071 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.072 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.072 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.072 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:48:53.072 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.072 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.072 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.072 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:48:53.072 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.073 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.073 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.073 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.073 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392146+0000", 2024-09-16T14:48:53.073 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.526e-06 2024-09-16T14:48:53.073 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.073 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.073 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.073 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392151+0000", 2024-09-16T14:48:53.073 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2980000000000002e-06 2024-09-16T14:48:53.073 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.074 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.074 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.074 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392153+0000", 2024-09-16T14:48:53.074 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5409999999999999e-06 2024-09-16T14:48:53.074 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.074 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.074 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.074 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396103+0000", 2024-09-16T14:48:53.074 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0039497559999999996 2024-09-16T14:48:53.074 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.074 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.075 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.075 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396188+0000", 2024-09-16T14:48:53.075 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5038999999999997e-05 2024-09-16T14:48:53.075 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.075 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.075 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.075 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396315+0000", 2024-09-16T14:48:53.075 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000126895 2024-09-16T14:48:53.075 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.075 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.076 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.076 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397839+0000", 2024-09-16T14:48:53.076 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015246859999999999 2024-09-16T14:48:53.076 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.076 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.076 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.076 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400356+0000", 2024-09-16T14:48:53.076 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002516524 2024-09-16T14:48:53.077 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.077 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.077 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.077 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400379+0000", 2024-09-16T14:48:53.077 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2793e-05 2024-09-16T14:48:53.077 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.077 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.077 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.077 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400405+0000", 2024-09-16T14:48:53.077 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6140999999999999e-05 2024-09-16T14:48:53.077 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.077 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.077 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.078 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.078 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.078 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:53.078 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:48:53.078 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.580690241999999, 2024-09-16T14:48:53.078 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083183530000000006, 2024-09-16T14:48:53.078 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.078 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.078 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.078 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.078 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.078 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 163 2024-09-16T14:48:53.078 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.079 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.079 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.079 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.079 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:48:53.079 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.079 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.079 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.079 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.079 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:48:53.079 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.079 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.079 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.079 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.079 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392155+0000", 2024-09-16T14:48:53.080 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:48:53.080 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.080 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.080 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.080 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392159+0000", 2024-09-16T14:48:53.080 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0459999999999999e-06 2024-09-16T14:48:53.080 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.080 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.080 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.080 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392162+0000", 2024-09-16T14:48:53.080 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:48:53.080 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.080 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.080 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.081 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392165+0000", 2024-09-16T14:48:53.081 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.424e-06 2024-09-16T14:48:53.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.081 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.081 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.081 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396578+0000", 2024-09-16T14:48:53.081 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044136139999999997 2024-09-16T14:48:53.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.081 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.081 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.081 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396625+0000", 2024-09-16T14:48:53.081 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6486000000000002e-05 2024-09-16T14:48:53.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.081 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.082 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.082 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396694+0000", 2024-09-16T14:48:53.082 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9049999999999998e-05 2024-09-16T14:48:53.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.082 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.082 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.082 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397843+0000", 2024-09-16T14:48:53.082 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011485740000000001 2024-09-16T14:48:53.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.082 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.082 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.082 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400438+0000", 2024-09-16T14:48:53.082 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025950909999999999 2024-09-16T14:48:53.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.083 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.083 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.083 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400458+0000", 2024-09-16T14:48:53.083 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0248e-05 2024-09-16T14:48:53.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.083 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.083 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.083 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400474+0000", 2024-09-16T14:48:53.083 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5999999999999999e-05 2024-09-16T14:48:53.083 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.083 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.083 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.084 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.084 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:53.084 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:48:53.084 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.580678644999999, 2024-09-16T14:48:53.084 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083881519999999994, 2024-09-16T14:48:53.084 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.084 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.084 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.084 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.084 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.084 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 164 2024-09-16T14:48:53.084 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.084 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:48:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:48:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.086 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.086 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:48:53.086 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:48:53.086 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.086 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.086 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.086 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392212+0000", 2024-09-16T14:48:53.086 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5108999999999998e-05 2024-09-16T14:48:53.086 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.086 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.086 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.087 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392216+0000", 2024-09-16T14:48:53.087 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2549999999999999e-06 2024-09-16T14:48:53.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.087 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.087 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.087 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392220+0000", 2024-09-16T14:48:53.087 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3850000000000002e-06 2024-09-16T14:48:53.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.087 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.087 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.088 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396961+0000", 2024-09-16T14:48:53.088 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.004740976 2024-09-16T14:48:53.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.088 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.088 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.088 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397029+0000", 2024-09-16T14:48:53.088 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7889e-05 2024-09-16T14:48:53.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.088 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.088 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.088 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397131+0000", 2024-09-16T14:48:53.089 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000102315 2024-09-16T14:48:53.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.089 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.089 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.089 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400259+0000", 2024-09-16T14:48:53.089 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031285229999999998 2024-09-16T14:48:53.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.089 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.089 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.089 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400501+0000", 2024-09-16T14:48:53.089 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00024211999999999999 2024-09-16T14:48:53.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.089 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.089 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.090 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400530+0000", 2024-09-16T14:48:53.090 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8935999999999999e-05 2024-09-16T14:48:53.090 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.090 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.090 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.090 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400555+0000", 2024-09-16T14:48:53.090 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4861000000000001e-05 2024-09-16T14:48:53.090 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.090 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.090 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.090 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.090 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.090 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:53.091 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:48:53.091 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.58062288, 2024-09-16T14:48:53.091 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010593673, 2024-09-16T14:48:53.091 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.091 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.091 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.091 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:53.091 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:53.091 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 165 2024-09-16T14:48:53.091 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.091 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.091 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.091 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.091 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:48:53.092 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.092 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.092 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.092 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:48:53.092 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.092 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.092 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.092 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:48:53.092 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.092 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.092 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.093 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392226+0000", 2024-09-16T14:48:53.093 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2789999999999999e-06 2024-09-16T14:48:53.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.093 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.093 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.093 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392234+0000", 2024-09-16T14:48:53.093 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2369999999999992e-06 2024-09-16T14:48:53.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.093 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.093 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.093 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392237+0000", 2024-09-16T14:48:53.093 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8619999999999999e-06 2024-09-16T14:48:53.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.094 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.094 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.094 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397503+0000", 2024-09-16T14:48:53.094 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0052656220000000002 2024-09-16T14:48:53.094 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.094 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.094 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.094 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397558+0000", 2024-09-16T14:48:53.094 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5386999999999997e-05 2024-09-16T14:48:53.094 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.094 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.094 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:53.094 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397628+0000", 2024-09-16T14:48:53.094 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0119999999999999e-05 2024-09-16T14:48:53.095 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.095 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.095 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:53.095 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402533+0000", 2024-09-16T14:48:53.095 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049046200000000002 2024-09-16T14:48:53.095 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.095 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.095 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:53.095 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402778+0000", 2024-09-16T14:48:53.095 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000245348 2024-09-16T14:48:53.095 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.095 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.095 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.096 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402800+0000", 2024-09-16T14:48:53.096 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2277e-05 2024-09-16T14:48:53.096 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.096 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.096 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.096 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402817+0000", 2024-09-16T14:48:53.096 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6265000000000001e-05 2024-09-16T14:48:53.096 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.096 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.096 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.096 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.096 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.097 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:309 3.ds0 3:b9c0d236:::smithi04040151-19:head [append 0~737280] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:48:53.097 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:48:53.097 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 8.0334317439999996, 2024-09-16T14:48:53.097 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01003656, 2024-09-16T14:48:53.097 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.097 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:53.097 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.097 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:48:53.097 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:53.097 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 309 2024-09-16T14:48:53.097 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.097 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.097 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.098 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.098 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:48:53.098 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.098 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.098 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.098 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:48:53.098 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.098 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.099 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.099 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939413+0000", 2024-09-16T14:48:53.099 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:53.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.099 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.099 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.099 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942226+0000", 2024-09-16T14:48:53.099 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0028130450000000001 2024-09-16T14:48:53.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.099 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.099 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.100 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942238+0000", 2024-09-16T14:48:53.100 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2639e-05 2024-09-16T14:48:53.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.100 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.100 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.100 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942245+0000", 2024-09-16T14:48:53.100 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3840000000000002e-06 2024-09-16T14:48:53.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.100 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.100 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.100 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942288+0000", 2024-09-16T14:48:53.100 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3186000000000003e-05 2024-09-16T14:48:53.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.101 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.101 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.101 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942368+0000", 2024-09-16T14:48:53.101 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0162999999999998e-05 2024-09-16T14:48:53.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.101 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.101 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.101 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943660+0000", 2024-09-16T14:48:53.101 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001291698 2024-09-16T14:48:53.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.102 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.102 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.102 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.947440+0000", 2024-09-16T14:48:53.102 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003780079 2024-09-16T14:48:53.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.102 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.102 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:53.102 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.949391+0000", 2024-09-16T14:48:53.102 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0019512990000000001 2024-09-16T14:48:53.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.102 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.102 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.949451+0000", 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9404000000000002e-05 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:312 3.ds0 3:b9c0d236:::smithi04040151-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 8.0294522290000003, 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084967020000000001, 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:53.103 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:48:53.104 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:53.104 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 312 2024-09-16T14:48:53.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.104 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:53.104 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.104 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:53.104 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:48:53.104 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.104 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.104 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:53.104 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:48:53.104 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:53.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.105 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.105 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:53.105 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943393+0000", 2024-09-16T14:48:53.105 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.105 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.105 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:53.105 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943396+0000", 2024-09-16T14:48:53.105 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6079999999999998e-06 2024-09-16T14:48:53.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.105 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.106 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:53.106 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943402+0000", 2024-09-16T14:48:53.106 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6100000000000002e-06 2024-09-16T14:48:53.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.106 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.106 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.106 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943413+0000", 2024-09-16T14:48:53.106 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1178e-05 2024-09-16T14:48:53.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.106 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.106 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.106 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.946818+0000", 2024-09-16T14:48:53.107 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034043049999999998 2024-09-16T14:48:53.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.107 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.107 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:48:53.107 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.946922+0000", 2024-09-16T14:48:53.107 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000104076 2024-09-16T14:48:53.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.107 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.107 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:53.107 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.950275+0000", 2024-09-16T14:48:53.107 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0033530220000000002 2024-09-16T14:48:53.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.108 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.108 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:53.108 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.950357+0000", 2024-09-16T14:48:53.108 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2087000000000002e-05 2024-09-16T14:48:53.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:53.108 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:53.108 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:53.108 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.951890+0000", 2024-09-16T14:48:53.108 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001533333 2024-09-16T14:48:53.108 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.108 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.109 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.109 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:53.109 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:53.109 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:53.111 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:48:53.111 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:53.109+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:48:53.211 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:53.210+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:53.225 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:53.226 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:53.226 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:53.235 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:48:53.312 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:53.311+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:53.379 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:53.379 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:53.379 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:53.379 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:53.379 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:53.388 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:48:53.431 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:53.430+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:53.493 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:53.493 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:53.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:53.494 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:53.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.494 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s6 34/27 ECSubWrite(tid=79, reqid=osd.3.0:79, at_version=34'18, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:48:53.494 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.907810+0000", 2024-09-16T14:48:53.494 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.583419841, 2024-09-16T14:48:53.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040310379999999998, 2024-09-16T14:48:53.494 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.494 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.494 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907810+0000", 2024-09-16T14:48:53.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907810+0000", 2024-09-16T14:48:53.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907810+0000", 2024-09-16T14:48:53.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:53.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907836+0000", 2024-09-16T14:48:53.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6449999999999999e-05 2024-09-16T14:48:53.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907848+0000", 2024-09-16T14:48:53.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1516e-05 2024-09-16T14:48:53.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907880+0000", 2024-09-16T14:48:53.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1931999999999999e-05 2024-09-16T14:48:53.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907938+0000", 2024-09-16T14:48:53.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8613999999999998e-05 2024-09-16T14:48:53.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907951+0000", 2024-09-16T14:48:53.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2469999999999999e-05 2024-09-16T14:48:53.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.911771+0000", 2024-09-16T14:48:53.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038205639999999998 2024-09-16T14:48:53.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.911841+0000", 2024-09-16T14:48:53.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9658000000000003e-05 2024-09-16T14:48:53.498 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.498 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.498 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.498 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es1 34/27 ECSubWrite(tid=74, reqid=client.4500.0:229, at_version=34'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:48:53.498 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.913443+0000", 2024-09-16T14:48:53.498 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.577786632, 2024-09-16T14:48:53.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046494379999999997, 2024-09-16T14:48:53.498 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913443+0000", 2024-09-16T14:48:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913443+0000", 2024-09-16T14:48:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913442+0000", 2024-09-16T14:48:53.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:53.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913534+0000", 2024-09-16T14:48:53.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2428000000000003e-05 2024-09-16T14:48:53.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913545+0000", 2024-09-16T14:48:53.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0872e-05 2024-09-16T14:48:53.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913554+0000", 2024-09-16T14:48:53.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4009999999999997e-06 2024-09-16T14:48:53.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913605+0000", 2024-09-16T14:48:53.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1536000000000003e-05 2024-09-16T14:48:53.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913616+0000", 2024-09-16T14:48:53.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.064e-05 2024-09-16T14:48:53.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.918072+0000", 2024-09-16T14:48:53.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044558799999999997 2024-09-16T14:48:53.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.918092+0000", 2024-09-16T14:48:53.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0673e-05 2024-09-16T14:48:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs2 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:48:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.913469+0000", 2024-09-16T14:48:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.577761002999999, 2024-09-16T14:48:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045537390000000002, 2024-09-16T14:48:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913469+0000", 2024-09-16T14:48:53.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913469+0000", 2024-09-16T14:48:53.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913468+0000", 2024-09-16T14:48:53.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913497+0000", 2024-09-16T14:48:53.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9302000000000002e-05 2024-09-16T14:48:53.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913510+0000", 2024-09-16T14:48:53.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2799000000000001e-05 2024-09-16T14:48:53.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913522+0000", 2024-09-16T14:48:53.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2292e-05 2024-09-16T14:48:53.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913854+0000", 2024-09-16T14:48:53.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00033176700000000001 2024-09-16T14:48:53.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913863+0000", 2024-09-16T14:48:53.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9700000000000005e-06 2024-09-16T14:48:53.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917991+0000", 2024-09-16T14:48:53.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041282180000000003 2024-09-16T14:48:53.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.918022+0000", 2024-09-16T14:48:53.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0982000000000003e-05 2024-09-16T14:48:53.507 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.507 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.508 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.508 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s6 34/27 ECSubWrite(tid=82, reqid=osd.3.0:80, at_version=0'0, trim_to=0'0, roll_forward_to=34'20))", 2024-09-16T14:48:53.508 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.917598+0000", 2024-09-16T14:48:53.508 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.573631985, 2024-09-16T14:48:53.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0056448269999999998, 2024-09-16T14:48:53.508 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.508 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.509 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917598+0000", 2024-09-16T14:48:53.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917598+0000", 2024-09-16T14:48:53.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917597+0000", 2024-09-16T14:48:53.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917606+0000", 2024-09-16T14:48:53.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9209999999999998e-06 2024-09-16T14:48:53.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917616+0000", 2024-09-16T14:48:53.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4769999999999992e-06 2024-09-16T14:48:53.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917624+0000", 2024-09-16T14:48:53.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4800000000000001e-06 2024-09-16T14:48:53.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917758+0000", 2024-09-16T14:48:53.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000133347 2024-09-16T14:48:53.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917767+0000", 2024-09-16T14:48:53.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2769999999999994e-06 2024-09-16T14:48:53.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923214+0000", 2024-09-16T14:48:53.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00544737 2024-09-16T14:48:53.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923243+0000", 2024-09-16T14:48:53.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8303999999999999e-05 2024-09-16T14:48:53.512 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.513 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.513 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.513 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es1 34/27 ECSubWrite(tid=75, reqid=client.4500.0:229, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-16T14:48:53.513 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.920850+0000", 2024-09-16T14:48:53.513 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.570379381, 2024-09-16T14:48:53.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053109919999999996, 2024-09-16T14:48:53.513 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.513 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.513 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920850+0000", 2024-09-16T14:48:53.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920850+0000", 2024-09-16T14:48:53.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920849+0000", 2024-09-16T14:48:53.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:53.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920863+0000", 2024-09-16T14:48:53.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3224000000000001e-05 2024-09-16T14:48:53.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920880+0000", 2024-09-16T14:48:53.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7085999999999999e-05 2024-09-16T14:48:53.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920889+0000", 2024-09-16T14:48:53.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9740000000000008e-06 2024-09-16T14:48:53.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920937+0000", 2024-09-16T14:48:53.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8097999999999999e-05 2024-09-16T14:48:53.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920946+0000", 2024-09-16T14:48:53.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5689999999999994e-06 2024-09-16T14:48:53.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.926131+0000", 2024-09-16T14:48:53.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005184587 2024-09-16T14:48:53.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.926161+0000", 2024-09-16T14:48:53.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0328000000000002e-05 2024-09-16T14:48:53.517 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.517 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.517 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.518 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs2 34/27 ECSubWrite(tid=169, reqid=osd.7.0:168, at_version=0'0, trim_to=0'0, roll_forward_to=34'8))", 2024-09-16T14:48:53.518 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.921037+0000", 2024-09-16T14:48:53.518 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.570193058999999, 2024-09-16T14:48:53.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051281499999999997, 2024-09-16T14:48:53.518 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.518 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.518 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921037+0000", 2024-09-16T14:48:53.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921037+0000", 2024-09-16T14:48:53.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921036+0000", 2024-09-16T14:48:53.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:53.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921057+0000", 2024-09-16T14:48:53.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1022999999999998e-05 2024-09-16T14:48:53.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921071+0000", 2024-09-16T14:48:53.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4579000000000001e-05 2024-09-16T14:48:53.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921080+0000", 2024-09-16T14:48:53.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0100000000000001e-06 2024-09-16T14:48:53.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921122+0000", 2024-09-16T14:48:53.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2227000000000001e-05 2024-09-16T14:48:53.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921130+0000", 2024-09-16T14:48:53.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8690000000000004e-06 2024-09-16T14:48:53.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.926137+0000", 2024-09-16T14:48:53.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050068430000000004 2024-09-16T14:48:53.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.926165+0000", 2024-09-16T14:48:53.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7579000000000001e-05 2024-09-16T14:48:53.522 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.522 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.522 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.522 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:295 3.9s0 3:96d2b183:::smithi04040151-29:head [copy-from ver 0] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T14:48:53.522 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:48:53.522 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5757968840000007, 2024-09-16T14:48:53.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010216596999999999, 2024-09-16T14:48:53.522 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.523 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:53.523 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:53.523 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:53.523 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:53.523 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 295 2024-09-16T14:48:53.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.523 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:48:53.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:48:53.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915430+0000", 2024-09-16T14:48:53.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999976 2024-09-16T14:48:53.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915441+0000", 2024-09-16T14:48:53.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0789e-05 2024-09-16T14:48:53.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915462+0000", 2024-09-16T14:48:53.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0845000000000001e-05 2024-09-16T14:48:53.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915473+0000", 2024-09-16T14:48:53.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0805000000000001e-05 2024-09-16T14:48:53.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915528+0000", 2024-09-16T14:48:53.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4772999999999999e-05 2024-09-16T14:48:53.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:48:53.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915537+0000", 2024-09-16T14:48:53.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6700000000000006e-06 2024-09-16T14:48:53.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.919813+0000", 2024-09-16T14:48:53.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042757840000000004 2024-09-16T14:48:53.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:53.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.919907+0000", 2024-09-16T14:48:53.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504999999999993e-05 2024-09-16T14:48:53.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925649+0000", 2024-09-16T14:48:53.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057427870000000001 2024-09-16T14:48:53.529 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.529 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.529 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.529 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:296 3.9s0 3:96d2b183:::smithi04040151-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-16T14:48:53.529 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:48:53.529 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5757449670000003, 2024-09-16T14:48:53.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010423623, 2024-09-16T14:48:53.530 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.530 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:53.530 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:53.530 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:53.530 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:53.530 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 296 2024-09-16T14:48:53.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.530 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:48:53.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:48:53.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915484+0000", 2024-09-16T14:48:53.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915489+0000", 2024-09-16T14:48:53.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6079999999999998e-06 2024-09-16T14:48:53.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915496+0000", 2024-09-16T14:48:53.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7589999999999997e-06 2024-09-16T14:48:53.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915506+0000", 2024-09-16T14:48:53.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0340000000000002e-06 2024-09-16T14:48:53.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915588+0000", 2024-09-16T14:48:53.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2198000000000001e-05 2024-09-16T14:48:53.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:48:53.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915594+0000", 2024-09-16T14:48:53.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:48:53.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920099+0000", 2024-09-16T14:48:53.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045048700000000002 2024-09-16T14:48:53.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:48:53.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920147+0000", 2024-09-16T14:48:53.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8239999999999999e-05 2024-09-16T14:48:53.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925726+0000", 2024-09-16T14:48:53.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055787529999999997 2024-09-16T14:48:53.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:53.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925816+0000", 2024-09-16T14:48:53.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0185000000000005e-05 2024-09-16T14:48:53.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925908+0000", 2024-09-16T14:48:53.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1948999999999997e-05 2024-09-16T14:48:53.535 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.535 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.535 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.535 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s4 37/27 ECSubWrite(tid=23, reqid=osd.0.0:23, at_version=37'17, trim_to=0'0, roll_forward_to=35'13))", 2024-09-16T14:48:53.536 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.917500+0000", 2024-09-16T14:48:53.536 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5737296670000003, 2024-09-16T14:48:53.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0043821340000000002, 2024-09-16T14:48:53.536 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.536 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.536 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.917500+0000", 2024-09-16T14:48:53.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.917500+0000", 2024-09-16T14:48:53.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.917499+0000", 2024-09-16T14:48:53.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:53.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.917520+0000", 2024-09-16T14:48:53.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0968999999999999e-05 2024-09-16T14:48:53.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.917532+0000", 2024-09-16T14:48:53.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2061e-05 2024-09-16T14:48:53.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.917540+0000", 2024-09-16T14:48:53.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6909999999999997e-06 2024-09-16T14:48:53.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.917589+0000", 2024-09-16T14:48:53.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8955000000000001e-05 2024-09-16T14:48:53.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.917600+0000", 2024-09-16T14:48:53.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0689e-05 2024-09-16T14:48:53.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921864+0000", 2024-09-16T14:48:53.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042646009999999998 2024-09-16T14:48:53.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921882+0000", 2024-09-16T14:48:53.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7957000000000001e-05 2024-09-16T14:48:53.540 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.540 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.540 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.540 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s6 37/27 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=37'32, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:53.540 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921399+0000", 2024-09-16T14:48:53.540 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5698303540000005, 2024-09-16T14:48:53.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071090340000000002, 2024-09-16T14:48:53.540 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.540 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.541 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921399+0000", 2024-09-16T14:48:53.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921399+0000", 2024-09-16T14:48:53.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921399+0000", 2024-09-16T14:48:53.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921420+0000", 2024-09-16T14:48:53.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1094000000000002e-05 2024-09-16T14:48:53.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921428+0000", 2024-09-16T14:48:53.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3450000000000006e-06 2024-09-16T14:48:53.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921436+0000", 2024-09-16T14:48:53.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5789999999999998e-06 2024-09-16T14:48:53.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921525+0000", 2024-09-16T14:48:53.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8943999999999994e-05 2024-09-16T14:48:53.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921535+0000", 2024-09-16T14:48:53.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0183000000000001e-05 2024-09-16T14:48:53.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928483+0000", 2024-09-16T14:48:53.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069473800000000004 2024-09-16T14:48:53.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928508+0000", 2024-09-16T14:48:53.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5856000000000001e-05 2024-09-16T14:48:53.545 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.545 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.545 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.545 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs3 37/27 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:53.545 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921846+0000", 2024-09-16T14:48:53.545 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5693837930000001, 2024-09-16T14:48:53.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071221770000000004, 2024-09-16T14:48:53.545 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.545 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.545 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921846+0000", 2024-09-16T14:48:53.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921846+0000", 2024-09-16T14:48:53.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921845+0000", 2024-09-16T14:48:53.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921862+0000", 2024-09-16T14:48:53.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6807999999999999e-05 2024-09-16T14:48:53.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921871+0000", 2024-09-16T14:48:53.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6810000000000004e-06 2024-09-16T14:48:53.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921878+0000", 2024-09-16T14:48:53.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.109e-06 2024-09-16T14:48:53.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921935+0000", 2024-09-16T14:48:53.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7082e-05 2024-09-16T14:48:53.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921944+0000", 2024-09-16T14:48:53.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7609999999999996e-06 2024-09-16T14:48:53.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928947+0000", 2024-09-16T14:48:53.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070028870000000002 2024-09-16T14:48:53.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928968+0000", 2024-09-16T14:48:53.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1251e-05 2024-09-16T14:48:53.549 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.549 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.549 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.550 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s6 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:53.550 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921894+0000", 2024-09-16T14:48:53.550 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5693354569999993, 2024-09-16T14:48:53.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0072329309999999997, 2024-09-16T14:48:53.550 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.550 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.550 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.550 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.550 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921894+0000", 2024-09-16T14:48:53.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921894+0000", 2024-09-16T14:48:53.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921894+0000", 2024-09-16T14:48:53.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921908+0000", 2024-09-16T14:48:53.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3976e-05 2024-09-16T14:48:53.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921918+0000", 2024-09-16T14:48:53.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0182e-05 2024-09-16T14:48:53.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921924+0000", 2024-09-16T14:48:53.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1419999999999998e-06 2024-09-16T14:48:53.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922184+0000", 2024-09-16T14:48:53.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000260315 2024-09-16T14:48:53.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922192+0000", 2024-09-16T14:48:53.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7179999999999996e-06 2024-09-16T14:48:53.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929113+0000", 2024-09-16T14:48:53.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069209529999999997 2024-09-16T14:48:53.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929127+0000", 2024-09-16T14:48:53.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3991999999999999e-05 2024-09-16T14:48:53.554 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.554 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.554 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.554 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s4 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:48:53.554 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923965+0000", 2024-09-16T14:48:53.554 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.567264325, 2024-09-16T14:48:53.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050952489999999996, 2024-09-16T14:48:53.554 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.554 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.555 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923965+0000", 2024-09-16T14:48:53.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923965+0000", 2024-09-16T14:48:53.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923965+0000", 2024-09-16T14:48:53.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:53.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923986+0000", 2024-09-16T14:48:53.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1616000000000001e-05 2024-09-16T14:48:53.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924002+0000", 2024-09-16T14:48:53.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6273000000000002e-05 2024-09-16T14:48:53.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924011+0000", 2024-09-16T14:48:53.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6989999999999992e-06 2024-09-16T14:48:53.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924053+0000", 2024-09-16T14:48:53.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1652e-05 2024-09-16T14:48:53.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924062+0000", 2024-09-16T14:48:53.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0899999999999994e-06 2024-09-16T14:48:53.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929036+0000", 2024-09-16T14:48:53.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049742789999999999 2024-09-16T14:48:53.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929061+0000", 2024-09-16T14:48:53.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4420999999999999e-05 2024-09-16T14:48:53.558 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.558 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.558 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.559 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as4 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:48:53.559 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:48:53.559 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5671903159999996, 2024-09-16T14:48:53.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088260140000000001, 2024-09-16T14:48:53.559 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.559 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.559 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:48:53.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:48:53.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:48:53.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924062+0000", 2024-09-16T14:48:53.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2892e-05 2024-09-16T14:48:53.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924070+0000", 2024-09-16T14:48:53.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8690000000000004e-06 2024-09-16T14:48:53.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924076+0000", 2024-09-16T14:48:53.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0739999999999998e-06 2024-09-16T14:48:53.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924119+0000", 2024-09-16T14:48:53.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2549999999999997e-05 2024-09-16T14:48:53.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924126+0000", 2024-09-16T14:48:53.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8960000000000003e-06 2024-09-16T14:48:53.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932835+0000", 2024-09-16T14:48:53.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087089200000000002 2024-09-16T14:48:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932865+0000", 2024-09-16T14:48:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0054000000000001e-05 2024-09-16T14:48:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es1 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:48:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:48:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5670757280000007, 2024-09-16T14:48:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087127249999999993, 2024-09-16T14:48:53.564 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.564 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.564 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:48:53.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:48:53.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:48:53.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924168+0000", 2024-09-16T14:48:53.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4800000000000001e-05 2024-09-16T14:48:53.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924176+0000", 2024-09-16T14:48:53.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.25e-06 2024-09-16T14:48:53.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924182+0000", 2024-09-16T14:48:53.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3640000000000004e-06 2024-09-16T14:48:53.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924229+0000", 2024-09-16T14:48:53.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6477999999999998e-05 2024-09-16T14:48:53.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924238+0000", 2024-09-16T14:48:53.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4429999999999992e-06 2024-09-16T14:48:53.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932848+0000", 2024-09-16T14:48:53.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086097910000000003 2024-09-16T14:48:53.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932867+0000", 2024-09-16T14:48:53.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8927999999999999e-05 2024-09-16T14:48:53.568 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.568 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.568 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.568 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s6 37/27 ECSubWrite(tid=123, reqid=osd.3.0:123, at_version=37'38, trim_to=0'0, roll_forward_to=37'34))", 2024-09-16T14:48:53.568 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.936039+0000", 2024-09-16T14:48:53.568 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5551902569999996, 2024-09-16T14:48:53.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052747879999999999, 2024-09-16T14:48:53.569 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.569 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.569 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936039+0000", 2024-09-16T14:48:53.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936039+0000", 2024-09-16T14:48:53.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936039+0000", 2024-09-16T14:48:53.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936056+0000", 2024-09-16T14:48:53.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7241e-05 2024-09-16T14:48:53.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936065+0000", 2024-09-16T14:48:53.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8920000000000006e-06 2024-09-16T14:48:53.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936075+0000", 2024-09-16T14:48:53.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0327e-05 2024-09-16T14:48:53.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936115+0000", 2024-09-16T14:48:53.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9453999999999999e-05 2024-09-16T14:48:53.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936122+0000", 2024-09-16T14:48:53.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5780000000000001e-06 2024-09-16T14:48:53.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941293+0000", 2024-09-16T14:48:53.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051707480000000002 2024-09-16T14:48:53.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941314+0000", 2024-09-16T14:48:53.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1121999999999999e-05 2024-09-16T14:48:53.573 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.573 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.573 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.573 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.0s5 37/27 ECSubWrite(tid=109, reqid=osd.11.0:109, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:53.573 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.940425+0000", 2024-09-16T14:48:53.573 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5508051349999992, 2024-09-16T14:48:53.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035354269999999998, 2024-09-16T14:48:53.573 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.574 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.574 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940425+0000", 2024-09-16T14:48:53.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940425+0000", 2024-09-16T14:48:53.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940424+0000", 2024-09-16T14:48:53.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:53.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940447+0000", 2024-09-16T14:48:53.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3541e-05 2024-09-16T14:48:53.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940461+0000", 2024-09-16T14:48:53.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3871e-05 2024-09-16T14:48:53.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940470+0000", 2024-09-16T14:48:53.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6540000000000005e-06 2024-09-16T14:48:53.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940591+0000", 2024-09-16T14:48:53.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012163300000000001 2024-09-16T14:48:53.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940602+0000", 2024-09-16T14:48:53.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0132999999999999e-05 2024-09-16T14:48:53.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943936+0000", 2024-09-16T14:48:53.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0033343240000000001 2024-09-16T14:48:53.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943960+0000", 2024-09-16T14:48:53.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4022000000000001e-05 2024-09-16T14:48:53.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.577 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.578 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.578 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.8s7 37/27 ECSubWrite(tid=108, reqid=osd.11.0:108, at_version=37'33, trim_to=0'0, roll_forward_to=35'31))", 2024-09-16T14:48:53.578 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.940492+0000", 2024-09-16T14:48:53.578 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5507376910000001, 2024-09-16T14:48:53.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061198069999999997, 2024-09-16T14:48:53.578 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.578 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.578 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940492+0000", 2024-09-16T14:48:53.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940492+0000", 2024-09-16T14:48:53.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940492+0000", 2024-09-16T14:48:53.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940508+0000", 2024-09-16T14:48:53.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6767999999999999e-05 2024-09-16T14:48:53.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940520+0000", 2024-09-16T14:48:53.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2078e-05 2024-09-16T14:48:53.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940527+0000", 2024-09-16T14:48:53.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8630000000000002e-06 2024-09-16T14:48:53.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940914+0000", 2024-09-16T14:48:53.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00038634200000000002 2024-09-16T14:48:53.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940922+0000", 2024-09-16T14:48:53.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7490000000000004e-06 2024-09-16T14:48:53.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.946588+0000", 2024-09-16T14:48:53.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0056654230000000002 2024-09-16T14:48:53.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.946612+0000", 2024-09-16T14:48:53.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4008000000000001e-05 2024-09-16T14:48:53.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.582 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.582 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.8s7 37/27 ECSubWrite(tid=110, reqid=osd.11.0:110, at_version=37'35, trim_to=0'0, roll_forward_to=35'31))", 2024-09-16T14:48:53.582 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.941394+0000", 2024-09-16T14:48:53.583 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.5498355939999993, 2024-09-16T14:48:53.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0075660129999999999, 2024-09-16T14:48:53.583 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.583 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.583 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941394+0000", 2024-09-16T14:48:53.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941394+0000", 2024-09-16T14:48:53.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941393+0000", 2024-09-16T14:48:53.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:53.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941415+0000", 2024-09-16T14:48:53.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1162000000000002e-05 2024-09-16T14:48:53.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941426+0000", 2024-09-16T14:48:53.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1319e-05 2024-09-16T14:48:53.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941435+0000", 2024-09-16T14:48:53.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8769999999999999e-06 2024-09-16T14:48:53.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941762+0000", 2024-09-16T14:48:53.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032733399999999998 2024-09-16T14:48:53.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941772+0000", 2024-09-16T14:48:53.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9799999999999993e-06 2024-09-16T14:48:53.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.948937+0000", 2024-09-16T14:48:53.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071645550000000004 2024-09-16T14:48:53.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.948960+0000", 2024-09-16T14:48:53.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3529000000000001e-05 2024-09-16T14:48:53.587 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.587 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.587 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.587 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es1 38/27 ECSubWrite(tid=116, reqid=client.4500.0:300, at_version=0'0, trim_to=0'0, roll_forward_to=38'14))", 2024-09-16T14:48:53.587 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.943655+0000", 2024-09-16T14:48:53.587 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.5475751280000001, 2024-09-16T14:48:53.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0043420459999999996, 2024-09-16T14:48:53.587 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.588 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.588 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943655+0000", 2024-09-16T14:48:53.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943655+0000", 2024-09-16T14:48:53.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943654+0000", 2024-09-16T14:48:53.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:53.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943664+0000", 2024-09-16T14:48:53.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0669000000000001e-05 2024-09-16T14:48:53.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943678+0000", 2024-09-16T14:48:53.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3071000000000001e-05 2024-09-16T14:48:53.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943686+0000", 2024-09-16T14:48:53.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7970000000000006e-06 2024-09-16T14:48:53.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943745+0000", 2024-09-16T14:48:53.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8619999999999998e-05 2024-09-16T14:48:53.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943754+0000", 2024-09-16T14:48:53.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2289999999999992e-06 2024-09-16T14:48:53.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.947962+0000", 2024-09-16T14:48:53.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042080729999999997 2024-09-16T14:48:53.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.947997+0000", 2024-09-16T14:48:53.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4343e-05 2024-09-16T14:48:53.591 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.591 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.591 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.591 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.592 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.592 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:53.593 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:48:53.594 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:53.592+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:53.694 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:53.693+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:53.737 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:53.737 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:53.737 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:53.737 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:53.746 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:48:53.796 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:53.796+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:53.848 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:53.848 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:53.848 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:53.848 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:53.848 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:53.857 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:48:53.857 INFO:tasks.rados.rados.0.smithi040.stdout:159: done (1 left) 2024-09-16T14:48:53.858 INFO:tasks.rados.rados.0.smithi040.stdout:160: done (0 left) 2024-09-16T14:48:53.858 INFO:tasks.rados.rados.0.smithi040.stdout:161: delete oid 33 current snap is 10 2024-09-16T14:48:53.864 INFO:tasks.rados.rados.0.smithi040.stdout:161: done (0 left) 2024-09-16T14:48:53.865 INFO:tasks.rados.rados.0.smithi040.stdout:162: read oid 37 snap -1 2024-09-16T14:48:53.865 INFO:tasks.rados.rados.0.smithi040.stdout:162: expect (ObjNum 73 snap 7 seq_num 73) 2024-09-16T14:48:53.865 INFO:tasks.rados.rados.0.smithi040.stdout:163: rollback oid 47 current snap is 10 2024-09-16T14:48:53.865 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 47 to 9 2024-09-16T14:48:53.865 INFO:tasks.rados.rados.0.smithi040.stdout:164: rmattr oid 2 current snap is 10 2024-09-16T14:48:53.865 INFO:tasks.rados.rados.0.smithi040.stdout:164: done (2 left) 2024-09-16T14:48:53.865 INFO:tasks.rados.rados.0.smithi040.stdout:165: rollback oid 17 current snap is 10 2024-09-16T14:48:53.865 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 17 to 5 2024-09-16T14:48:53.865 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 32 current snap is 10 2024-09-16T14:48:53.865 INFO:tasks.rados.rados.0.smithi040.stdout:166: seq_num 75 ranges {196608=491520,688128=770048} 2024-09-16T14:48:53.869 INFO:tasks.rados.rados.0.smithi040.stdout:166: writing smithi04040151-32 from 196608 to 688128 tid 1 2024-09-16T14:48:53.874 INFO:tasks.rados.rados.0.smithi040.stdout:166: writing smithi04040151-32 from 688128 to 1458176 tid 2 2024-09-16T14:48:53.874 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 20 current snap is 10 2024-09-16T14:48:53.874 INFO:tasks.rados.rados.0.smithi040.stdout:167: seq_num 76 ranges {327680=507904,835584=507904,1343488=262144} 2024-09-16T14:48:53.879 INFO:tasks.rados.rados.0.smithi040.stdout:167: writing smithi04040151-20 from 327680 to 835584 tid 1 2024-09-16T14:48:53.882 INFO:tasks.rados.rados.0.smithi040.stdout:167: writing smithi04040151-20 from 835584 to 1343488 tid 2 2024-09-16T14:48:53.884 INFO:tasks.rados.rados.0.smithi040.stdout:167: writing smithi04040151-20 from 1343488 to 1605632 tid 3 2024-09-16T14:48:53.884 INFO:tasks.rados.rados.0.smithi040.stdout:168: copy_from oid 11 from oid 12 current snap is 10 2024-09-16T14:48:53.884 INFO:tasks.rados.rados.0.smithi040.stdout:163: finishing rollback tid 0 to smithi04040151-47 2024-09-16T14:48:53.884 INFO:tasks.rados.rados.0.smithi040.stdout:163: finishing rollback tid 1 to smithi04040151-47 2024-09-16T14:48:53.884 INFO:tasks.rados.rados.0.smithi040.stdout:163: finishing rollback tid 2 to smithi04040151-47 2024-09-16T14:48:53.884 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 47 v 27 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-16T14:48:53.884 INFO:tasks.rados.rados.0.smithi040.stdout:165: finishing rollback tid 0 to smithi04040151-17 2024-09-16T14:48:53.884 INFO:tasks.rados.rados.0.smithi040.stdout:165: finishing rollback tid 1 to smithi04040151-17 2024-09-16T14:48:53.885 INFO:tasks.rados.rados.0.smithi040.stdout:163: done (5 left) 2024-09-16T14:48:53.885 INFO:tasks.rados.rados.0.smithi040.stdout:169: read oid 50 snap -1 2024-09-16T14:48:53.885 INFO:tasks.rados.rados.0.smithi040.stdout:169: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-16T14:48:53.885 INFO:tasks.rados.rados.0.smithi040.stdout:170: snap_create 2024-09-16T14:48:53.885 INFO:tasks.rados.rados.0.smithi040.stdout:165: finishing rollback tid 2 to smithi04040151-17 2024-09-16T14:48:53.885 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 17 v 34 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-16T14:48:53.885 INFO:tasks.rados.rados.0.smithi040.stdout:162: expect (ObjNum 73 snap 7 seq_num 73) 2024-09-16T14:48:53.897 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:53.896+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:53.972 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:53.972 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:53.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:53.973 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:53.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.973 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=79, reqid=osd.3.0:79, at_version=34'18, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:48:53.973 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:53.973 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.062641701, 2024-09-16T14:48:53.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069050049999999997, 2024-09-16T14:48:53.973 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.973 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.973 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:53.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.974 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:53.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.974 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:53.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907863+0000", 2024-09-16T14:48:53.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5972000000000001e-05 2024-09-16T14:48:53.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907876+0000", 2024-09-16T14:48:53.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3230999999999999e-05 2024-09-16T14:48:53.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907900+0000", 2024-09-16T14:48:53.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3716000000000001e-05 2024-09-16T14:48:53.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907956+0000", 2024-09-16T14:48:53.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6307999999999997e-05 2024-09-16T14:48:53.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907965+0000", 2024-09-16T14:48:53.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1530000000000003e-06 2024-09-16T14:48:53.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914715+0000", 2024-09-16T14:48:53.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067495990000000002 2024-09-16T14:48:53.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914742+0000", 2024-09-16T14:48:53.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.728e-05 2024-09-16T14:48:53.977 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.977 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.977 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.977 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=80, reqid=osd.3.0:80, at_version=34'20, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:48:53.977 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:48:53.977 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.061225030999999, 2024-09-16T14:48:53.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065560849999999997, 2024-09-16T14:48:53.978 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.978 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.978 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:48:53.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:48:53.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909253+0000", 2024-09-16T14:48:53.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:53.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909278+0000", 2024-09-16T14:48:53.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4905e-05 2024-09-16T14:48:53.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909295+0000", 2024-09-16T14:48:53.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6713000000000001e-05 2024-09-16T14:48:53.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909304+0000", 2024-09-16T14:48:53.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6739999999999992e-06 2024-09-16T14:48:53.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909367+0000", 2024-09-16T14:48:53.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2700999999999995e-05 2024-09-16T14:48:53.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909391+0000", 2024-09-16T14:48:53.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3614000000000001e-05 2024-09-16T14:48:53.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915784+0000", 2024-09-16T14:48:53.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063938479999999997 2024-09-16T14:48:53.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915810+0000", 2024-09-16T14:48:53.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5457999999999999e-05 2024-09-16T14:48:53.982 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.982 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.982 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.983 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:48:53.983 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:53.983 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.060597442000001, 2024-09-16T14:48:53.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007542137, 2024-09-16T14:48:53.983 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.983 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.983 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:53.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:53.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:53.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909917+0000", 2024-09-16T14:48:53.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6513000000000001e-05 2024-09-16T14:48:53.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:48:53.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:48:53.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909941+0000", 2024-09-16T14:48:53.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1859999999999992e-06 2024-09-16T14:48:53.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910004+0000", 2024-09-16T14:48:53.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2706999999999994e-05 2024-09-16T14:48:53.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910012+0000", 2024-09-16T14:48:53.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.157e-06 2024-09-16T14:48:53.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917395+0000", 2024-09-16T14:48:53.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073827659999999998 2024-09-16T14:48:53.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917423+0000", 2024-09-16T14:48:53.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8568000000000002e-05 2024-09-16T14:48:53.987 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.987 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.987 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.987 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:48:53.987 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:48:53.987 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.056435201999999, 2024-09-16T14:48:53.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065747590000000003, 2024-09-16T14:48:53.987 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.987 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.988 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:48:53.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:48:53.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914043+0000", 2024-09-16T14:48:53.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914071+0000", 2024-09-16T14:48:53.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7633000000000001e-05 2024-09-16T14:48:53.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914086+0000", 2024-09-16T14:48:53.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4973999999999999e-05 2024-09-16T14:48:53.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914095+0000", 2024-09-16T14:48:53.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2119999999999992e-06 2024-09-16T14:48:53.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915626+0000", 2024-09-16T14:48:53.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001530998 2024-09-16T14:48:53.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915635+0000", 2024-09-16T14:48:53.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0140000000000004e-06 2024-09-16T14:48:53.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920590+0000", 2024-09-16T14:48:53.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049547740000000003 2024-09-16T14:48:53.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920618+0000", 2024-09-16T14:48:53.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8767000000000001e-05 2024-09-16T14:48:53.991 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.992 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.992 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.992 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:48:53.992 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:53.992 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.054112712, 2024-09-16T14:48:53.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006879947, 2024-09-16T14:48:53.992 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.992 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.992 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:53.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:53.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:53.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:53.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8702000000000002e-05 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916394+0000", 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6350000000000001e-06 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916403+0000", 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3950000000000001e-06 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916459+0000", 2024-09-16T14:48:53.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5889e-05 2024-09-16T14:48:53.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:53.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916467+0000", 2024-09-16T14:48:53.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2770000000000005e-06 2024-09-16T14:48:53.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:53.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923224+0000", 2024-09-16T14:48:53.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067573099999999999 2024-09-16T14:48:53.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:53.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923246+0000", 2024-09-16T14:48:53.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1882000000000002e-05 2024-09-16T14:48:53.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.996 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:53.996 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:53.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.996 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=82, reqid=osd.3.0:80, at_version=0'0, trim_to=0'0, roll_forward_to=34'20))", 2024-09-16T14:48:53.996 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:48:53.996 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.052538846999999, 2024-09-16T14:48:53.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005378954, 2024-09-16T14:48:53.997 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:53.997 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:53.997 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:53.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:53.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:48:53.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:53.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:48:53.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:53.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917939+0000", 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917952+0000", 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2278e-05 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917963+0000", 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.102e-05 2024-09-16T14:48:53.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:53.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917970+0000", 2024-09-16T14:48:53.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9589999999999998e-06 2024-09-16T14:48:53.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:53.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:53.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:53.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.918179+0000", 2024-09-16T14:48:53.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020897 2024-09-16T14:48:54.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.918188+0000", 2024-09-16T14:48:54.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9600000000000006e-06 2024-09-16T14:48:54.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923288+0000", 2024-09-16T14:48:54.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051001830000000003 2024-09-16T14:48:54.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923319+0000", 2024-09-16T14:48:54.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1131999999999999e-05 2024-09-16T14:48:54.001 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.001 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.001 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.001 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=37'32, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:54.001 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:48:54.001 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.049487098, 2024-09-16T14:48:54.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070991120000000003, 2024-09-16T14:48:54.001 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.001 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.001 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:48:54.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:48:54.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920991+0000", 2024-09-16T14:48:54.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921010+0000", 2024-09-16T14:48:54.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8529000000000002e-05 2024-09-16T14:48:54.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921018+0000", 2024-09-16T14:48:54.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4670000000000004e-06 2024-09-16T14:48:54.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921025+0000", 2024-09-16T14:48:54.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9330000000000004e-06 2024-09-16T14:48:54.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921076+0000", 2024-09-16T14:48:54.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0662999999999999e-05 2024-09-16T14:48:54.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921084+0000", 2024-09-16T14:48:54.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4780000000000008e-06 2024-09-16T14:48:54.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928073+0000", 2024-09-16T14:48:54.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069883940000000002 2024-09-16T14:48:54.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928091+0000", 2024-09-16T14:48:54.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7992999999999999e-05 2024-09-16T14:48:54.005 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.005 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.005 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.005 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s6 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:54.005 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:48:54.006 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.048523843, 2024-09-16T14:48:54.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059936520000000004, 2024-09-16T14:48:54.006 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.006 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.006 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:48:54.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:48:54.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:48:54.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921983+0000", 2024-09-16T14:48:54.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8493999999999999e-05 2024-09-16T14:48:54.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:48:54.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3110000000000006e-06 2024-09-16T14:48:54.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921999+0000", 2024-09-16T14:48:54.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5499999999999997e-06 2024-09-16T14:48:54.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922042+0000", 2024-09-16T14:48:54.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3387999999999999e-05 2024-09-16T14:48:54.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922053+0000", 2024-09-16T14:48:54.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0244999999999999e-05 2024-09-16T14:48:54.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.927921+0000", 2024-09-16T14:48:54.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0058679159999999999 2024-09-16T14:48:54.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.927949+0000", 2024-09-16T14:48:54.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8038000000000002e-05 2024-09-16T14:48:54.010 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.010 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.010 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.010 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:54.010 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:48:54.010 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.048487793, 2024-09-16T14:48:54.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070100509999999998, 2024-09-16T14:48:54.010 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.010 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.011 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:48:54.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:48:54.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921990+0000", 2024-09-16T14:48:54.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922012+0000", 2024-09-16T14:48:54.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1681000000000001e-05 2024-09-16T14:48:54.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922025+0000", 2024-09-16T14:48:54.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3101999999999999e-05 2024-09-16T14:48:54.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922033+0000", 2024-09-16T14:48:54.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1899999999999995e-06 2024-09-16T14:48:54.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922727+0000", 2024-09-16T14:48:54.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00069429699999999997 2024-09-16T14:48:54.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922737+0000", 2024-09-16T14:48:54.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.285e-06 2024-09-16T14:48:54.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928986+0000", 2024-09-16T14:48:54.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062492679999999997 2024-09-16T14:48:54.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929001+0000", 2024-09-16T14:48:54.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5058999999999999e-05 2024-09-16T14:48:54.014 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.014 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.014 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.015 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 37/27 ECSubWrite(tid=189, reqid=osd.7.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'12))", 2024-09-16T14:48:54.015 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922523+0000", 2024-09-16T14:48:54.015 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.047955929, 2024-09-16T14:48:54.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055324470000000002, 2024-09-16T14:48:54.015 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.015 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.015 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922523+0000", 2024-09-16T14:48:54.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922523+0000", 2024-09-16T14:48:54.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922522+0000", 2024-09-16T14:48:54.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922531+0000", 2024-09-16T14:48:54.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6079999999999996e-06 2024-09-16T14:48:54.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922540+0000", 2024-09-16T14:48:54.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-16T14:48:54.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922548+0000", 2024-09-16T14:48:54.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4089999999999997e-06 2024-09-16T14:48:54.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922589+0000", 2024-09-16T14:48:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0853999999999999e-05 2024-09-16T14:48:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922597+0000", 2024-09-16T14:48:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3270000000000001e-06 2024-09-16T14:48:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928016+0000", 2024-09-16T14:48:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054186269999999996 2024-09-16T14:48:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928055+0000", 2024-09-16T14:48:54.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9749999999999997e-05 2024-09-16T14:48:54.019 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.019 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.019 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.019 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.ds2 37/27 ECSubWrite(tid=190, reqid=osd.7.0:188, at_version=0'0, trim_to=0'0, roll_forward_to=37'20))", 2024-09-16T14:48:54.019 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:48:54.019 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.047729338, 2024-09-16T14:48:54.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062183510000000004, 2024-09-16T14:48:54.019 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.019 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.019 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:48:54.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:48:54.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:48:54.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922758+0000", 2024-09-16T14:48:54.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6209999999999993e-06 2024-09-16T14:48:54.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922767+0000", 2024-09-16T14:48:54.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4159999999999993e-06 2024-09-16T14:48:54.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922780+0000", 2024-09-16T14:48:54.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2992e-05 2024-09-16T14:48:54.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922834+0000", 2024-09-16T14:48:54.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3835999999999998e-05 2024-09-16T14:48:54.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922911+0000", 2024-09-16T14:48:54.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7534999999999995e-05 2024-09-16T14:48:54.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928945+0000", 2024-09-16T14:48:54.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060339210000000002 2024-09-16T14:48:54.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928968+0000", 2024-09-16T14:48:54.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2450999999999999e-05 2024-09-16T14:48:54.023 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.023 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.023 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.024 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s6 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:48:54.024 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923698+0000", 2024-09-16T14:48:54.024 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.046781083999999, 2024-09-16T14:48:54.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055372479999999998, 2024-09-16T14:48:54.024 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.024 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.024 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923698+0000", 2024-09-16T14:48:54.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923698+0000", 2024-09-16T14:48:54.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923697+0000", 2024-09-16T14:48:54.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923708+0000", 2024-09-16T14:48:54.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1524e-05 2024-09-16T14:48:54.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923720+0000", 2024-09-16T14:48:54.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1620000000000001e-05 2024-09-16T14:48:54.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923732+0000", 2024-09-16T14:48:54.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1711e-05 2024-09-16T14:48:54.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923796+0000", 2024-09-16T14:48:54.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4673999999999998e-05 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923805+0000", 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3240000000000003e-06 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929203+0000", 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005398351 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929235+0000", 2024-09-16T14:48:54.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1902000000000001e-05 2024-09-16T14:48:54.028 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.028 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.028 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.028 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:48:54.028 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:48:54.028 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.045278728, 2024-09-16T14:48:54.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079457759999999999, 2024-09-16T14:48:54.028 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.028 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.028 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925222+0000", 2024-09-16T14:48:54.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1871000000000001e-05 2024-09-16T14:48:54.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925238+0000", 2024-09-16T14:48:54.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6719999999999999e-05 2024-09-16T14:48:54.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925244+0000", 2024-09-16T14:48:54.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3630000000000002e-06 2024-09-16T14:48:54.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925291+0000", 2024-09-16T14:48:54.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6944000000000003e-05 2024-09-16T14:48:54.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925299+0000", 2024-09-16T14:48:54.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:48:54.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933129+0000", 2024-09-16T14:48:54.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078297330000000002 2024-09-16T14:48:54.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933146+0000", 2024-09-16T14:48:54.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.664e-05 2024-09-16T14:48:54.032 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.032 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.032 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.032 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es5 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:48:54.032 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:48:54.032 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.045227092999999, 2024-09-16T14:48:54.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078358790000000005, 2024-09-16T14:48:54.033 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.033 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.033 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:48:54.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:48:54.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925251+0000", 2024-09-16T14:48:54.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925265+0000", 2024-09-16T14:48:54.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4078e-05 2024-09-16T14:48:54.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925273+0000", 2024-09-16T14:48:54.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1069999999999999e-06 2024-09-16T14:48:54.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925279+0000", 2024-09-16T14:48:54.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4409999999999998e-06 2024-09-16T14:48:54.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925330+0000", 2024-09-16T14:48:54.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1208999999999997e-05 2024-09-16T14:48:54.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925339+0000", 2024-09-16T14:48:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8109999999999992e-06 2024-09-16T14:48:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933064+0000", 2024-09-16T14:48:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077252149999999997 2024-09-16T14:48:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933088+0000", 2024-09-16T14:48:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3317999999999999e-05 2024-09-16T14:48:54.037 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.037 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.037 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.037 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=123, reqid=osd.3.0:123, at_version=37'38, trim_to=0'0, roll_forward_to=37'34))", 2024-09-16T14:48:54.037 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.936249+0000", 2024-09-16T14:48:54.037 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.034230104000001, 2024-09-16T14:48:54.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050599219999999997, 2024-09-16T14:48:54.037 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.037 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.038 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936249+0000", 2024-09-16T14:48:54.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936249+0000", 2024-09-16T14:48:54.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936248+0000", 2024-09-16T14:48:54.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936265+0000", 2024-09-16T14:48:54.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7010000000000001e-05 2024-09-16T14:48:54.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936274+0000", 2024-09-16T14:48:54.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9840000000000007e-06 2024-09-16T14:48:54.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936281+0000", 2024-09-16T14:48:54.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8109999999999997e-06 2024-09-16T14:48:54.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936317+0000", 2024-09-16T14:48:54.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5923000000000001e-05 2024-09-16T14:48:54.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936322+0000", 2024-09-16T14:48:54.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1390000000000003e-06 2024-09-16T14:48:54.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941287+0000", 2024-09-16T14:48:54.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049647529999999997 2024-09-16T14:48:54.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941309+0000", 2024-09-16T14:48:54.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1767e-05 2024-09-16T14:48:54.041 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.041 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.041 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.042 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 37/27 ECSubWrite(tid=124, reqid=osd.3.0:120, at_version=0'0, trim_to=0'0, roll_forward_to=37'41))", 2024-09-16T14:48:54.042 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.942813+0000", 2024-09-16T14:48:54.042 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.027665931, 2024-09-16T14:48:54.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050535570000000002, 2024-09-16T14:48:54.042 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.042 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.042 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942813+0000", 2024-09-16T14:48:54.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942813+0000", 2024-09-16T14:48:54.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942812+0000", 2024-09-16T14:48:54.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942833+0000", 2024-09-16T14:48:54.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0823999999999999e-05 2024-09-16T14:48:54.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942848+0000", 2024-09-16T14:48:54.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5577999999999999e-05 2024-09-16T14:48:54.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942857+0000", 2024-09-16T14:48:54.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0010000000000007e-06 2024-09-16T14:48:54.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942901+0000", 2024-09-16T14:48:54.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3832000000000003e-05 2024-09-16T14:48:54.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942911+0000", 2024-09-16T14:48:54.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6039999999999999e-06 2024-09-16T14:48:54.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.947844+0000", 2024-09-16T14:48:54.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049332129999999997 2024-09-16T14:48:54.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.947866+0000", 2024-09-16T14:48:54.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2393999999999999e-05 2024-09-16T14:48:54.046 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.046 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.046 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.047 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 37/27 ECSubWrite(tid=125, reqid=osd.3.0:125, at_version=37'43, trim_to=0'0, roll_forward_to=37'41))", 2024-09-16T14:48:54.047 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.943728+0000", 2024-09-16T14:48:54.047 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.026751008, 2024-09-16T14:48:54.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005567626, 2024-09-16T14:48:54.047 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.047 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.047 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943728+0000", 2024-09-16T14:48:54.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943728+0000", 2024-09-16T14:48:54.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943727+0000", 2024-09-16T14:48:54.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943749+0000", 2024-09-16T14:48:54.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1634999999999999e-05 2024-09-16T14:48:54.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943760+0000", 2024-09-16T14:48:54.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1727e-05 2024-09-16T14:48:54.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943768+0000", 2024-09-16T14:48:54.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4529999999999996e-06 2024-09-16T14:48:54.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943814+0000", 2024-09-16T14:48:54.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6346e-05 2024-09-16T14:48:54.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943823+0000", 2024-09-16T14:48:54.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3230000000000004e-06 2024-09-16T14:48:54.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949278+0000", 2024-09-16T14:48:54.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054548649999999997 2024-09-16T14:48:54.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949295+0000", 2024-09-16T14:48:54.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7079e-05 2024-09-16T14:48:54.051 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.051 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.051 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.051 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.ds2 47/27 ECSubWrite(tid=208, reqid=client.4500.0:390, at_version=0'0, trim_to=0'0, roll_forward_to=47'27))", 2024-09-16T14:48:54.051 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.873647+0000", 2024-09-16T14:48:54.051 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.096832104000000002, 2024-09-16T14:48:54.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065883130000000002, 2024-09-16T14:48:54.051 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.052 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.052 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873647+0000", 2024-09-16T14:48:54.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873647+0000", 2024-09-16T14:48:54.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873646+0000", 2024-09-16T14:48:54.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873660+0000", 2024-09-16T14:48:54.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4056000000000001e-05 2024-09-16T14:48:54.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873675+0000", 2024-09-16T14:48:54.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.543e-05 2024-09-16T14:48:54.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873683+0000", 2024-09-16T14:48:54.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0879999999999995e-06 2024-09-16T14:48:54.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873732+0000", 2024-09-16T14:48:54.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8738e-05 2024-09-16T14:48:54.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873742+0000", 2024-09-16T14:48:54.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8949999999999993e-06 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880210+0000", 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006468321 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880235+0000", 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.495e-05 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.056 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 47/27 ECSubWrite(tid=45, reqid=client.4500.0:393, at_version=47'34, trim_to=0'0, roll_forward_to=43'30))", 2024-09-16T14:48:54.056 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.873840+0000", 2024-09-16T14:48:54.056 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.096638781000000007, 2024-09-16T14:48:54.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064368940000000003, 2024-09-16T14:48:54.056 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.056 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.056 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873840+0000", 2024-09-16T14:48:54.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873840+0000", 2024-09-16T14:48:54.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873839+0000", 2024-09-16T14:48:54.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.057 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873858+0000", 2024-09-16T14:48:54.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9228e-05 2024-09-16T14:48:54.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873870+0000", 2024-09-16T14:48:54.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1450999999999999e-05 2024-09-16T14:48:54.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873878+0000", 2024-09-16T14:48:54.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-16T14:48:54.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873936+0000", 2024-09-16T14:48:54.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7374999999999999e-05 2024-09-16T14:48:54.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873945+0000", 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.662e-06 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880251+0000", 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063056099999999997 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880277+0000", 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6013999999999999e-05 2024-09-16T14:48:54.059 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.060 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.060 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.060 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.9s3 47/27 ECSubWrite(tid=28, reqid=client.4500.0:396, at_version=47'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-16T14:48:54.060 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.885200+0000", 2024-09-16T14:48:54.060 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.085279158999999993, 2024-09-16T14:48:54.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053724640000000004, 2024-09-16T14:48:54.060 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.060 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.060 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885200+0000", 2024-09-16T14:48:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885200+0000", 2024-09-16T14:48:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885199+0000", 2024-09-16T14:48:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885217+0000", 2024-09-16T14:48:54.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8190999999999999e-05 2024-09-16T14:48:54.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885232+0000", 2024-09-16T14:48:54.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4676e-05 2024-09-16T14:48:54.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885240+0000", 2024-09-16T14:48:54.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6240000000000008e-06 2024-09-16T14:48:54.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885296+0000", 2024-09-16T14:48:54.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5705000000000003e-05 2024-09-16T14:48:54.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885307+0000", 2024-09-16T14:48:54.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1601e-05 2024-09-16T14:48:54.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890555+0000", 2024-09-16T14:48:54.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052472229999999996 2024-09-16T14:48:54.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890572+0000", 2024-09-16T14:48:54.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7394999999999999e-05 2024-09-16T14:48:54.064 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.064 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.064 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.064 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.065 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.065 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:54.066 INFO:tasks.rados.rados.0.smithi040.stdout:166: finishing write tid 1 to smithi04040151-32 2024-09-16T14:48:54.066 INFO:tasks.rados.rados.0.smithi040.stdout:169: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-16T14:48:54.067 INFO:tasks.rados.rados.0.smithi040.stdout:167: finishing write tid 1 to smithi04040151-20 2024-09-16T14:48:54.067 INFO:tasks.rados.rados.0.smithi040.stdout:167: finishing write tid 2 to smithi04040151-20 2024-09-16T14:48:54.067 INFO:tasks.rados.rados.0.smithi040.stdout:166: finishing write tid 2 to smithi04040151-32 2024-09-16T14:48:54.067 INFO:tasks.rados.rados.0.smithi040.stdout:166: finishing write tid 3 to smithi04040151-32 2024-09-16T14:48:54.067 INFO:tasks.rados.rados.0.smithi040.stdout:167: finishing write tid 3 to smithi04040151-20 2024-09-16T14:48:54.067 INFO:tasks.rados.rados.0.smithi040.stdout:167: finishing write tid 4 to smithi04040151-20 2024-09-16T14:48:54.067 INFO:tasks.rados.rados.0.smithi040.stdout:166: finishing write tid 4 to smithi04040151-32 2024-09-16T14:48:54.067 INFO:tasks.rados.rados.0.smithi040.stdout:166: oid 32 updating version 0 to 18 2024-09-16T14:48:54.067 INFO:tasks.rados.rados.0.smithi040.stdout:166: oid 32 updating version 18 to 19 2024-09-16T14:48:54.067 INFO:tasks.rados.rados.0.smithi040.stdout:166: oid 32 version 19 is already newer than 17 2024-09-16T14:48:54.067 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 32 v 19 (ObjNum 75 snap 10 seq_num 75) dirty exists 2024-09-16T14:48:54.067 INFO:tasks.rados.rados.0.smithi040.stdout:166: left oid 32 (ObjNum 75 snap 10 seq_num 75) 2024-09-16T14:48:54.067 INFO:tasks.rados.rados.0.smithi040.stdout:167: finishing write tid 5 to smithi04040151-20 2024-09-16T14:48:54.068 INFO:tasks.rados.rados.0.smithi040.stdout:167: oid 20 updating version 0 to 14 2024-09-16T14:48:54.068 INFO:tasks.rados.rados.0.smithi040.stdout:167: oid 20 updating version 14 to 15 2024-09-16T14:48:54.068 INFO:tasks.rados.rados.0.smithi040.stdout:167: oid 20 updating version 15 to 16 2024-09-16T14:48:54.068 INFO:tasks.rados.rados.0.smithi040.stdout:167: oid 20 version 16 is already newer than 13 2024-09-16T14:48:54.068 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 20 v 16 (ObjNum 76 snap 10 seq_num 76) dirty exists 2024-09-16T14:48:54.068 INFO:tasks.rados.rados.0.smithi040.stdout:167: left oid 20 (ObjNum 76 snap 10 seq_num 76) 2024-09-16T14:48:54.068 INFO:tasks.rados.rados.0.smithi040.stdout:168: finishing copy_from racing read to smithi04040151-11 2024-09-16T14:48:54.069 INFO:tasks.rados.rados.0.smithi040.stdout:168: finishing copy_from to smithi04040151-11 2024-09-16T14:48:54.069 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 11 v 2 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-16T14:48:54.069 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:54.065+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:54.069 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:48:54.167 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:54.166+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:54.172 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:54.172 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:54.172 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:54.181 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:48:54.268 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:54.266+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:54.283 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:54.283 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:54.283 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:54.283 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:54.283 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:54.292 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:48:54.369 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:48:54.368+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:48:54.436 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.436 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 34/27 ECSubWrite(tid=80, reqid=osd.3.0:80, at_version=34'20, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:48:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:48:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.524888468, 2024-09-16T14:48:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055013639999999999, 2024-09-16T14:48:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:48:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:48:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909414+0000", 2024-09-16T14:48:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909442+0000", 2024-09-16T14:48:54.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7662999999999999e-05 2024-09-16T14:48:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909452+0000", 2024-09-16T14:48:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0773e-05 2024-09-16T14:48:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909463+0000", 2024-09-16T14:48:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0144e-05 2024-09-16T14:48:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909522+0000", 2024-09-16T14:48:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9651000000000002e-05 2024-09-16T14:48:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909531+0000", 2024-09-16T14:48:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8529999999999998e-06 2024-09-16T14:48:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914874+0000", 2024-09-16T14:48:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053432829999999999 2024-09-16T14:48:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914916+0000", 2024-09-16T14:48:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1928000000000002e-05 2024-09-16T14:48:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:48:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:48:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.524819658, 2024-09-16T14:48:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079855010000000008, 2024-09-16T14:48:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.441 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:48:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:48:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909483+0000", 2024-09-16T14:48:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909521+0000", 2024-09-16T14:48:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7185999999999999e-05 2024-09-16T14:48:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909543+0000", 2024-09-16T14:48:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2802999999999998e-05 2024-09-16T14:48:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909551+0000", 2024-09-16T14:48:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5039999999999996e-06 2024-09-16T14:48:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909927+0000", 2024-09-16T14:48:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037637400000000001 2024-09-16T14:48:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:48:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2040000000000002e-06 2024-09-16T14:48:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917442+0000", 2024-09-16T14:48:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007508659 2024-09-16T14:48:54.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917469+0000", 2024-09-16T14:48:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7197999999999999e-05 2024-09-16T14:48:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:48:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.520747311999999, 2024-09-16T14:48:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067701159999999996, 2024-09-16T14:48:54.446 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913579+0000", 2024-09-16T14:48:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2829000000000001e-05 2024-09-16T14:48:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913591+0000", 2024-09-16T14:48:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2264e-05 2024-09-16T14:48:54.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913600+0000", 2024-09-16T14:48:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9959999999999999e-06 2024-09-16T14:48:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913648+0000", 2024-09-16T14:48:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7759000000000001e-05 2024-09-16T14:48:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913657+0000", 2024-09-16T14:48:54.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6709999999999994e-06 2024-09-16T14:48:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920296+0000", 2024-09-16T14:48:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066385120000000001 2024-09-16T14:48:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920326+0000", 2024-09-16T14:48:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0405000000000001e-05 2024-09-16T14:48:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.450 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 34/27 ECSubWrite(tid=74, reqid=client.4500.0:229, at_version=34'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:48:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.519931354000001, 2024-09-16T14:48:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060056090000000003, 2024-09-16T14:48:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:54.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914468+0000", 2024-09-16T14:48:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6398999999999996e-05 2024-09-16T14:48:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914483+0000", 2024-09-16T14:48:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5238e-05 2024-09-16T14:48:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914494+0000", 2024-09-16T14:48:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0192e-05 2024-09-16T14:48:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914622+0000", 2024-09-16T14:48:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012855599999999999 2024-09-16T14:48:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914635+0000", 2024-09-16T14:48:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2510000000000001e-05 2024-09-16T14:48:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920339+0000", 2024-09-16T14:48:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057041549999999998 2024-09-16T14:48:54.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920378+0000", 2024-09-16T14:48:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8918999999999998e-05 2024-09-16T14:48:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:48:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.517918578, 2024-09-16T14:48:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069011319999999999, 2024-09-16T14:48:54.455 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916410+0000", 2024-09-16T14:48:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5343000000000001e-05 2024-09-16T14:48:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916418+0000", 2024-09-16T14:48:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2360000000000004e-06 2024-09-16T14:48:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916428+0000", 2024-09-16T14:48:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6779999999999995e-06 2024-09-16T14:48:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916492+0000", 2024-09-16T14:48:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4219000000000003e-05 2024-09-16T14:48:54.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916502+0000", 2024-09-16T14:48:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6690000000000001e-06 2024-09-16T14:48:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923254+0000", 2024-09-16T14:48:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067518070000000003 2024-09-16T14:48:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923286+0000", 2024-09-16T14:48:54.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2367999999999999e-05 2024-09-16T14:48:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 34/27 ECSubWrite(tid=75, reqid=client.4500.0:229, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-16T14:48:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.920873+0000", 2024-09-16T14:48:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.513430164000001, 2024-09-16T14:48:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049917640000000001, 2024-09-16T14:48:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.460 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920873+0000", 2024-09-16T14:48:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920873+0000", 2024-09-16T14:48:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920872+0000", 2024-09-16T14:48:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920884+0000", 2024-09-16T14:48:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1501999999999999e-05 2024-09-16T14:48:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920898+0000", 2024-09-16T14:48:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4391e-05 2024-09-16T14:48:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920916+0000", 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8057e-05 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920965+0000", 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8962000000000003e-05 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920975+0000", 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3989999999999993e-06 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925839+0000", 2024-09-16T14:48:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048646330000000002 2024-09-16T14:48:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925865+0000", 2024-09-16T14:48:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5703000000000001e-05 2024-09-16T14:48:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/27 ECSubWrite(tid=169, reqid=osd.7.0:168, at_version=0'0, trim_to=0'0, roll_forward_to=34'8))", 2024-09-16T14:48:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.921046+0000", 2024-09-16T14:48:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.513257439, 2024-09-16T14:48:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048734829999999996, 2024-09-16T14:48:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921046+0000", 2024-09-16T14:48:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921046+0000", 2024-09-16T14:48:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921045+0000", 2024-09-16T14:48:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921056+0000", 2024-09-16T14:48:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0733e-05 2024-09-16T14:48:54.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921067+0000", 2024-09-16T14:48:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1048e-05 2024-09-16T14:48:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921076+0000", 2024-09-16T14:48:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:48:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921135+0000", 2024-09-16T14:48:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8681000000000003e-05 2024-09-16T14:48:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921144+0000", 2024-09-16T14:48:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9399999999999997e-06 2024-09-16T14:48:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925903+0000", 2024-09-16T14:48:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047588860000000004 2024-09-16T14:48:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925920+0000", 2024-09-16T14:48:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6147999999999999e-05 2024-09-16T14:48:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s7 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:48:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.512614193999999, 2024-09-16T14:48:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006972692, 2024-09-16T14:48:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.469 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:48:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:48:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:48:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921709+0000", 2024-09-16T14:48:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0411000000000002e-05 2024-09-16T14:48:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921719+0000", 2024-09-16T14:48:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2650000000000002e-06 2024-09-16T14:48:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921726+0000", 2024-09-16T14:48:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8770000000000004e-06 2024-09-16T14:48:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921773+0000", 2024-09-16T14:48:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7435999999999998e-05 2024-09-16T14:48:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921782+0000", 2024-09-16T14:48:54.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6039999999999993e-06 2024-09-16T14:48:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928635+0000", 2024-09-16T14:48:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068533129999999998 2024-09-16T14:48:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928662+0000", 2024-09-16T14:48:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7089000000000001e-05 2024-09-16T14:48:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:48:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.512117706, 2024-09-16T14:48:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064783530000000001, 2024-09-16T14:48:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:48:54.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:48:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922185+0000", 2024-09-16T14:48:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922205+0000", 2024-09-16T14:48:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9871e-05 2024-09-16T14:48:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922219+0000", 2024-09-16T14:48:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4266e-05 2024-09-16T14:48:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922229+0000", 2024-09-16T14:48:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5619999999999993e-06 2024-09-16T14:48:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922270+0000", 2024-09-16T14:48:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0917000000000001e-05 2024-09-16T14:48:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922278+0000", 2024-09-16T14:48:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3890000000000005e-06 2024-09-16T14:48:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928630+0000", 2024-09-16T14:48:54.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063524439999999996 2024-09-16T14:48:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928664+0000", 2024-09-16T14:48:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3651e-05 2024-09-16T14:48:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s5 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:48:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:48:54.478 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.510669302, 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069615850000000002, 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923633+0000", 2024-09-16T14:48:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923644+0000", 2024-09-16T14:48:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.08e-05 2024-09-16T14:48:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923657+0000", 2024-09-16T14:48:54.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2836999999999999e-05 2024-09-16T14:48:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923664+0000", 2024-09-16T14:48:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1470000000000004e-06 2024-09-16T14:48:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923885+0000", 2024-09-16T14:48:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022155800000000001 2024-09-16T14:48:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923894+0000", 2024-09-16T14:48:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9260000000000006e-06 2024-09-16T14:48:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930571+0000", 2024-09-16T14:48:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066770420000000002 2024-09-16T14:48:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930596+0000", 2024-09-16T14:48:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4383e-05 2024-09-16T14:48:54.482 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.483 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.483 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.483 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as3 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:48:54.483 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:48:54.483 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.509923066000001, 2024-09-16T14:48:54.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062471519999999997, 2024-09-16T14:48:54.483 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.483 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.483 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:48:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:48:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:48:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924405+0000", 2024-09-16T14:48:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5147000000000002e-05 2024-09-16T14:48:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924414+0000", 2024-09-16T14:48:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4810000000000006e-06 2024-09-16T14:48:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924421+0000", 2024-09-16T14:48:54.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0899999999999999e-06 2024-09-16T14:48:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924468+0000", 2024-09-16T14:48:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.727e-05 2024-09-16T14:48:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924477+0000", 2024-09-16T14:48:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7360000000000007e-06 2024-09-16T14:48:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930601+0000", 2024-09-16T14:48:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061242529999999996 2024-09-16T14:48:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930628+0000", 2024-09-16T14:48:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6517e-05 2024-09-16T14:48:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:48:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:48:54.487 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.509672862, 2024-09-16T14:48:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006019328, 2024-09-16T14:48:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:48:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:48:54.488 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924630+0000", 2024-09-16T14:48:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924650+0000", 2024-09-16T14:48:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0024e-05 2024-09-16T14:48:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.489 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924660+0000", 2024-09-16T14:48:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9219999999999992e-06 2024-09-16T14:48:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924668+0000", 2024-09-16T14:48:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6820000000000003e-06 2024-09-16T14:48:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924711+0000", 2024-09-16T14:48:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3009000000000001e-05 2024-09-16T14:48:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924720+0000", 2024-09-16T14:48:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9290000000000004e-06 2024-09-16T14:48:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930634+0000", 2024-09-16T14:48:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059144810000000001 2024-09-16T14:48:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930650+0000", 2024-09-16T14:48:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5688999999999998e-05 2024-09-16T14:48:54.491 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.5s7 37/27 ECSubWrite(tid=48, reqid=osd.5.0:47, at_version=0'0, trim_to=0'0, roll_forward_to=37'11))", 2024-09-16T14:48:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925599+0000", 2024-09-16T14:48:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.508704274999999, 2024-09-16T14:48:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050916670000000002, 2024-09-16T14:48:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925599+0000", 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925599+0000", 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925598+0000", 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925608+0000", 2024-09-16T14:48:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0034000000000001e-05 2024-09-16T14:48:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925619+0000", 2024-09-16T14:48:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0575e-05 2024-09-16T14:48:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925624+0000", 2024-09-16T14:48:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3909999999999997e-06 2024-09-16T14:48:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925664+0000", 2024-09-16T14:48:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9379999999999999e-05 2024-09-16T14:48:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925670+0000", 2024-09-16T14:48:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6619999999999999e-06 2024-09-16T14:48:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930669+0000", 2024-09-16T14:48:54.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049987199999999999 2024-09-16T14:48:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930691+0000", 2024-09-16T14:48:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1695000000000002e-05 2024-09-16T14:48:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 37/27 ECSubWrite(tid=119, reqid=osd.3.0:119, at_version=37'39, trim_to=0'0, roll_forward_to=36'37))", 2024-09-16T14:48:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.933227+0000", 2024-09-16T14:48:54.496 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.501076056, 2024-09-16T14:48:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041478280000000001, 2024-09-16T14:48:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933227+0000", 2024-09-16T14:48:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933227+0000", 2024-09-16T14:48:54.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933226+0000", 2024-09-16T14:48:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933247+0000", 2024-09-16T14:48:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1259000000000001e-05 2024-09-16T14:48:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933264+0000", 2024-09-16T14:48:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6362e-05 2024-09-16T14:48:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933273+0000", 2024-09-16T14:48:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4150000000000005e-06 2024-09-16T14:48:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933318+0000", 2024-09-16T14:48:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4799999999999998e-05 2024-09-16T14:48:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933327+0000", 2024-09-16T14:48:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0080000000000008e-06 2024-09-16T14:48:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.937345+0000", 2024-09-16T14:48:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0040184189999999996 2024-09-16T14:48:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.937375+0000", 2024-09-16T14:48:54.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9745000000000001e-05 2024-09-16T14:48:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 37/27 ECSubWrite(tid=123, reqid=osd.3.0:123, at_version=37'38, trim_to=0'0, roll_forward_to=37'34))", 2024-09-16T14:48:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.935961+0000", 2024-09-16T14:48:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.498342293, 2024-09-16T14:48:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052566519999999997, 2024-09-16T14:48:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.935961+0000", 2024-09-16T14:48:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.935961+0000", 2024-09-16T14:48:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.935961+0000", 2024-09-16T14:48:54.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936022+0000", 2024-09-16T14:48:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1205999999999996e-05 2024-09-16T14:48:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936037+0000", 2024-09-16T14:48:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5478e-05 2024-09-16T14:48:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936045+0000", 2024-09-16T14:48:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7060000000000004e-06 2024-09-16T14:48:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936604+0000", 2024-09-16T14:48:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055949199999999998 2024-09-16T14:48:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936612+0000", 2024-09-16T14:48:54.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:48:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941192+0000", 2024-09-16T14:48:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045797370000000004 2024-09-16T14:48:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941218+0000", 2024-09-16T14:48:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5826e-05 2024-09-16T14:48:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.8s6 37/27 ECSubWrite(tid=110, reqid=osd.11.0:110, at_version=37'35, trim_to=0'0, roll_forward_to=35'31))", 2024-09-16T14:48:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.941403+0000", 2024-09-16T14:48:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.492900493, 2024-09-16T14:48:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053034789999999998, 2024-09-16T14:48:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941403+0000", 2024-09-16T14:48:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941403+0000", 2024-09-16T14:48:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941402+0000", 2024-09-16T14:48:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941417+0000", 2024-09-16T14:48:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4578e-05 2024-09-16T14:48:54.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941431+0000", 2024-09-16T14:48:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3729e-05 2024-09-16T14:48:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941439+0000", 2024-09-16T14:48:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0199999999999994e-06 2024-09-16T14:48:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941487+0000", 2024-09-16T14:48:54.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8936e-05 2024-09-16T14:48:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941496+0000", 2024-09-16T14:48:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5320000000000006e-06 2024-09-16T14:48:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.946682+0000", 2024-09-16T14:48:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051858779999999997 2024-09-16T14:48:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.946706+0000", 2024-09-16T14:48:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4541000000000001e-05 2024-09-16T14:48:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 37/27 ECSubWrite(tid=125, reqid=osd.3.0:125, at_version=37'43, trim_to=0'0, roll_forward_to=37'41))", 2024-09-16T14:48:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.943746+0000", 2024-09-16T14:48:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.490556966, 2024-09-16T14:48:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054472219999999998, 2024-09-16T14:48:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.510 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943746+0000", 2024-09-16T14:48:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943746+0000", 2024-09-16T14:48:54.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943746+0000", 2024-09-16T14:48:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943767+0000", 2024-09-16T14:48:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1325e-05 2024-09-16T14:48:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943779+0000", 2024-09-16T14:48:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1865000000000001e-05 2024-09-16T14:48:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943787+0000", 2024-09-16T14:48:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7449999999999995e-06 2024-09-16T14:48:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943828+0000", 2024-09-16T14:48:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0985000000000001e-05 2024-09-16T14:48:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943836+0000", 2024-09-16T14:48:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5459999999999992e-06 2024-09-16T14:48:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949139+0000", 2024-09-16T14:48:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053031220000000004 2024-09-16T14:48:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949194+0000", 2024-09-16T14:48:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4360000000000001e-05 2024-09-16T14:48:54.514 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 38/27 ECSubWrite(tid=116, reqid=client.4500.0:300, at_version=0'0, trim_to=0'0, roll_forward_to=38'14))", 2024-09-16T14:48:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.943674+0000", 2024-09-16T14:48:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.490629234, 2024-09-16T14:48:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045685220000000002, 2024-09-16T14:48:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943674+0000", 2024-09-16T14:48:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943674+0000", 2024-09-16T14:48:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943673+0000", 2024-09-16T14:48:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943684+0000", 2024-09-16T14:48:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0599e-05 2024-09-16T14:48:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943699+0000", 2024-09-16T14:48:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5235000000000001e-05 2024-09-16T14:48:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943708+0000", 2024-09-16T14:48:54.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2439999999999993e-06 2024-09-16T14:48:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943756+0000", 2024-09-16T14:48:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8350000000000003e-05 2024-09-16T14:48:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943765+0000", 2024-09-16T14:48:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5119999999999997e-06 2024-09-16T14:48:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.948215+0000", 2024-09-16T14:48:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044499429999999996 2024-09-16T14:48:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.948243+0000", 2024-09-16T14:48:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7396999999999999e-05 2024-09-16T14:48:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 43/27 ECSubWrite(tid=149, reqid=osd.3.0:148, at_version=0'0, trim_to=0'0, roll_forward_to=43'58))", 2024-09-16T14:48:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:49.863865+0000", 2024-09-16T14:48:54.519 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.5704386939999999, 2024-09-16T14:48:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042146639999999999, 2024-09-16T14:48:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:49.863865+0000", 2024-09-16T14:48:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:49.863865+0000", 2024-09-16T14:48:54.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:49.863864+0000", 2024-09-16T14:48:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:49.863873+0000", 2024-09-16T14:48:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3149999999999998e-06 2024-09-16T14:48:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:49.863884+0000", 2024-09-16T14:48:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0376e-05 2024-09-16T14:48:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:49.863892+0000", 2024-09-16T14:48:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9859999999999994e-06 2024-09-16T14:48:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:49.863941+0000", 2024-09-16T14:48:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8891000000000003e-05 2024-09-16T14:48:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:49.863951+0000", 2024-09-16T14:48:54.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0036e-05 2024-09-16T14:48:54.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:49.868057+0000", 2024-09-16T14:48:54.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041060810000000001 2024-09-16T14:48:54.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:49.868079+0000", 2024-09-16T14:48:54.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2670000000000001e-05 2024-09-16T14:48:54.524 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.524 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.524 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.524 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as3 47/27 ECSubWrite(tid=140, reqid=client.4500.0:400, at_version=47'15, trim_to=0'0, roll_forward_to=38'11))", 2024-09-16T14:48:54.524 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.888761+0000", 2024-09-16T14:48:54.524 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.54554207399999999, 2024-09-16T14:48:54.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046964830000000004, 2024-09-16T14:48:54.524 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.524 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.524 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.888761+0000", 2024-09-16T14:48:54.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.888761+0000", 2024-09-16T14:48:54.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.888760+0000", 2024-09-16T14:48:54.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.888821+0000", 2024-09-16T14:48:54.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0715999999999999e-05 2024-09-16T14:48:54.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.888837+0000", 2024-09-16T14:48:54.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6314e-05 2024-09-16T14:48:54.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.888854+0000", 2024-09-16T14:48:54.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6847e-05 2024-09-16T14:48:54.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.888899+0000", 2024-09-16T14:48:54.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4749999999999997e-05 2024-09-16T14:48:54.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.888908+0000", 2024-09-16T14:48:54.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0520000000000007e-06 2024-09-16T14:48:54.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893432+0000", 2024-09-16T14:48:54.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045243949999999996 2024-09-16T14:48:54.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893458+0000", 2024-09-16T14:48:54.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5415999999999999e-05 2024-09-16T14:48:54.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.528 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.528 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.528 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:54.530 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:48:54.531 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:54.529+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:48:54.631 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:54.629+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:54.633 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:54.633 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:54.633 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:54.633 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:54.642 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:48:54.732 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:54.731+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:54.785 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:54.785 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:54.785 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:54.785 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:54.785 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:54.794 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:48:54.835 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:48:54.834+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:48:54.851 INFO:tasks.rados.rados.0.smithi040.stdout:162: done (6 left) 2024-09-16T14:48:54.851 INFO:tasks.rados.rados.0.smithi040.stdout:165: done (5 left) 2024-09-16T14:48:54.852 INFO:tasks.rados.rados.0.smithi040.stdout:166: done (4 left) 2024-09-16T14:48:54.852 INFO:tasks.rados.rados.0.smithi040.stdout:167: done (3 left) 2024-09-16T14:48:54.852 INFO:tasks.rados.rados.0.smithi040.stdout:168: done (2 left) 2024-09-16T14:48:54.852 INFO:tasks.rados.rados.0.smithi040.stdout:169: done (1 left) 2024-09-16T14:48:54.852 INFO:tasks.rados.rados.0.smithi040.stdout:170: done (0 left) 2024-09-16T14:48:54.852 INFO:tasks.rados.rados.0.smithi040.stdout:171: snap_create 2024-09-16T14:48:54.874 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:48:54.934 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:54.933+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:54.938 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:54.938 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:54.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:54.938 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:54.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.938 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:229 3.es0 3:7ecfd044:::smithi04040151-35:head [copy-from ver 4] snapc 4=[4,2] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:48:54.938 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:48:54.939 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.032102466, 2024-09-16T14:48:54.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016506461, 2024-09-16T14:48:54.939 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.939 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:54.939 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:54.939 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:54.939 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:54.939 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 229 2024-09-16T14:48:54.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.939 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:48:54.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:48:54.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.940 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.940 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904168+0000", 2024-09-16T14:48:54.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:48:54.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904187+0000", 2024-09-16T14:48:54.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9449999999999998e-05 2024-09-16T14:48:54.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904206+0000", 2024-09-16T14:48:54.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8698e-05 2024-09-16T14:48:54.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904215+0000", 2024-09-16T14:48:54.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:48:54.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904276+0000", 2024-09-16T14:48:54.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1130999999999995e-05 2024-09-16T14:48:54.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:48:54.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904284+0000", 2024-09-16T14:48:54.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0609999999999996e-06 2024-09-16T14:48:54.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906272+0000", 2024-09-16T14:48:54.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019884210000000002 2024-09-16T14:48:54.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:54.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906391+0000", 2024-09-16T14:48:54.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000118843 2024-09-16T14:48:54.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.912595+0000", 2024-09-16T14:48:54.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062038190000000002 2024-09-16T14:48:54.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917584+0000", 2024-09-16T14:48:54.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049889069999999999 2024-09-16T14:48:54.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:54.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920592+0000", 2024-09-16T14:48:54.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030081639999999998 2024-09-16T14:48:54.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920676+0000", 2024-09-16T14:48:54.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3974999999999995e-05 2024-09-16T14:48:54.945 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.945 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.945 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.945 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:230 3.es0 3:7ecfd044:::smithi04040151-35:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:48:54.945 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:48:54.945 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.032040983, 2024-09-16T14:48:54.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0096834569999999995, 2024-09-16T14:48:54.946 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.946 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:54.946 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:54.946 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:54.946 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:54.946 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 230 2024-09-16T14:48:54.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.946 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:48:54.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:48:54.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:48:54.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904235+0000", 2024-09-16T14:48:54.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4059999999999998e-06 2024-09-16T14:48:54.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904244+0000", 2024-09-16T14:48:54.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-16T14:48:54.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.948 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904250+0000", 2024-09-16T14:48:54.948 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0719999999999996e-06 2024-09-16T14:48:54.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.948 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.948 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904338+0000", 2024-09-16T14:48:54.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7923000000000005e-05 2024-09-16T14:48:54.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:48:54.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904344+0000", 2024-09-16T14:48:54.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8669999999999999e-06 2024-09-16T14:48:54.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.949 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.949 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.949 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906575+0000", 2024-09-16T14:48:54.949 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002230994 2024-09-16T14:48:54.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:48:54.950 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906632+0000", 2024-09-16T14:48:54.950 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7357e-05 2024-09-16T14:48:54.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.950 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.950 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913716+0000", 2024-09-16T14:48:54.951 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070835070000000002 2024-09-16T14:48:54.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.951 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.951 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:54.951 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913822+0000", 2024-09-16T14:48:54.952 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010566299999999999 2024-09-16T14:48:54.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.952 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.952 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.952 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913915+0000", 2024-09-16T14:48:54.953 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3018999999999998e-05 2024-09-16T14:48:54.953 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.953 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.953 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.953 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.953 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs4 37/27 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:54.954 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:48:54.954 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 11.015504172, 2024-09-16T14:48:54.954 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0098104279999999995, 2024-09-16T14:48:54.954 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.954 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.954 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:48:54.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.955 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.955 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.955 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:48:54.955 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:48:54.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.956 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.956 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.956 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920785+0000", 2024-09-16T14:48:54.956 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7218e-05 2024-09-16T14:48:54.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.957 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920793+0000", 2024-09-16T14:48:54.957 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9300000000000003e-06 2024-09-16T14:48:54.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.957 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.957 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920800+0000", 2024-09-16T14:48:54.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1860000000000003e-06 2024-09-16T14:48:54.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.958 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923692+0000", 2024-09-16T14:48:54.958 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028921329999999999 2024-09-16T14:48:54.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.958 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.958 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923701+0000", 2024-09-16T14:48:54.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6109999999999994e-06 2024-09-16T14:48:54.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930554+0000", 2024-09-16T14:48:54.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068534019999999998 2024-09-16T14:48:54.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.959 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.959 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.959 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930578+0000", 2024-09-16T14:48:54.959 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4232000000000001e-05 2024-09-16T14:48:54.960 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.960 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.960 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.960 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.960 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s1 37/27 ECSubWrite(tid=120, reqid=osd.3.0:120, at_version=37'41, trim_to=0'0, roll_forward_to=36'37))", 2024-09-16T14:48:54.960 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.934770+0000", 2024-09-16T14:48:54.960 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 11.001501713, 2024-09-16T14:48:54.960 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0076309480000000002, 2024-09-16T14:48:54.960 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.961 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:54.961 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.961 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.961 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.961 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934770+0000", 2024-09-16T14:48:54.961 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.962 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934770+0000", 2024-09-16T14:48:54.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934770+0000", 2024-09-16T14:48:54.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934786+0000", 2024-09-16T14:48:54.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6401000000000001e-05 2024-09-16T14:48:54.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.965 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.965 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934796+0000", 2024-09-16T14:48:54.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0267e-05 2024-09-16T14:48:54.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.965 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.965 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934804+0000", 2024-09-16T14:48:54.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1040000000000007e-06 2024-09-16T14:48:54.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934848+0000", 2024-09-16T14:48:54.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4070000000000003e-05 2024-09-16T14:48:54.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.934858+0000", 2024-09-16T14:48:54.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7079999999999992e-06 2024-09-16T14:48:54.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942379+0000", 2024-09-16T14:48:54.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0075206379999999996 2024-09-16T14:48:54.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.942401+0000", 2024-09-16T14:48:54.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2597999999999999e-05 2024-09-16T14:48:54.968 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.968 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.968 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.968 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:298 3.es0 3:7ecfd044:::smithi04040151-35:head [append 0~720896] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:48:54.968 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:48:54.968 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.007438650999999, 2024-09-16T14:48:54.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014187808, 2024-09-16T14:48:54.969 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.969 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:54.969 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:54.969 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:54.969 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:54.969 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 298 2024-09-16T14:48:54.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.970 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:48:54.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:48:54.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928832+0000", 2024-09-16T14:48:54.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:54.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929746+0000", 2024-09-16T14:48:54.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00091434199999999995 2024-09-16T14:48:54.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929760+0000", 2024-09-16T14:48:54.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3519e-05 2024-09-16T14:48:54.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929767+0000", 2024-09-16T14:48:54.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3180000000000001e-06 2024-09-16T14:48:54.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.972 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.972 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929849+0000", 2024-09-16T14:48:54.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2020000000000004e-05 2024-09-16T14:48:54.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:48:54.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929854+0000", 2024-09-16T14:48:54.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1209999999999998e-06 2024-09-16T14:48:54.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931714+0000", 2024-09-16T14:48:54.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001859989 2024-09-16T14:48:54.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:54.974 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931845+0000", 2024-09-16T14:48:54.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130901 2024-09-16T14:48:54.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.974 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933089+0000", 2024-09-16T14:48:54.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001243535 2024-09-16T14:48:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936370+0000", 2024-09-16T14:48:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032817660000000002 2024-09-16T14:48:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942945+0000", 2024-09-16T14:48:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065741560000000003 2024-09-16T14:48:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943021+0000", 2024-09-16T14:48:54.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6734000000000001e-05 2024-09-16T14:48:54.976 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.976 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.976 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.976 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:299 3.es0 3:7ecfd044:::smithi04040151-35:head [append 0~376832] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:48:54.976 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:48:54.976 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.005148851, 2024-09-16T14:48:54.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012149627, 2024-09-16T14:48:54.976 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.976 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:54.977 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:54.977 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:54.977 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:54.977 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 299 2024-09-16T14:48:54.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.977 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:48:54.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:48:54.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931122+0000", 2024-09-16T14:48:54.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931722+0000", 2024-09-16T14:48:54.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059944100000000004 2024-09-16T14:48:54.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931732+0000", 2024-09-16T14:48:54.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9369999999999999e-06 2024-09-16T14:48:54.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931739+0000", 2024-09-16T14:48:54.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2819999999999999e-06 2024-09-16T14:48:54.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933835+0000", 2024-09-16T14:48:54.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020961140000000001 2024-09-16T14:48:54.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:54.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933929+0000", 2024-09-16T14:48:54.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3683000000000004e-05 2024-09-16T14:48:54.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.981 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.981 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.934917+0000", 2024-09-16T14:48:54.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00098803200000000002 2024-09-16T14:48:54.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.939558+0000", 2024-09-16T14:48:54.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00464175 2024-09-16T14:48:54.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:54.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943205+0000", 2024-09-16T14:48:54.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036465820000000002 2024-09-16T14:48:54.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943273+0000", 2024-09-16T14:48:54.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7917999999999996e-05 2024-09-16T14:48:54.983 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.983 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.984 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.984 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:300 3.es0 3:7ecfd044:::smithi04040151-35:head [setxattr _header (56) in=7b] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:48:54.984 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:48:54.984 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.004522757, 2024-09-16T14:48:54.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011740911, 2024-09-16T14:48:54.985 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.985 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:54.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:54.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:54.985 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:54.985 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 300 2024-09-16T14:48:54.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.985 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:48:54.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:48:54.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:48:54.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:54.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931753+0000", 2024-09-16T14:48:54.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0380000000000001e-06 2024-09-16T14:48:54.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931759+0000", 2024-09-16T14:48:54.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3430000000000001e-06 2024-09-16T14:48:54.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931765+0000", 2024-09-16T14:48:54.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-16T14:48:54.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.935623+0000", 2024-09-16T14:48:54.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038583049999999998 2024-09-16T14:48:54.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:54.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.935717+0000", 2024-09-16T14:48:54.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4587999999999996e-05 2024-09-16T14:48:54.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:54.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936150+0000", 2024-09-16T14:48:54.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043270599999999999 2024-09-16T14:48:54.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:54.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.939576+0000", 2024-09-16T14:48:54.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034257620000000002 2024-09-16T14:48:54.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:54.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943399+0000", 2024-09-16T14:48:54.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003823145 2024-09-16T14:48:54.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:54.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943490+0000", 2024-09-16T14:48:54.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1241000000000003e-05 2024-09-16T14:48:54.994 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.995 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:54.995 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:54.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.995 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:301 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:48:54.995 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:48:54.995 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.004501457, 2024-09-16T14:48:54.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014215955000000001, 2024-09-16T14:48:54.995 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:54.996 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:54.996 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:54.996 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:54.996 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:54.996 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 301 2024-09-16T14:48:54.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.996 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:54.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:54.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:48:54.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:54.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:48:54.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:54.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:54.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931770+0000", 2024-09-16T14:48:54.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:54.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:54.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931775+0000", 2024-09-16T14:48:54.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4530000000000004e-06 2024-09-16T14:48:54.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:54.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931782+0000", 2024-09-16T14:48:54.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0550000000000002e-06 2024-09-16T14:48:54.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:54.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931787+0000", 2024-09-16T14:48:54.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2310000000000004e-06 2024-09-16T14:48:54.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:54.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936401+0000", 2024-09-16T14:48:54.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046143419999999996 2024-09-16T14:48:54.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:54.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:54.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:48:54.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936482+0000", 2024-09-16T14:48:54.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0407000000000005e-05 2024-09-16T14:48:55.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943851+0000", 2024-09-16T14:48:55.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073695000000000002 2024-09-16T14:48:55.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:55.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943951+0000", 2024-09-16T14:48:55.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9327000000000006e-05 2024-09-16T14:48:55.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:55.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.945987+0000", 2024-09-16T14:48:55.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002036057 2024-09-16T14:48:55.001 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.001 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:55.001 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.001 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:368 3.8s0 3:1f96bdfc:::smithi04040151-37:head [append 0~458752] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:48:55.001 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.954633+0000", 2024-09-16T14:48:55.002 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9816394180000003, 2024-09-16T14:48:55.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090058240000000008, 2024-09-16T14:48:55.002 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:55.002 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:55.002 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:55.002 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:55.002 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:55.002 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 368 2024-09-16T14:48:55.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.002 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:55.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:55.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.954633+0000", 2024-09-16T14:48:55.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:55.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.954633+0000", 2024-09-16T14:48:55.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:55.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.954631+0000", 2024-09-16T14:48:55.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:55.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:55.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.956022+0000", 2024-09-16T14:48:55.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001390591 2024-09-16T14:48:55.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:55.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.956034+0000", 2024-09-16T14:48:55.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2432999999999999e-05 2024-09-16T14:48:55.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:55.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.956041+0000", 2024-09-16T14:48:55.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8950000000000001e-06 2024-09-16T14:48:55.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.956093+0000", 2024-09-16T14:48:55.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1641999999999998e-05 2024-09-16T14:48:55.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:55.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.956194+0000", 2024-09-16T14:48:55.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000100797 2024-09-16T14:48:55.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:55.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957200+0000", 2024-09-16T14:48:55.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001006847 2024-09-16T14:48:55.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:55.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960279+0000", 2024-09-16T14:48:55.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030784900000000001 2024-09-16T14:48:55.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:55.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.963564+0000", 2024-09-16T14:48:55.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003285217 2024-09-16T14:48:55.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:55.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.963639+0000", 2024-09-16T14:48:55.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4412000000000005e-05 2024-09-16T14:48:55.007 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.007 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:55.007 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.007 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:369 3.8s0 3:1f96bdfc:::smithi04040151-37:head [append 0~475136] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:48:55.008 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:48:55.008 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9785594639999999, 2024-09-16T14:48:55.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010979078999999999, 2024-09-16T14:48:55.008 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:55.008 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:55.008 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:55.008 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:55.008 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:55.008 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 369 2024-09-16T14:48:55.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.008 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:55.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:55.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:48:55.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:55.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:48:55.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:55.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957711+0000", 2024-09-16T14:48:55.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:48:55.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:55.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958323+0000", 2024-09-16T14:48:55.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000612439 2024-09-16T14:48:55.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:55.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958339+0000", 2024-09-16T14:48:55.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5985999999999999e-05 2024-09-16T14:48:55.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:55.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958348+0000", 2024-09-16T14:48:55.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8699999999999998e-06 2024-09-16T14:48:55.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958393+0000", 2024-09-16T14:48:55.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4981000000000002e-05 2024-09-16T14:48:55.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:55.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958478+0000", 2024-09-16T14:48:55.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4845999999999994e-05 2024-09-16T14:48:55.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:55.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.959659+0000", 2024-09-16T14:48:55.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011808999999999999 2024-09-16T14:48:55.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:55.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962687+0000", 2024-09-16T14:48:55.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030278330000000002 2024-09-16T14:48:55.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:55.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968651+0000", 2024-09-16T14:48:55.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005964764 2024-09-16T14:48:55.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:55.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968692+0000", 2024-09-16T14:48:55.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0374999999999999e-05 2024-09-16T14:48:55.014 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.014 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:55.014 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.014 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:370 3.8s0 3:1f96bdfc:::smithi04040151-37:head [append 0~294912] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:48:55.014 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.959879+0000", 2024-09-16T14:48:55.014 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9763927130000001, 2024-09-16T14:48:55.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089453780000000004, 2024-09-16T14:48:55.014 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:55.015 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:55.015 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:55.015 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:55.015 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:55.015 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 370 2024-09-16T14:48:55.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.015 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:55.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:55.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.959879+0000", 2024-09-16T14:48:55.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:55.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.959879+0000", 2024-09-16T14:48:55.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:55.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.959878+0000", 2024-09-16T14:48:55.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:55.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:55.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960381+0000", 2024-09-16T14:48:55.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000502871 2024-09-16T14:48:55.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:55.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960395+0000", 2024-09-16T14:48:55.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3997e-05 2024-09-16T14:48:55.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:55.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960403+0000", 2024-09-16T14:48:55.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8960000000000003e-06 2024-09-16T14:48:55.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960433+0000", 2024-09-16T14:48:55.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0239999999999998e-05 2024-09-16T14:48:55.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:55.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960514+0000", 2024-09-16T14:48:55.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0451000000000006e-05 2024-09-16T14:48:55.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:55.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.961193+0000", 2024-09-16T14:48:55.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000679804 2024-09-16T14:48:55.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:55.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.963712+0000", 2024-09-16T14:48:55.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0025186549999999999 2024-09-16T14:48:55.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:55.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968794+0000", 2024-09-16T14:48:55.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050814529999999997 2024-09-16T14:48:55.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:55.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968825+0000", 2024-09-16T14:48:55.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1300999999999997e-05 2024-09-16T14:48:55.020 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.020 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:55.021 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.021 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:371 3.8s0 3:1f96bdfc:::smithi04040151-37:head [setxattr _header (56) in=7b] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:48:55.021 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.960414+0000", 2024-09-16T14:48:55.021 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9758583229999997, 2024-09-16T14:48:55.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088134779999999996, 2024-09-16T14:48:55.021 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:55.021 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:55.021 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:55.021 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:55.022 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:55.022 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 371 2024-09-16T14:48:55.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.022 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:55.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:55.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960414+0000", 2024-09-16T14:48:55.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:55.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960414+0000", 2024-09-16T14:48:55.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:55.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960413+0000", 2024-09-16T14:48:55.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:55.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:55.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960417+0000", 2024-09-16T14:48:55.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0330000000000002e-06 2024-09-16T14:48:55.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:55.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960424+0000", 2024-09-16T14:48:55.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0829999999999998e-06 2024-09-16T14:48:55.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:55.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960434+0000", 2024-09-16T14:48:55.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7019999999999996e-06 2024-09-16T14:48:55.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.961938+0000", 2024-09-16T14:48:55.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0015041340000000001 2024-09-16T14:48:55.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:55.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962037+0000", 2024-09-16T14:48:55.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9324e-05 2024-09-16T14:48:55.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:55.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962471+0000", 2024-09-16T14:48:55.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000433798 2024-09-16T14:48:55.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:55.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.964825+0000", 2024-09-16T14:48:55.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0023542699999999999 2024-09-16T14:48:55.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:55.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.969185+0000", 2024-09-16T14:48:55.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0043595029999999998 2024-09-16T14:48:55.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:55.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.969227+0000", 2024-09-16T14:48:55.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2366000000000001e-05 2024-09-16T14:48:55.027 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.027 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:55.027 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.027 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:372 3.8s0 3:1f96bdfc:::smithi04040151-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:48:55.027 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:48:55.027 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.9758311280000003, 2024-09-16T14:48:55.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010116534999999999, 2024-09-16T14:48:55.027 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:55.028 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:55.028 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:55.028 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:55.028 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:55.028 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 372 2024-09-16T14:48:55.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.028 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:55.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:55.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:48:55.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:55.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:48:55.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:55.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:48:55.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:55.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:55.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960445+0000", 2024-09-16T14:48:55.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0579999999999999e-06 2024-09-16T14:48:55.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:55.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960452+0000", 2024-09-16T14:48:55.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5360000000000004e-06 2024-09-16T14:48:55.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:55.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960458+0000", 2024-09-16T14:48:55.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3260000000000003e-06 2024-09-16T14:48:55.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962902+0000", 2024-09-16T14:48:55.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002444313 2024-09-16T14:48:55.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:48:55.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962973+0000", 2024-09-16T14:48:55.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1179000000000004e-05 2024-09-16T14:48:55.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.969538+0000", 2024-09-16T14:48:55.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006565169 2024-09-16T14:48:55.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:55.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.969635+0000", 2024-09-16T14:48:55.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6427000000000004e-05 2024-09-16T14:48:55.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:55.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.970558+0000", 2024-09-16T14:48:55.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00092299600000000004 2024-09-16T14:48:55.033 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.033 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:55.033 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.034 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:386 3.8s0 3:1f96bdfc:::smithi04040151-37:head [read 0~2686976,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:48:55.034 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.864440+0000", 2024-09-16T14:48:55.034 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.071832296, 2024-09-16T14:48:55.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088364930000000008, 2024-09-16T14:48:55.034 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:55.034 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:55.034 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:55.034 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:55.034 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:55.034 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 386 2024-09-16T14:48:55.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.035 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:55.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:55.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864440+0000", 2024-09-16T14:48:55.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:55.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864440+0000", 2024-09-16T14:48:55.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:55.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864439+0000", 2024-09-16T14:48:55.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:55.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:55.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864444+0000", 2024-09-16T14:48:55.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2780000000000004e-06 2024-09-16T14:48:55.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:55.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864452+0000", 2024-09-16T14:48:55.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.32e-06 2024-09-16T14:48:55.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:55.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864458+0000", 2024-09-16T14:48:55.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5419999999999996e-06 2024-09-16T14:48:55.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.865058+0000", 2024-09-16T14:48:55.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00060041900000000004 2024-09-16T14:48:55.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:55.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.865115+0000", 2024-09-16T14:48:55.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6678999999999997e-05 2024-09-16T14:48:55.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:55.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873276+0000", 2024-09-16T14:48:55.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081617029999999993 2024-09-16T14:48:55.038 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.039 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:55.039 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.039 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:387 3.8s0 3:1f96bdfc:::smithi04040151-37:head [read 0~2686976,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:48:55.039 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:48:55.039 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.071809104, 2024-09-16T14:48:55.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010154198999999999, 2024-09-16T14:48:55.039 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:55.039 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:55.040 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:55.040 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:55.040 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:55.040 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 387 2024-09-16T14:48:55.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.040 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:55.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:55.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:48:55.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:55.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:48:55.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:55.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:48:55.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:55.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:55.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864465+0000", 2024-09-16T14:48:55.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.774e-06 2024-09-16T14:48:55.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:55.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864472+0000", 2024-09-16T14:48:55.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0040000000000002e-06 2024-09-16T14:48:55.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:55.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864477+0000", 2024-09-16T14:48:55.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0779999999999997e-06 2024-09-16T14:48:55.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.866679+0000", 2024-09-16T14:48:55.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00220228 2024-09-16T14:48:55.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:55.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.866770+0000", 2024-09-16T14:48:55.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0952999999999995e-05 2024-09-16T14:48:55.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:55.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874617+0000", 2024-09-16T14:48:55.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007847527 2024-09-16T14:48:55.044 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.044 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:55.044 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.044 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:398 3.as0 3:572aff57:::smithi04040151-20:head [append 0~507904] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:48:55.044 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:48:55.044 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.057194698, 2024-09-16T14:48:55.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010447988, 2024-09-16T14:48:55.045 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:55.045 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:55.045 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:55.045 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:55.045 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:55.045 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 398 2024-09-16T14:48:55.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.045 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:55.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:55.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:48:55.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:55.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:48:55.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:55.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879076+0000", 2024-09-16T14:48:55.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:48:55.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:55.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881851+0000", 2024-09-16T14:48:55.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0027753420000000001 2024-09-16T14:48:55.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:55.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881863+0000", 2024-09-16T14:48:55.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2203e-05 2024-09-16T14:48:55.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:55.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881871+0000", 2024-09-16T14:48:55.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4059999999999999e-06 2024-09-16T14:48:55.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881924+0000", 2024-09-16T14:48:55.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2932000000000001e-05 2024-09-16T14:48:55.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:55.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882118+0000", 2024-09-16T14:48:55.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019413500000000001 2024-09-16T14:48:55.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:55.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884114+0000", 2024-09-16T14:48:55.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001996154 2024-09-16T14:48:55.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:55.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889295+0000", 2024-09-16T14:48:55.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051809899999999999 2024-09-16T14:48:55.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:55.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889480+0000", 2024-09-16T14:48:55.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018519700000000001 2024-09-16T14:48:55.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:55.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889525+0000", 2024-09-16T14:48:55.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5399999999999999e-05 2024-09-16T14:48:55.050 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.050 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:55.050 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.051 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:399 3.as0 3:572aff57:::smithi04040151-20:head [append 0~507904] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:48:55.051 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.882057+0000", 2024-09-16T14:48:55.051 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.0542151980000001, 2024-09-16T14:48:55.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0093536399999999999, 2024-09-16T14:48:55.051 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:55.051 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:55.051 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:55.051 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:55.051 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:55.051 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 399 2024-09-16T14:48:55.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.052 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:55.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:55.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882057+0000", 2024-09-16T14:48:55.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:55.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882057+0000", 2024-09-16T14:48:55.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:55.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882056+0000", 2024-09-16T14:48:55.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:55.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:55.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882502+0000", 2024-09-16T14:48:55.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000446685 2024-09-16T14:48:55.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:55.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882514+0000", 2024-09-16T14:48:55.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1941e-05 2024-09-16T14:48:55.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:55.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882523+0000", 2024-09-16T14:48:55.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3860000000000007e-06 2024-09-16T14:48:55.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885049+0000", 2024-09-16T14:48:55.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0025259929999999998 2024-09-16T14:48:55.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:55.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885157+0000", 2024-09-16T14:48:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108651 2024-09-16T14:48:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886157+0000", 2024-09-16T14:48:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001000061 2024-09-16T14:48:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.891223+0000", 2024-09-16T14:48:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050660089999999998 2024-09-16T14:48:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.891355+0000", 2024-09-16T14:48:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013204699999999999 2024-09-16T14:48:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.891411+0000", 2024-09-16T14:48:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5214999999999999e-05 2024-09-16T14:48:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:400 3.as0 3:572aff57:::smithi04040151-20:head [append 0~262144] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:48:55.056 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:48:55.057 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.0526856680000001, 2024-09-16T14:48:55.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010619327, 2024-09-16T14:48:55.057 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:55.057 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:55.057 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:55.057 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:55.057 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:55.057 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 400 2024-09-16T14:48:55.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.057 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:55.057 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.057 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:55.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:48:55.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:55.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:48:55.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:55.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883585+0000", 2024-09-16T14:48:55.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:48:55.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:55.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883868+0000", 2024-09-16T14:48:55.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028385599999999997 2024-09-16T14:48:55.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:55.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883880+0000", 2024-09-16T14:48:55.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1181e-05 2024-09-16T14:48:55.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:55.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883888+0000", 2024-09-16T14:48:55.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.926e-06 2024-09-16T14:48:55.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887739+0000", 2024-09-16T14:48:55.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003851658 2024-09-16T14:48:55.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:55.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887849+0000", 2024-09-16T14:48:55.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109616 2024-09-16T14:48:55.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:55.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.888684+0000", 2024-09-16T14:48:55.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083518000000000004 2024-09-16T14:48:55.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.061 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:55.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894010+0000", 2024-09-16T14:48:55.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053260670000000003 2024-09-16T14:48:55.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:55.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894143+0000", 2024-09-16T14:48:55.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000133077 2024-09-16T14:48:55.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:55.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894206+0000", 2024-09-16T14:48:55.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2658999999999998e-05 2024-09-16T14:48:55.063 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.063 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:55.063 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.063 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.063 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:401 3.as0 3:572aff57:::smithi04040151-20:head [setxattr _header (57) in=7b] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:48:55.063 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:48:55.063 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.0523757439999999, 2024-09-16T14:48:55.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010524774000000001, 2024-09-16T14:48:55.063 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:55.063 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:55.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:55.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:55.063 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:55.064 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 401 2024-09-16T14:48:55.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.064 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:55.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:55.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:48:55.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.064 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:55.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:48:55.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:55.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:48:55.065 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:55.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:55.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883900+0000", 2024-09-16T14:48:55.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1400000000000002e-06 2024-09-16T14:48:55.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:55.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883908+0000", 2024-09-16T14:48:55.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2689999999999999e-06 2024-09-16T14:48:55.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:55.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883913+0000", 2024-09-16T14:48:55.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-16T14:48:55.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889560+0000", 2024-09-16T14:48:55.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005647165 2024-09-16T14:48:55.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:55.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889627+0000", 2024-09-16T14:48:55.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6951000000000003e-05 2024-09-16T14:48:55.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:55.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889898+0000", 2024-09-16T14:48:55.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270709 2024-09-16T14:48:55.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:55.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894012+0000", 2024-09-16T14:48:55.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041137810000000004 2024-09-16T14:48:55.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.068 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:55.068 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894344+0000", 2024-09-16T14:48:55.068 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000332121 2024-09-16T14:48:55.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.069 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.069 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:55.069 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894421+0000", 2024-09-16T14:48:55.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7406999999999999e-05 2024-09-16T14:48:55.070 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.070 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:55.070 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.070 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:402 3.as0 3:572aff57:::smithi04040151-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:48:55.071 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:48:55.071 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.052353868, 2024-09-16T14:48:55.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011864743000000001, 2024-09-16T14:48:55.071 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:55.071 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:55.071 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:55.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:55.072 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:55.072 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 402 2024-09-16T14:48:55.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.072 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:55.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:55.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:48:55.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:55.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:48:55.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:55.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:55.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:48:55.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:55.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:55.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883921+0000", 2024-09-16T14:48:55.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7520000000000001e-06 2024-09-16T14:48:55.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:55.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883927+0000", 2024-09-16T14:48:55.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4679999999999997e-06 2024-09-16T14:48:55.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:55.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883932+0000", 2024-09-16T14:48:55.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1440000000000002e-06 2024-09-16T14:48:55.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890477+0000", 2024-09-16T14:48:55.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065446430000000002 2024-09-16T14:48:55.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:48:55.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890586+0000", 2024-09-16T14:48:55.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010909 2024-09-16T14:48:55.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:55.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894750+0000", 2024-09-16T14:48:55.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00416359 2024-09-16T14:48:55.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:55.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894834+0000", 2024-09-16T14:48:55.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4596000000000001e-05 2024-09-16T14:48:55.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:55.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:55.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:55.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895783+0000", 2024-09-16T14:48:55.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00094892000000000002 2024-09-16T14:48:55.077 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.077 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:55.077 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.077 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:55.077 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:55.077 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:55.078 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:48:55.079 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:55.078+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:55.080 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:48:55.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:55.178+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:55.196 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:55.196 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:48:55.196 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:48:55.196 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:55.201 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:48:55.201 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:48:55.205 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:48:55.218 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:48:55.218 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:48:55.234 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:48:55.234 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:48:55.249 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:48:55.250 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:48:55.266 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:48:55.266 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:48:55.280 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:55.279+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:55.282 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:48:55.283 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:48:55.300 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:48:55.300 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:48:55.310 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:55.310 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:55.316 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:48:55.316 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:48:55.319 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:48:55.333 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:48:55.333 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:48:55.349 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:48:55.349 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:48:55.366 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:48:55.366 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:48:55.381 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:55.380+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:55.382 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:48:55.382 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:48:55.423 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:55.423 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:55.431 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:48:55.483 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:55.482+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:55.540 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:55.540 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:55.549 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:48:55.585 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:55.584+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:55.685 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:55.684+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:55.696 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:55.696 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:55.705 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:48:55.787 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:55.786+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:55.850 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is set 2024-09-16T14:48:55.850 INFO:tasks.rados.rados.0.smithi040.stdout:171: done (0 left) 2024-09-16T14:48:55.850 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 7 current snap is 12 2024-09-16T14:48:55.850 INFO:tasks.rados.rados.0.smithi040.stdout:172: seq_num 77 ranges {0=524288,524288=524288,1048576=245760} 2024-09-16T14:48:55.852 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:55.852 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:55.852 INFO:tasks.rados.rados.0.smithi040.stdout:172: writing smithi04040151-7 from 0 to 524288 tid 1 2024-09-16T14:48:55.856 INFO:tasks.rados.rados.0.smithi040.stdout:172: writing smithi04040151-7 from 524288 to 1048576 tid 2 2024-09-16T14:48:55.857 INFO:tasks.rados.rados.0.smithi040.stdout:172: writing smithi04040151-7 from 1048576 to 1294336 tid 3 2024-09-16T14:48:55.857 INFO:tasks.rados.rados.0.smithi040.stdout:173: delete oid 45 current snap is 12 2024-09-16T14:48:55.861 INFO:tasks.rados.rados.0.smithi040.stdout:172: finishing write tid 1 to smithi04040151-7 2024-09-16T14:48:55.863 INFO:tasks.rados.rados.0.smithi040.stdout:172: finishing write tid 2 to smithi04040151-7 2024-09-16T14:48:55.865 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:48:55.866 INFO:tasks.rados.rados.0.smithi040.stdout:172: finishing write tid 3 to smithi04040151-7 2024-09-16T14:48:55.866 INFO:tasks.rados.rados.0.smithi040.stdout:173: done (1 left) 2024-09-16T14:48:55.866 INFO:tasks.rados.rados.0.smithi040.stdout:174: delete oid 10 current snap is 12 2024-09-16T14:48:55.866 INFO:tasks.rados.rados.0.smithi040.stdout:172: finishing write tid 4 to smithi04040151-7 2024-09-16T14:48:55.868 INFO:tasks.rados.rados.0.smithi040.stdout:172: finishing write tid 5 to smithi04040151-7 2024-09-16T14:48:55.868 INFO:tasks.rados.rados.0.smithi040.stdout:172: oid 7 updating version 0 to 19 2024-09-16T14:48:55.869 INFO:tasks.rados.rados.0.smithi040.stdout:172: oid 7 version 19 is already newer than 18 2024-09-16T14:48:55.869 INFO:tasks.rados.rados.0.smithi040.stdout:172: oid 7 version 19 is already newer than 17 2024-09-16T14:48:55.869 INFO:tasks.rados.rados.0.smithi040.stdout:172: oid 7 updating version 19 to 20 2024-09-16T14:48:55.869 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 7 v 20 (ObjNum 77 snap 12 seq_num 77) dirty exists 2024-09-16T14:48:55.869 INFO:tasks.rados.rados.0.smithi040.stdout:172: left oid 7 (ObjNum 77 snap 12 seq_num 77) 2024-09-16T14:48:55.871 INFO:tasks.rados.rados.0.smithi040.stdout:172: done (1 left) 2024-09-16T14:48:55.871 INFO:tasks.rados.rados.0.smithi040.stdout:174: done (0 left) 2024-09-16T14:48:55.871 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 22 current snap is 12 2024-09-16T14:48:55.871 INFO:tasks.rados.rados.0.smithi040.stdout:175: seq_num 80 ranges {0=589824,589824=573440,1163264=196608} 2024-09-16T14:48:55.874 INFO:tasks.rados.rados.0.smithi040.stdout:175: writing smithi04040151-22 from 0 to 589824 tid 1 2024-09-16T14:48:55.878 INFO:tasks.rados.rados.0.smithi040.stdout:175: writing smithi04040151-22 from 589824 to 1163264 tid 2 2024-09-16T14:48:55.879 INFO:tasks.rados.rados.0.smithi040.stdout:175: writing smithi04040151-22 from 1163264 to 1359872 tid 3 2024-09-16T14:48:55.879 INFO:tasks.rados.rados.0.smithi040.stdout:176: rollback oid 49 current snap is 12 2024-09-16T14:48:55.879 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 49 to 10 2024-09-16T14:48:55.879 INFO:tasks.rados.rados.0.smithi040.stdout:177: delete oid 37 current snap is 12 2024-09-16T14:48:55.882 INFO:tasks.rados.rados.0.smithi040.stdout:175: finishing write tid 1 to smithi04040151-22 2024-09-16T14:48:55.884 INFO:tasks.rados.rados.0.smithi040.stdout:175: finishing write tid 2 to smithi04040151-22 2024-09-16T14:48:55.886 INFO:tasks.rados.rados.0.smithi040.stdout:177: done (2 left) 2024-09-16T14:48:55.886 INFO:tasks.rados.rados.0.smithi040.stdout:178: snap_create 2024-09-16T14:48:55.887 INFO:tasks.rados.rados.0.smithi040.stdout:175: finishing write tid 3 to smithi04040151-22 2024-09-16T14:48:55.887 INFO:tasks.rados.rados.0.smithi040.stdout:176: finishing rollback tid 0 to smithi04040151-49 2024-09-16T14:48:55.887 INFO:tasks.rados.rados.0.smithi040.stdout:175: finishing write tid 4 to smithi04040151-22 2024-09-16T14:48:55.887 INFO:tasks.rados.rados.0.smithi040.stdout:176: finishing rollback tid 1 to smithi04040151-49 2024-09-16T14:48:55.887 INFO:tasks.rados.rados.0.smithi040.stdout:176: finishing rollback tid 2 to smithi04040151-49 2024-09-16T14:48:55.888 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 49 v 18 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-16T14:48:55.888 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:55.887+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:55.889 INFO:tasks.rados.rados.0.smithi040.stdout:175: finishing write tid 5 to smithi04040151-22 2024-09-16T14:48:55.889 INFO:tasks.rados.rados.0.smithi040.stdout:175: oid 22 updating version 0 to 18 2024-09-16T14:48:55.889 INFO:tasks.rados.rados.0.smithi040.stdout:175: oid 22 updating version 18 to 19 2024-09-16T14:48:55.889 INFO:tasks.rados.rados.0.smithi040.stdout:175: oid 22 updating version 19 to 20 2024-09-16T14:48:55.889 INFO:tasks.rados.rados.0.smithi040.stdout:175: oid 22 version 20 is already newer than 17 2024-09-16T14:48:55.889 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 22 v 20 (ObjNum 80 snap 12 seq_num 80) dirty exists 2024-09-16T14:48:55.889 INFO:tasks.rados.rados.0.smithi040.stdout:175: left oid 22 (ObjNum 80 snap 12 seq_num 80) 2024-09-16T14:48:55.977 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:55.977 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:55.985 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:48:55.989 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:55.988+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:56.090 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:56.089+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:56.133 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:56.133 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:56.142 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:48:56.191 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:56.190+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:56.249 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:56.249 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:56.258 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:48:56.292 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:48:56.291+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:48:56.365 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:56.366 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:56.375 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:48:56.393 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:56.392+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:48:56.481 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:56.481 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:56.490 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:48:56.494 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:56.494+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:56.595 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:56.594+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:56.596 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:56.596 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:56.605 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:48:56.697 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:56.696+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:56.752 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:56.752 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:56.761 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:48:56.802 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:56.801+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:56.872 INFO:tasks.rados.rados.0.smithi040.stdout:175: done (2 left) 2024-09-16T14:48:56.873 INFO:tasks.rados.rados.0.smithi040.stdout:176: done (1 left) 2024-09-16T14:48:56.873 INFO:tasks.rados.rados.0.smithi040.stdout:178: done (0 left) 2024-09-16T14:48:56.873 INFO:tasks.rados.rados.0.smithi040.stdout:179: setattr oid 2 current snap is 13 2024-09-16T14:48:56.874 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 40 current snap is 13 2024-09-16T14:48:56.874 INFO:tasks.rados.rados.0.smithi040.stdout:180: seq_num 82 ranges {81920=622592,704512=475136,1179648=294912} 2024-09-16T14:48:56.878 INFO:tasks.rados.rados.0.smithi040.stdout:180: writing smithi04040151-40 from 81920 to 704512 tid 1 2024-09-16T14:48:56.881 INFO:tasks.rados.rados.0.smithi040.stdout:180: writing smithi04040151-40 from 704512 to 1179648 tid 2 2024-09-16T14:48:56.884 INFO:tasks.rados.rados.0.smithi040.stdout:180: writing smithi04040151-40 from 1179648 to 1474560 tid 3 2024-09-16T14:48:56.884 INFO:tasks.rados.rados.0.smithi040.stdout:181: rollback oid 43 current snap is 13 2024-09-16T14:48:56.884 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 43 to 12 2024-09-16T14:48:56.884 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 2 v 16 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-16T14:48:56.884 INFO:tasks.rados.rados.0.smithi040.stdout:179: done (2 left) 2024-09-16T14:48:56.884 INFO:tasks.rados.rados.0.smithi040.stdout:182: read oid 14 snap 5 2024-09-16T14:48:56.884 INFO:tasks.rados.rados.0.smithi040.stdout:182: expect (ObjNum 54 snap 1 seq_num 54) 2024-09-16T14:48:56.884 INFO:tasks.rados.rados.0.smithi040.stdout:183: snap_remove snap 9 2024-09-16T14:48:56.898 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:56.897+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:56.916 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:56.916 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:56.924 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:48:57.000 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:56.999+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:57.031 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:57.031 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:57.039 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:48:57.100 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:57.099+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:57.155 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:57.156 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:57.164 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:48:57.201 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:57.200+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:57.301 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:57.301+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:57.308 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:57.308 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:57.317 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:48:57.393 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:48:57.403 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:57.402+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:57.419 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:57.419 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:57.428 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:48:57.504 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:57.503+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:57.522 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:48:57.522 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:48:57.531 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:48:57.531 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:48:57.538 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:48:57.538 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:48:57.539 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:48:57.554 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:48:57.554 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:48:57.570 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:48:57.571 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:48:57.587 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:48:57.587 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:48:57.603 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:48:57.604 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:48:57.604 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:57.604+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:48:57.608 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 6] 2024-09-16T14:48:57.608 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:48:57.608 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-16T14:48:57.608 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-16T14:48:57.619 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:48:57.620 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:48:57.636 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:48:57.637 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:48:57.643 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:57.643 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:48:57.643 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:48:57.643 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:48:57.643 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:57.652 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:48:57.653 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:48:57.653 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:48:57.670 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:48:57.670 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:48:57.687 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:48:57.687 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:48:57.704 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:48:57.704 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:48:57.705 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:57.704+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:57.796 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:57.796 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:48:57.796 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:48:57.796 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:48:57.796 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.796 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-16T14:48:57.796 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:24.146547+0000", 2024-09-16T14:48:57.796 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 33.647113877999999, 2024-09-16T14:48:57.796 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003423672, 2024-09-16T14:48:57.796 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.797 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:57.797 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:57.797 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4272", 2024-09-16T14:48:57.797 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/2494424138", 2024-09-16T14:48:57.797 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 1 2024-09-16T14:48:57.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.797 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.797 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.797 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.797 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.146547+0000", 2024-09-16T14:48:57.797 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.797 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.798 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.798 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.146547+0000", 2024-09-16T14:48:57.798 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.798 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.798 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.798 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.798 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.146544+0000", 2024-09-16T14:48:57.798 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999971 2024-09-16T14:48:57.798 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.798 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.798 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.798 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.146555+0000", 2024-09-16T14:48:57.798 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0784999999999999e-05 2024-09-16T14:48:57.798 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.799 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.799 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.799 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.146566+0000", 2024-09-16T14:48:57.799 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0665999999999999e-05 2024-09-16T14:48:57.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.799 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.799 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.799 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.146584+0000", 2024-09-16T14:48:57.799 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8247e-05 2024-09-16T14:48:57.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.799 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.799 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.799 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.146629+0000", 2024-09-16T14:48:57.800 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5584000000000003e-05 2024-09-16T14:48:57.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.800 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.800 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:48:57.800 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.146637+0000", 2024-09-16T14:48:57.800 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2609999999999996e-06 2024-09-16T14:48:57.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.800 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.800 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.800 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.148076+0000", 2024-09-16T14:48:57.800 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0014393660000000001 2024-09-16T14:48:57.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.800 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.801 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:57.801 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.148200+0000", 2024-09-16T14:48:57.801 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000123974 2024-09-16T14:48:57.801 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.801 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.801 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 2", 2024-09-16T14:48:57.801 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.148297+0000", 2024-09-16T14:48:57.801 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6607000000000005e-05 2024-09-16T14:48:57.801 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.801 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.801 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:57.801 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.149404+0000", 2024-09-16T14:48:57.801 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011078069999999999 2024-09-16T14:48:57.802 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.802 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.802 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:57.802 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.149919+0000", 2024-09-16T14:48:57.802 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00051455700000000001 2024-09-16T14:48:57.802 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.802 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.802 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:57.802 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.149951+0000", 2024-09-16T14:48:57.802 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1541999999999997e-05 2024-09-16T14:48:57.802 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.802 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.802 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.803 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:24.149971+0000", 2024-09-16T14:48:57.803 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0037000000000001e-05 2024-09-16T14:48:57.803 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.803 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.803 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.803 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.803 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.803 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4272.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-16T14:48:57.803 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:25.147135+0000", 2024-09-16T14:48:57.803 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 32.646525849, 2024-09-16T14:48:57.803 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045540479999999998, 2024-09-16T14:48:57.803 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.803 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:57.803 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:57.804 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4272", 2024-09-16T14:48:57.804 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/2494424138", 2024-09-16T14:48:57.804 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 5 2024-09-16T14:48:57.804 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.804 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.804 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.804 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.804 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.147135+0000", 2024-09-16T14:48:57.804 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.804 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.804 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.804 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.804 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.147135+0000", 2024-09-16T14:48:57.805 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.805 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.805 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.805 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.805 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.147133+0000", 2024-09-16T14:48:57.805 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:48:57.805 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.805 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.805 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.805 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.147149+0000", 2024-09-16T14:48:57.805 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6181999999999999e-05 2024-09-16T14:48:57.805 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.805 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.805 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.147167+0000", 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7890999999999999e-05 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.147176+0000", 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1910000000000006e-06 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.147242+0000", 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.5574999999999995e-05 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.806 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:48:57.807 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.147247+0000", 2024-09-16T14:48:57.807 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3290000000000001e-06 2024-09-16T14:48:57.807 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.807 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.807 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.807 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.148745+0000", 2024-09-16T14:48:57.807 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0014982489999999999 2024-09-16T14:48:57.807 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.807 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.807 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:57.807 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.148898+0000", 2024-09-16T14:48:57.807 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015255499999999999 2024-09-16T14:48:57.807 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.807 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.808 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 1", 2024-09-16T14:48:57.808 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.149030+0000", 2024-09-16T14:48:57.808 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00013256699999999999 2024-09-16T14:48:57.808 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.808 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.808 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:57.808 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.150622+0000", 2024-09-16T14:48:57.808 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015915790000000001 2024-09-16T14:48:57.808 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.808 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.808 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:57.808 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.151600+0000", 2024-09-16T14:48:57.808 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00097822300000000007 2024-09-16T14:48:57.808 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.809 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:57.809 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.151644+0000", 2024-09-16T14:48:57.809 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3826999999999999e-05 2024-09-16T14:48:57.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.809 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.809 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:25.151689+0000", 2024-09-16T14:48:57.809 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5019999999999999e-05 2024-09-16T14:48:57.809 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.809 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.809 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.810 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4272.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-16T14:48:57.810 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:26.157385+0000", 2024-09-16T14:48:57.810 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 31.636275607999998, 2024-09-16T14:48:57.810 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036930489999999999, 2024-09-16T14:48:57.810 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.810 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:57.810 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:57.810 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4272", 2024-09-16T14:48:57.810 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/2494424138", 2024-09-16T14:48:57.810 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 7 2024-09-16T14:48:57.810 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.810 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.810 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.811 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.811 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.157385+0000", 2024-09-16T14:48:57.811 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.811 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.811 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.811 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.811 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.157385+0000", 2024-09-16T14:48:57.811 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.811 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.811 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.811 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.811 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.157383+0000", 2024-09-16T14:48:57.811 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999976 2024-09-16T14:48:57.812 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.812 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.812 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.812 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.157396+0000", 2024-09-16T14:48:57.812 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3392e-05 2024-09-16T14:48:57.812 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.812 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.812 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.812 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.157414+0000", 2024-09-16T14:48:57.812 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7753000000000001e-05 2024-09-16T14:48:57.812 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.812 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.812 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.813 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.157434+0000", 2024-09-16T14:48:57.813 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0551999999999999e-05 2024-09-16T14:48:57.813 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.813 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.813 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.813 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.157535+0000", 2024-09-16T14:48:57.813 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000100932 2024-09-16T14:48:57.813 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.813 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.813 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:48:57.813 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.157543+0000", 2024-09-16T14:48:57.813 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.6480000000000003e-06 2024-09-16T14:48:57.813 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.813 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.814 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.814 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.159270+0000", 2024-09-16T14:48:57.814 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001727092 2024-09-16T14:48:57.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.814 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.814 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:57.814 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.159347+0000", 2024-09-16T14:48:57.814 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.6919e-05 2024-09-16T14:48:57.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.814 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.814 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 1", 2024-09-16T14:48:57.814 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.159458+0000", 2024-09-16T14:48:57.814 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000110612 2024-09-16T14:48:57.815 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.815 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.815 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:57.815 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.160100+0000", 2024-09-16T14:48:57.815 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00064230600000000004 2024-09-16T14:48:57.815 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.815 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.815 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:57.815 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.160985+0000", 2024-09-16T14:48:57.815 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00088509700000000005 2024-09-16T14:48:57.815 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.815 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.815 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:57.816 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.161037+0000", 2024-09-16T14:48:57.816 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2030999999999997e-05 2024-09-16T14:48:57.816 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.816 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.816 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.816 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:26.161078+0000", 2024-09-16T14:48:57.816 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1137999999999998e-05 2024-09-16T14:48:57.816 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.816 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.816 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.816 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.816 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.816 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s4 33/27 ECSubWrite(tid=75, reqid=client.4500.0:225, at_version=33'14, trim_to=0'0, roll_forward_to=30'10))", 2024-09-16T14:48:57.817 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:39.916518+0000", 2024-09-16T14:48:57.817 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 17.877142814999999, 2024-09-16T14:48:57.817 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031494539999999999, 2024-09-16T14:48:57.817 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.817 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.817 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.817 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.817 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.817 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.916518+0000", 2024-09-16T14:48:57.817 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.817 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.817 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.818 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.818 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.916518+0000", 2024-09-16T14:48:57.818 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.818 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.818 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.818 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.916517+0000", 2024-09-16T14:48:57.818 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:48:57.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.818 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.818 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.818 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.916539+0000", 2024-09-16T14:48:57.818 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1815999999999999e-05 2024-09-16T14:48:57.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.819 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.819 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.819 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.916551+0000", 2024-09-16T14:48:57.819 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1547e-05 2024-09-16T14:48:57.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.819 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.819 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.819 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.916559+0000", 2024-09-16T14:48:57.819 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0369999999999995e-06 2024-09-16T14:48:57.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.819 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.819 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.819 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.916608+0000", 2024-09-16T14:48:57.820 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9752000000000001e-05 2024-09-16T14:48:57.820 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.820 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.820 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.820 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.916617+0000", 2024-09-16T14:48:57.820 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.405e-06 2024-09-16T14:48:57.820 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.820 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.820 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.820 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.919641+0000", 2024-09-16T14:48:57.820 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003024433 2024-09-16T14:48:57.820 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.820 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.820 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.919667+0000", 2024-09-16T14:48:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6191e-05 2024-09-16T14:48:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s4 33/27 ECSubWrite(tid=76, reqid=client.4500.0:227, at_version=33'16, trim_to=0'0, roll_forward_to=30'10))", 2024-09-16T14:48:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:39.917607+0000", 2024-09-16T14:48:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 17.876053949999999, 2024-09-16T14:48:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048754200000000001, 2024-09-16T14:48:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.821 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.824 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.824 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.824 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917607+0000", 2024-09-16T14:48:57.824 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.824 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.824 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.824 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.824 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917607+0000", 2024-09-16T14:48:57.824 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.824 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.825 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.825 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.825 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917606+0000", 2024-09-16T14:48:57.825 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:48:57.825 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.825 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.825 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.825 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917631+0000", 2024-09-16T14:48:57.825 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4905e-05 2024-09-16T14:48:57.825 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.825 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.825 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.825 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917646+0000", 2024-09-16T14:48:57.826 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4640999999999999e-05 2024-09-16T14:48:57.826 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.826 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.826 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.826 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917654+0000", 2024-09-16T14:48:57.826 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0779999999999996e-06 2024-09-16T14:48:57.826 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.826 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.826 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.826 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917697+0000", 2024-09-16T14:48:57.826 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3245999999999999e-05 2024-09-16T14:48:57.826 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.826 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.826 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917705+0000", 2024-09-16T14:48:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0350000000000001e-06 2024-09-16T14:48:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.922460+0000", 2024-09-16T14:48:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0047549569999999998 2024-09-16T14:48:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.922482+0000", 2024-09-16T14:48:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2280999999999999e-05 2024-09-16T14:48:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.828 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.828 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.828 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.828 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.6s7 37/27 ECSubWrite(tid=23, reqid=osd.0.0:23, at_version=37'17, trim_to=0'0, roll_forward_to=35'13))", 2024-09-16T14:48:57.828 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.916820+0000", 2024-09-16T14:48:57.828 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.876841188, 2024-09-16T14:48:57.828 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036128309999999999, 2024-09-16T14:48:57.828 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.828 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.828 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.828 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.828 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.829 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.916820+0000", 2024-09-16T14:48:57.829 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.829 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.829 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.829 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.829 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.916820+0000", 2024-09-16T14:48:57.829 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.829 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.829 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.829 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.829 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.916819+0000", 2024-09-16T14:48:57.829 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:48:57.829 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.829 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.830 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.830 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.916845+0000", 2024-09-16T14:48:57.830 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6253000000000001e-05 2024-09-16T14:48:57.830 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.830 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.830 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.830 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.916865+0000", 2024-09-16T14:48:57.830 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0479000000000002e-05 2024-09-16T14:48:57.830 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.830 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.830 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.830 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.916875+0000", 2024-09-16T14:48:57.830 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8989999999999996e-06 2024-09-16T14:48:57.830 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.831 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.831 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.831 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.916925+0000", 2024-09-16T14:48:57.831 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9558000000000002e-05 2024-09-16T14:48:57.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.831 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.831 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.831 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.916935+0000", 2024-09-16T14:48:57.831 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0458999999999999e-05 2024-09-16T14:48:57.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.831 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.831 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.831 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920409+0000", 2024-09-16T14:48:57.831 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034739480000000001 2024-09-16T14:48:57.832 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.832 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.832 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.832 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920432+0000", 2024-09-16T14:48:57.832 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3144000000000001e-05 2024-09-16T14:48:57.832 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.832 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.832 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.832 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.832 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.832 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s4 37/27 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=37'32, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:57.832 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921312+0000", 2024-09-16T14:48:57.832 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.872348314, 2024-09-16T14:48:57.833 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0046209789999999999, 2024-09-16T14:48:57.833 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.833 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.833 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.833 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.833 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.833 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921312+0000", 2024-09-16T14:48:57.833 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.833 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.833 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.833 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921312+0000", 2024-09-16T14:48:57.833 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.834 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.834 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.834 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921312+0000", 2024-09-16T14:48:57.834 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:57.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.834 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.834 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.834 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921336+0000", 2024-09-16T14:48:57.834 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3723999999999999e-05 2024-09-16T14:48:57.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.834 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.834 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.834 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921346+0000", 2024-09-16T14:48:57.834 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0349e-05 2024-09-16T14:48:57.835 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.835 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.835 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.835 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921354+0000", 2024-09-16T14:48:57.835 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5750000000000003e-06 2024-09-16T14:48:57.835 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.835 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.835 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.835 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921452+0000", 2024-09-16T14:48:57.835 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8214999999999994e-05 2024-09-16T14:48:57.835 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.835 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.835 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.835 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921463+0000", 2024-09-16T14:48:57.836 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0963e-05 2024-09-16T14:48:57.836 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.836 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.836 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.836 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.925907+0000", 2024-09-16T14:48:57.836 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044440540000000002 2024-09-16T14:48:57.836 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.836 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.836 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.836 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.925933+0000", 2024-09-16T14:48:57.836 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6614999999999998e-05 2024-09-16T14:48:57.836 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.836 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.837 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.837 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.837 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.837 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s5 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:57.837 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921363+0000", 2024-09-16T14:48:57.837 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.872298154999999, 2024-09-16T14:48:57.837 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044828029999999996, 2024-09-16T14:48:57.837 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.837 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.837 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.837 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.837 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.837 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921363+0000", 2024-09-16T14:48:57.837 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.838 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.838 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.838 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.838 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921363+0000", 2024-09-16T14:48:57.838 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.838 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.838 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.838 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.838 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921362+0000", 2024-09-16T14:48:57.838 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:57.838 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.838 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.838 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.838 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921388+0000", 2024-09-16T14:48:57.839 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6095e-05 2024-09-16T14:48:57.839 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.839 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.839 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.839 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921401+0000", 2024-09-16T14:48:57.839 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3025e-05 2024-09-16T14:48:57.839 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.839 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.839 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.839 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921414+0000", 2024-09-16T14:48:57.839 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2123000000000001e-05 2024-09-16T14:48:57.839 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.839 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.839 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.840 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921456+0000", 2024-09-16T14:48:57.840 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2923999999999998e-05 2024-09-16T14:48:57.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.840 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.840 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.840 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921464+0000", 2024-09-16T14:48:57.840 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2309999999999999e-06 2024-09-16T14:48:57.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.840 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.840 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.840 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.925814+0000", 2024-09-16T14:48:57.840 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0043504640000000001 2024-09-16T14:48:57.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.840 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.841 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.841 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.925845+0000", 2024-09-16T14:48:57.841 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1226999999999998e-05 2024-09-16T14:48:57.841 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.841 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.841 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.841 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.841 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.841 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s4 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:57.841 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921852+0000", 2024-09-16T14:48:57.841 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.871808943, 2024-09-16T14:48:57.841 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0065836089999999998, 2024-09-16T14:48:57.841 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.842 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.842 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.842 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.842 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.842 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921852+0000", 2024-09-16T14:48:57.842 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.842 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.842 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.842 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.842 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921852+0000", 2024-09-16T14:48:57.842 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.842 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.842 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.842 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.843 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921851+0000", 2024-09-16T14:48:57.843 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:48:57.843 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.843 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.843 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.843 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921890+0000", 2024-09-16T14:48:57.843 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8668000000000003e-05 2024-09-16T14:48:57.843 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.843 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.843 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.843 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921906+0000", 2024-09-16T14:48:57.843 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6772000000000001e-05 2024-09-16T14:48:57.843 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.843 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.844 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.844 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921920+0000", 2024-09-16T14:48:57.844 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3525e-05 2024-09-16T14:48:57.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.844 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.844 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.844 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923071+0000", 2024-09-16T14:48:57.844 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001150812 2024-09-16T14:48:57.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.844 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.844 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.844 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923079+0000", 2024-09-16T14:48:57.844 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6119999999999999e-06 2024-09-16T14:48:57.845 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.845 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.845 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.845 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.928412+0000", 2024-09-16T14:48:57.845 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053327419999999997 2024-09-16T14:48:57.845 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.845 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.845 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.845 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.928435+0000", 2024-09-16T14:48:57.845 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3419999999999999e-05 2024-09-16T14:48:57.845 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.845 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.845 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.845 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.846 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.846 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.6s7 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:48:57.846 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.923528+0000", 2024-09-16T14:48:57.846 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.870133184, 2024-09-16T14:48:57.846 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049476429999999998, 2024-09-16T14:48:57.846 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.846 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.846 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.846 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.846 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.846 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923528+0000", 2024-09-16T14:48:57.846 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.846 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.847 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.847 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.847 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923528+0000", 2024-09-16T14:48:57.847 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.847 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.847 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.847 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.847 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923527+0000", 2024-09-16T14:48:57.847 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:48:57.847 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.847 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.847 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.847 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923536+0000", 2024-09-16T14:48:57.847 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5999999999999996e-06 2024-09-16T14:48:57.848 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.848 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.848 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.848 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923551+0000", 2024-09-16T14:48:57.848 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4459e-05 2024-09-16T14:48:57.848 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.848 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.848 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.848 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923560+0000", 2024-09-16T14:48:57.848 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5519999999999993e-06 2024-09-16T14:48:57.848 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.848 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.848 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.849 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923614+0000", 2024-09-16T14:48:57.849 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3538000000000001e-05 2024-09-16T14:48:57.849 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.849 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.849 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.849 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923623+0000", 2024-09-16T14:48:57.849 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9299999999999992e-06 2024-09-16T14:48:57.849 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.849 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.849 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.849 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.928446+0000", 2024-09-16T14:48:57.849 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048234369999999999 2024-09-16T14:48:57.849 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.849 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.928475+0000", 2024-09-16T14:48:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8904999999999999e-05 2024-09-16T14:48:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s3 37/27 ECSubWrite(tid=34, reqid=osd.1.0:33, at_version=0'0, trim_to=0'0, roll_forward_to=37'24))", 2024-09-16T14:48:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.931191+0000", 2024-09-16T14:48:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.862470161999999, 2024-09-16T14:48:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050404129999999997, 2024-09-16T14:48:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.851 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.851 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.851 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.851 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931191+0000", 2024-09-16T14:48:57.851 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.851 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.851 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.851 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931191+0000", 2024-09-16T14:48:57.851 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.851 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.851 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.852 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931190+0000", 2024-09-16T14:48:57.852 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:57.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.852 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.852 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.852 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931199+0000", 2024-09-16T14:48:57.852 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5289999999999997e-06 2024-09-16T14:48:57.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.852 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.852 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.852 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931211+0000", 2024-09-16T14:48:57.852 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1600000000000001e-05 2024-09-16T14:48:57.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.853 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.853 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.853 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931217+0000", 2024-09-16T14:48:57.853 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3790000000000003e-06 2024-09-16T14:48:57.853 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.853 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.853 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.853 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931260+0000", 2024-09-16T14:48:57.853 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2577999999999998e-05 2024-09-16T14:48:57.853 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.853 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.853 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.853 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931268+0000", 2024-09-16T14:48:57.853 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0679999999999997e-06 2024-09-16T14:48:57.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.854 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.854 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.936212+0000", 2024-09-16T14:48:57.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049436840000000003 2024-09-16T14:48:57.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.854 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.854 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.936231+0000", 2024-09-16T14:48:57.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9198000000000002e-05 2024-09-16T14:48:57.854 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.854 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.854 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.855 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.855 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s5 37/27 ECSubWrite(tid=17, reqid=osd.4.0:16, at_version=0'0, trim_to=0'0, roll_forward_to=37'14))", 2024-09-16T14:48:57.855 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.933858+0000", 2024-09-16T14:48:57.855 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.859802879, 2024-09-16T14:48:57.855 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034882009999999998, 2024-09-16T14:48:57.855 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.855 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.855 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.855 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.855 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.855 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933858+0000", 2024-09-16T14:48:57.855 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933858+0000", 2024-09-16T14:48:57.856 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933857+0000", 2024-09-16T14:48:57.856 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:48:57.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.857 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933869+0000", 2024-09-16T14:48:57.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1719000000000001e-05 2024-09-16T14:48:57.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.857 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.857 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933885+0000", 2024-09-16T14:48:57.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5798999999999999e-05 2024-09-16T14:48:57.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.857 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.857 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933894+0000", 2024-09-16T14:48:57.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0920000000000004e-06 2024-09-16T14:48:57.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.858 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.858 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933951+0000", 2024-09-16T14:48:57.858 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.7717000000000003e-05 2024-09-16T14:48:57.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.858 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.858 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.858 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933961+0000", 2024-09-16T14:48:57.858 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2220000000000008e-06 2024-09-16T14:48:57.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.858 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.858 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.858 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.937323+0000", 2024-09-16T14:48:57.858 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0033623759999999998 2024-09-16T14:48:57.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.859 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.859 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.859 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.937346+0000", 2024-09-16T14:48:57.859 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3040999999999998e-05 2024-09-16T14:48:57.859 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.859 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.859 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.859 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.859 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s4 37/27 ECSubWrite(tid=121, reqid=osd.3.0:117, at_version=0'0, trim_to=0'0, roll_forward_to=37'34))", 2024-09-16T14:48:57.859 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.934111+0000", 2024-09-16T14:48:57.859 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.85954942, 2024-09-16T14:48:57.859 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003467674, 2024-09-16T14:48:57.860 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.860 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.860 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.860 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.860 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.860 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.934111+0000", 2024-09-16T14:48:57.860 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.860 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.860 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.860 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.860 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.934111+0000", 2024-09-16T14:48:57.860 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.860 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.861 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.861 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.861 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.934110+0000", 2024-09-16T14:48:57.861 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:48:57.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.861 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.861 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.861 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.934123+0000", 2024-09-16T14:48:57.861 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2500000000000001e-05 2024-09-16T14:48:57.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.861 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.861 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.861 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.934137+0000", 2024-09-16T14:48:57.861 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3658999999999999e-05 2024-09-16T14:48:57.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.934145+0000", 2024-09-16T14:48:57.862 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5949999999999999e-06 2024-09-16T14:48:57.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.934190+0000", 2024-09-16T14:48:57.862 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5111999999999998e-05 2024-09-16T14:48:57.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.934198+0000", 2024-09-16T14:48:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.3119999999999996e-06 2024-09-16T14:48:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.937554+0000", 2024-09-16T14:48:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0033569049999999999 2024-09-16T14:48:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.937579+0000", 2024-09-16T14:48:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4528e-05 2024-09-16T14:48:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.863 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.864 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.864 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.864 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.864 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 37/27 ECSubWrite(tid=112, reqid=osd.11.0:110, at_version=0'0, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:48:57.864 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.950192+0000", 2024-09-16T14:48:57.864 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.843469020000001, 2024-09-16T14:48:57.864 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0046078450000000002, 2024-09-16T14:48:57.864 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.864 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.864 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.864 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.864 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.864 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.950192+0000", 2024-09-16T14:48:57.864 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.950192+0000", 2024-09-16T14:48:57.865 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.950191+0000", 2024-09-16T14:48:57.865 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:57.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.950201+0000", 2024-09-16T14:48:57.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3899999999999999e-06 2024-09-16T14:48:57.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.866 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.950210+0000", 2024-09-16T14:48:57.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7680000000000003e-06 2024-09-16T14:48:57.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.866 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.950218+0000", 2024-09-16T14:48:57.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5820000000000004e-06 2024-09-16T14:48:57.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.866 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.867 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.950278+0000", 2024-09-16T14:48:57.867 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.0186999999999998e-05 2024-09-16T14:48:57.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.867 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.867 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.950288+0000", 2024-09-16T14:48:57.867 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0114e-05 2024-09-16T14:48:57.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.867 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.867 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.954777+0000", 2024-09-16T14:48:57.867 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044884349999999998 2024-09-16T14:48:57.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.868 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.868 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.954800+0000", 2024-09-16T14:48:57.868 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2898999999999999e-05 2024-09-16T14:48:57.868 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.868 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.868 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.868 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.868 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s3 38/27 ECSubWrite(tid=36, reqid=client.4500.0:302, at_version=0'0, trim_to=0'0, roll_forward_to=38'25))", 2024-09-16T14:48:57.868 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:44.942275+0000", 2024-09-16T14:48:57.868 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 12.851385704, 2024-09-16T14:48:57.868 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0041456619999999996, 2024-09-16T14:48:57.868 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.868 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.869 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.869 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.869 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942275+0000", 2024-09-16T14:48:57.869 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.869 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.869 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942275+0000", 2024-09-16T14:48:57.869 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.869 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.869 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942274+0000", 2024-09-16T14:48:57.870 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:57.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.870 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.870 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.870 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942285+0000", 2024-09-16T14:48:57.870 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0778000000000001e-05 2024-09-16T14:48:57.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.870 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.870 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.870 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942298+0000", 2024-09-16T14:48:57.870 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2733e-05 2024-09-16T14:48:57.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.870 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.871 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.871 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942306+0000", 2024-09-16T14:48:57.871 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7789999999999996e-06 2024-09-16T14:48:57.871 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.871 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.871 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.871 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942354+0000", 2024-09-16T14:48:57.871 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7982999999999998e-05 2024-09-16T14:48:57.871 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.871 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.871 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.871 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942365+0000", 2024-09-16T14:48:57.871 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1286e-05 2024-09-16T14:48:57.871 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.872 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.872 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.872 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.946368+0000", 2024-09-16T14:48:57.872 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0040030459999999997 2024-09-16T14:48:57.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.872 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.872 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.872 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.946421+0000", 2024-09-16T14:48:57.872 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2707999999999997e-05 2024-09-16T14:48:57.872 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.872 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.872 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.873 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.873 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds1 38/27 ECSubWrite(tid=191, reqid=client.4500.0:309, at_version=38'21, trim_to=0'0, roll_forward_to=37'20))", 2024-09-16T14:48:57.873 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:44.944050+0000", 2024-09-16T14:48:57.873 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 12.849610833, 2024-09-16T14:48:57.873 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034154580000000001, 2024-09-16T14:48:57.873 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.873 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.873 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.873 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.873 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.873 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.944050+0000", 2024-09-16T14:48:57.873 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.874 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.874 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.944050+0000", 2024-09-16T14:48:57.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.874 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.874 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.944049+0000", 2024-09-16T14:48:57.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:57.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.874 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.874 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.944143+0000", 2024-09-16T14:48:57.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3424000000000005e-05 2024-09-16T14:48:57.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.875 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.875 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.875 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.944158+0000", 2024-09-16T14:48:57.875 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.539e-05 2024-09-16T14:48:57.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.875 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.875 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.875 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.944167+0000", 2024-09-16T14:48:57.875 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4060000000000005e-06 2024-09-16T14:48:57.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.875 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.875 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.944230+0000", 2024-09-16T14:48:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3498999999999996e-05 2024-09-16T14:48:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.944240+0000", 2024-09-16T14:48:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0039e-05 2024-09-16T14:48:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.947435+0000", 2024-09-16T14:48:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031947859999999998 2024-09-16T14:48:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.877 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.877 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.947465+0000", 2024-09-16T14:48:57.877 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0355999999999999e-05 2024-09-16T14:48:57.877 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.877 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.877 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.877 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.877 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.877 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s3 42/27 ECSubWrite(tid=39, reqid=client.4500.0:365, at_version=42'28, trim_to=0'0, roll_forward_to=38'26))", 2024-09-16T14:48:57.877 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:48.953739+0000", 2024-09-16T14:48:57.877 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 8.8399219900000006, 2024-09-16T14:48:57.877 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0030143829999999998, 2024-09-16T14:48:57.877 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.878 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.878 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.878 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.878 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.878 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:48.953739+0000", 2024-09-16T14:48:57.878 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.878 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.878 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.878 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.878 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:48.953739+0000", 2024-09-16T14:48:57.878 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.878 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.878 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.879 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.879 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:48.953739+0000", 2024-09-16T14:48:57.879 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:48:57.879 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.879 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.879 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:48.955143+0000", 2024-09-16T14:48:57.879 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001404701 2024-09-16T14:48:57.879 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.879 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.879 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:48.955156+0000", 2024-09-16T14:48:57.879 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2567e-05 2024-09-16T14:48:57.880 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.880 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.880 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.880 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:48.955163+0000", 2024-09-16T14:48:57.880 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7059999999999998e-06 2024-09-16T14:48:57.880 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.880 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.880 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.880 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:48.955253+0000", 2024-09-16T14:48:57.880 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0628e-05 2024-09-16T14:48:57.880 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.880 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.880 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.880 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:48.955262+0000", 2024-09-16T14:48:57.881 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3109999999999995e-06 2024-09-16T14:48:57.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.881 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.881 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.881 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:48.956728+0000", 2024-09-16T14:48:57.881 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0014655880000000001 2024-09-16T14:48:57.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.881 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.881 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.881 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:48.956753+0000", 2024-09-16T14:48:57.881 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5100999999999999e-05 2024-09-16T14:48:57.881 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.881 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 43/27 ECSubWrite(tid=133, reqid=osd.11.0:133, at_version=43'42, trim_to=0'0, roll_forward_to=42'40))", 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:49.853829+0000", 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 7.939831957, 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0040899839999999996, 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.853829+0000", 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.882 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.883 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.853829+0000", 2024-09-16T14:48:57.883 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.883 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.883 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.853828+0000", 2024-09-16T14:48:57.883 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:57.883 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.883 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.853844+0000", 2024-09-16T14:48:57.884 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5336999999999999e-05 2024-09-16T14:48:57.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.884 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.884 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.884 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.853856+0000", 2024-09-16T14:48:57.884 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1823e-05 2024-09-16T14:48:57.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.884 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.884 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.884 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.853863+0000", 2024-09-16T14:48:57.884 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8739999999999995e-06 2024-09-16T14:48:57.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.884 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.853911+0000", 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7314000000000001e-05 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.853921+0000", 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0515e-05 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.857898+0000", 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0039770710000000004 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.857919+0000", 2024-09-16T14:48:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0428000000000002e-05 2024-09-16T14:48:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 43/27 ECSubWrite(tid=134, reqid=osd.11.0:133, at_version=0'0, trim_to=0'0, roll_forward_to=43'42))", 2024-09-16T14:48:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:49.862044+0000", 2024-09-16T14:48:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 7.9316168019999997, 2024-09-16T14:48:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003739619, 2024-09-16T14:48:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.862044+0000", 2024-09-16T14:48:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.862044+0000", 2024-09-16T14:48:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.862043+0000", 2024-09-16T14:48:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.862052+0000", 2024-09-16T14:48:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5590000000000006e-06 2024-09-16T14:48:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.862063+0000", 2024-09-16T14:48:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0917999999999999e-05 2024-09-16T14:48:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.862073+0000", 2024-09-16T14:48:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5959999999999993e-06 2024-09-16T14:48:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.862119+0000", 2024-09-16T14:48:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6762999999999999e-05 2024-09-16T14:48:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.862129+0000", 2024-09-16T14:48:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7259999999999997e-06 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.865760+0000", 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036305790000000001 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:49.865784+0000", 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3977000000000001e-05 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds1 47/27 ECSubWrite(tid=207, reqid=client.4500.0:390, at_version=47'27, trim_to=0'0, roll_forward_to=38'23))", 2024-09-16T14:48:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:53.867472+0000", 2024-09-16T14:48:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.9261884390000001, 2024-09-16T14:48:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003098544, 2024-09-16T14:48:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.867472+0000", 2024-09-16T14:48:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.867472+0000", 2024-09-16T14:48:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.867472+0000", 2024-09-16T14:48:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.867487+0000", 2024-09-16T14:48:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5549e-05 2024-09-16T14:48:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.867499+0000", 2024-09-16T14:48:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2196e-05 2024-09-16T14:48:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.867505+0000", 2024-09-16T14:48:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.7790000000000001e-06 2024-09-16T14:48:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.867562+0000", 2024-09-16T14:48:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.6839000000000002e-05 2024-09-16T14:48:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.867573+0000", 2024-09-16T14:48:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1270000000000001e-05 2024-09-16T14:48:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.870546+0000", 2024-09-16T14:48:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0029724500000000002 2024-09-16T14:48:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.870571+0000", 2024-09-16T14:48:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5111000000000001e-05 2024-09-16T14:48:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:57.895 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:57.895 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:48:57.897 INFO:tasks.rados.rados.0.smithi040.stdout:183: done (3 left) 2024-09-16T14:48:57.897 INFO:tasks.rados.rados.0.smithi040.stdout:184: read oid 12 snap -1 2024-09-16T14:48:57.897 INFO:tasks.rados.rados.0.smithi040.stdout:184: expect (ObjNum 11 snap 0 seq_num 11) 2024-09-16T14:48:57.898 INFO:tasks.rados.rados.0.smithi040.stdout:180: finishing write tid 1 to smithi04040151-40 2024-09-16T14:48:57.898 INFO:tasks.rados.rados.0.smithi040.stdout:181: finishing rollback tid 0 to smithi04040151-43 2024-09-16T14:48:57.898 INFO:tasks.rados.rados.0.smithi040.stdout:181: finishing rollback tid 1 to smithi04040151-43 2024-09-16T14:48:57.898 INFO:tasks.rados.rados.0.smithi040.stdout:185: read oid 35 snap -1 2024-09-16T14:48:57.898 INFO:tasks.rados.rados.0.smithi040.stdout:185: expect (ObjNum 65 snap 4 seq_num 65) 2024-09-16T14:48:57.898 INFO:tasks.rados.rados.0.smithi040.stdout:186: read oid 24 snap -1 2024-09-16T14:48:57.898 INFO:tasks.rados.rados.0.smithi040.stdout:186: expect deleted 2024-09-16T14:48:57.898 INFO:tasks.rados.rados.0.smithi040.stdout:187: delete oid 45 current snap is 13 2024-09-16T14:48:57.898 INFO:tasks.rados.rados.0.smithi040.stdout:180: finishing write tid 2 to smithi04040151-40 2024-09-16T14:48:57.898 INFO:tasks.rados.rados.0.smithi040.stdout:180: finishing write tid 3 to smithi04040151-40 2024-09-16T14:48:57.898 INFO:tasks.rados.rados.0.smithi040.stdout:180: finishing write tid 4 to smithi04040151-40 2024-09-16T14:48:57.898 INFO:tasks.rados.rados.0.smithi040.stdout:182: expect (ObjNum 54 snap 1 seq_num 54) 2024-09-16T14:48:57.899 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:48:57.899 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:57.895+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:48:57.909 INFO:teuthology.orchestra.run.smithi040.stdout:moved 0 / 146 (0%) 2024-09-16T14:48:57.909 INFO:teuthology.orchestra.run.smithi040.stdout:avg 13.2727 2024-09-16T14:48:57.909 INFO:teuthology.orchestra.run.smithi040.stdout:stddev 1.71044 -> 1.71044 (expected baseline 3.47363) 2024-09-16T14:48:57.909 INFO:teuthology.orchestra.run.smithi040.stdout:min osd.2 with 10 -> 10 pgs (0.753425 -> 0.753425 * mean) 2024-09-16T14:48:57.909 INFO:teuthology.orchestra.run.smithi040.stdout:max osd.1 with 16 -> 16 pgs (1.20548 -> 1.20548 * mean) 2024-09-16T14:48:57.909 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:48:57.909 INFO:teuthology.orchestra.run.smithi040.stdout:oload 1000 2024-09-16T14:48:57.909 INFO:teuthology.orchestra.run.smithi040.stdout:max_change 0.05 2024-09-16T14:48:57.909 INFO:teuthology.orchestra.run.smithi040.stdout:max_change_osds 4 2024-09-16T14:48:57.909 INFO:teuthology.orchestra.run.smithi040.stdout:average_utilization 0.0004 2024-09-16T14:48:57.909 INFO:teuthology.orchestra.run.smithi040.stdout:overload_utilization 0.0037 2024-09-16T14:48:57.910 INFO:teuthology.orchestra.run.smithi040.stderr:no change 2024-09-16T14:48:57.920 INFO:tasks.rados.rados.0.smithi040.stdout:181: finishing rollback tid 2 to smithi04040151-43 2024-09-16T14:48:57.921 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 43 v 50 (ObjNum 62 snap 3 seq_num 62) dirty exists 2024-09-16T14:48:57.921 INFO:tasks.rados.rados.0.smithi040.stdout:180: finishing write tid 5 to smithi04040151-40 2024-09-16T14:48:57.921 INFO:tasks.rados.rados.0.smithi040.stdout:180: oid 40 updating version 0 to 46 2024-09-16T14:48:57.921 INFO:tasks.rados.rados.0.smithi040.stdout:180: oid 40 version 46 is already newer than 43 2024-09-16T14:48:57.921 INFO:tasks.rados.rados.0.smithi040.stdout:180: oid 40 version 46 is already newer than 45 2024-09-16T14:48:57.921 INFO:tasks.rados.rados.0.smithi040.stdout:180: oid 40 version 46 is already newer than 44 2024-09-16T14:48:57.921 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 40 v 46 (ObjNum 82 snap 13 seq_num 82) dirty exists 2024-09-16T14:48:57.921 INFO:tasks.rados.rados.0.smithi040.stdout:180: left oid 40 (ObjNum 82 snap 13 seq_num 82) 2024-09-16T14:48:57.921 INFO:tasks.rados.rados.0.smithi040.stdout:180: done (6 left) 2024-09-16T14:48:57.921 INFO:tasks.rados.rados.0.smithi040.stdout:181: done (5 left) 2024-09-16T14:48:57.921 INFO:tasks.rados.rados.0.smithi040.stdout:182: done (4 left) 2024-09-16T14:48:57.921 INFO:tasks.rados.rados.0.smithi040.stdout:187: done (3 left) 2024-09-16T14:48:57.921 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 48 current snap is 13 2024-09-16T14:48:57.922 INFO:tasks.rados.rados.0.smithi040.stdout:188: seq_num 84 ranges {0=655360,655360=638976,1294336=131072} 2024-09-16T14:48:57.922 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T14:48:57.922 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-16T14:48:57.924 INFO:tasks.rados.rados.0.smithi040.stdout:188: writing smithi04040151-48 from 0 to 655360 tid 1 2024-09-16T14:48:57.928 INFO:tasks.rados.rados.0.smithi040.stdout:188: writing smithi04040151-48 from 655360 to 1294336 tid 2 2024-09-16T14:48:57.929 INFO:tasks.rados.rados.0.smithi040.stdout:188: writing smithi04040151-48 from 1294336 to 1425408 tid 3 2024-09-16T14:48:57.929 INFO:tasks.rados.rados.0.smithi040.stdout:186: done (3 left) 2024-09-16T14:48:57.929 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 30 current snap is 13 2024-09-16T14:48:57.929 INFO:tasks.rados.rados.0.smithi040.stdout:189: seq_num 85 ranges {0=671744,671744=770048} 2024-09-16T14:48:57.934 INFO:tasks.rados.rados.0.smithi040.stdout:189: writing smithi04040151-30 from 0 to 671744 tid 1 2024-09-16T14:48:57.939 INFO:tasks.rados.rados.0.smithi040.stdout:189: writing smithi04040151-30 from 671744 to 1441792 tid 2 2024-09-16T14:48:57.939 INFO:tasks.rados.rados.0.smithi040.stdout:190: setattr oid 19 current snap is 13 2024-09-16T14:48:57.939 INFO:tasks.rados.rados.0.smithi040.stdout:184: expect (ObjNum 11 snap 0 seq_num 11) 2024-09-16T14:48:57.946 INFO:tasks.rados.rados.0.smithi040.stdout:185: expect (ObjNum 65 snap 4 seq_num 65) 2024-09-16T14:48:57.994 INFO:tasks.rados.rados.0.smithi040.stdout:188: finishing write tid 1 to smithi04040151-48 2024-09-16T14:48:57.994 INFO:tasks.rados.rados.0.smithi040.stdout:188: finishing write tid 2 to smithi04040151-48 2024-09-16T14:48:57.994 INFO:tasks.rados.rados.0.smithi040.stdout:188: finishing write tid 3 to smithi04040151-48 2024-09-16T14:48:57.994 INFO:tasks.rados.rados.0.smithi040.stdout:188: finishing write tid 4 to smithi04040151-48 2024-09-16T14:48:57.995 INFO:tasks.rados.rados.0.smithi040.stdout:189: finishing write tid 1 to smithi04040151-30 2024-09-16T14:48:57.995 INFO:tasks.rados.rados.0.smithi040.stdout:188: finishing write tid 5 to smithi04040151-48 2024-09-16T14:48:57.995 INFO:tasks.rados.rados.0.smithi040.stdout:188: oid 48 updating version 0 to 22 2024-09-16T14:48:57.995 INFO:tasks.rados.rados.0.smithi040.stdout:188: oid 48 version 22 is already newer than 20 2024-09-16T14:48:57.995 INFO:tasks.rados.rados.0.smithi040.stdout:188: oid 48 updating version 22 to 23 2024-09-16T14:48:57.995 INFO:tasks.rados.rados.0.smithi040.stdout:188: oid 48 version 23 is already newer than 21 2024-09-16T14:48:57.995 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 48 v 23 (ObjNum 84 snap 13 seq_num 84) dirty exists 2024-09-16T14:48:57.995 INFO:tasks.rados.rados.0.smithi040.stdout:188: left oid 48 (ObjNum 84 snap 13 seq_num 84) 2024-09-16T14:48:57.995 INFO:tasks.rados.rados.0.smithi040.stdout:189: finishing write tid 2 to smithi04040151-30 2024-09-16T14:48:57.995 INFO:tasks.rados.rados.0.smithi040.stdout:189: finishing write tid 3 to smithi04040151-30 2024-09-16T14:48:57.995 INFO:tasks.rados.rados.0.smithi040.stdout:189: finishing write tid 4 to smithi04040151-30 2024-09-16T14:48:57.995 INFO:tasks.rados.rados.0.smithi040.stdout:189: oid 30 updating version 0 to 51 2024-09-16T14:48:57.995 INFO:tasks.rados.rados.0.smithi040.stdout:189: oid 30 version 51 is already newer than 49 2024-09-16T14:48:57.996 INFO:tasks.rados.rados.0.smithi040.stdout:189: oid 30 version 51 is already newer than 50 2024-09-16T14:48:57.996 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 30 v 51 (ObjNum 85 snap 13 seq_num 85) dirty exists 2024-09-16T14:48:57.996 INFO:tasks.rados.rados.0.smithi040.stdout:189: left oid 30 (ObjNum 85 snap 13 seq_num 85) 2024-09-16T14:48:57.996 INFO:tasks.rados.rados.0.smithi040.stdout:184: done (4 left) 2024-09-16T14:48:57.996 INFO:tasks.rados.rados.0.smithi040.stdout:185: done (3 left) 2024-09-16T14:48:57.996 INFO:tasks.rados.rados.0.smithi040.stdout:188: done (2 left) 2024-09-16T14:48:57.996 INFO:tasks.rados.rados.0.smithi040.stdout:189: done (1 left) 2024-09-16T14:48:57.996 INFO:tasks.rados.rados.0.smithi040.stdout:191: rollback oid 11 current snap is 13 2024-09-16T14:48:57.996 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 11 to 8 2024-09-16T14:48:57.996 INFO:tasks.rados.rados.0.smithi040.stdout:192: rollback oid 8 current snap is 13 2024-09-16T14:48:57.996 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 8 to 8 2024-09-16T14:48:57.996 INFO:tasks.rados.rados.0.smithi040.stdout:193: setattr oid 31 current snap is 13 2024-09-16T14:48:57.997 INFO:tasks.rados.rados.0.smithi040.stdout:194: rollback oid 39 current snap is 13 2024-09-16T14:48:57.997 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 39 to 11 2024-09-16T14:48:57.997 INFO:tasks.rados.rados.0.smithi040.stdout:195: read oid 40 snap -1 2024-09-16T14:48:57.997 INFO:tasks.rados.rados.0.smithi040.stdout:195: expect (ObjNum 82 snap 13 seq_num 82) 2024-09-16T14:48:57.997 INFO:tasks.rados.rados.0.smithi040.stdout:196: read oid 23 snap -1 2024-09-16T14:48:57.997 INFO:tasks.rados.rados.0.smithi040.stdout:196: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-16T14:48:57.998 INFO:tasks.rados.rados.0.smithi040.stdout:197: snap_remove snap 5 2024-09-16T14:48:58.000 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:57.998+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:58.046 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:58.046 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:48:58.046 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:48:58.046 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:58.055 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:48:58.100 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:58.099+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:58.156 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:58.156 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:48:58.156 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:48:58.156 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:48:58.157 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:58.164 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:48:58.199 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:58.198+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:58.300 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:58.299+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:48:58.310 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:48:58.310 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:48:58.310 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:48:58.310 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:48:58.310 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.310 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:48:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 36.168380204999998, 2024-09-16T14:48:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.223181782, 2024-09-16T14:48:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 1 2024-09-16T14:48:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.311 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:48:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:48:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.312 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139495+0000", 2024-09-16T14:48:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139504+0000", 2024-09-16T14:48:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6119999999999999e-06 2024-09-16T14:48:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139514+0000", 2024-09-16T14:48:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:48:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.313 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139533+0000", 2024-09-16T14:48:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9573e-05 2024-09-16T14:48:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139585+0000", 2024-09-16T14:48:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2170999999999999e-05 2024-09-16T14:48:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:48:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139594+0000", 2024-09-16T14:48:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.884e-06 2024-09-16T14:48:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.314 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359233+0000", 2024-09-16T14:48:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.21963890899999999 2024-09-16T14:48:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359437+0000", 2024-09-16T14:48:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00020374999999999999 2024-09-16T14:48:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.315 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359611+0000", 2024-09-16T14:48:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017445899999999999 2024-09-16T14:48:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.360506+0000", 2024-09-16T14:48:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00089459299999999995 2024-09-16T14:48:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362594+0000", 2024-09-16T14:48:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002087776 2024-09-16T14:48:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.316 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362636+0000", 2024-09-16T14:48:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1933e-05 2024-09-16T14:48:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362679+0000", 2024-09-16T14:48:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2877999999999998e-05 2024-09-16T14:48:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.317 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:48:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 35.919668798000004, 2024-09-16T14:48:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077691490000000004, 2024-09-16T14:48:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 21 2024-09-16T14:48:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.318 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.319 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.319 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:48:58.319 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.319 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.319 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.319 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.319 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:48:58.319 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.319 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.319 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.319 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.319 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:48:58.319 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.320 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.320 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.320 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.320 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388278+0000", 2024-09-16T14:48:58.320 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0809000000000006e-05 2024-09-16T14:48:58.320 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.320 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.320 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.320 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388286+0000", 2024-09-16T14:48:58.320 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.204e-06 2024-09-16T14:48:58.320 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.320 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.320 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.321 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388291+0000", 2024-09-16T14:48:58.321 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9349999999999999e-06 2024-09-16T14:48:58.321 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.321 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.321 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.321 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388713+0000", 2024-09-16T14:48:58.321 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00042189899999999999 2024-09-16T14:48:58.321 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.321 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.321 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.321 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388766+0000", 2024-09-16T14:48:58.321 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2750999999999997e-05 2024-09-16T14:48:58.321 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.321 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.322 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.322 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388853+0000", 2024-09-16T14:48:58.322 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6635e-05 2024-09-16T14:48:58.322 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.322 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.322 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.322 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.390635+0000", 2024-09-16T14:48:58.322 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0017817950000000001 2024-09-16T14:48:58.322 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.322 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.322 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.322 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.395898+0000", 2024-09-16T14:48:58.322 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0052630510000000004 2024-09-16T14:48:58.322 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.323 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.323 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.323 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.395941+0000", 2024-09-16T14:48:58.323 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3248999999999999e-05 2024-09-16T14:48:58.323 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.323 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.323 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.323 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.395977+0000", 2024-09-16T14:48:58.323 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6501000000000002e-05 2024-09-16T14:48:58.323 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.323 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.323 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.323 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.323 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.324 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:58.324 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:48:58.324 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 35.919577789999998, 2024-09-16T14:48:58.324 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077998870000000001, 2024-09-16T14:48:58.324 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.324 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.324 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.324 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.324 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.324 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 22 2024-09-16T14:48:58.324 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.325 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.325 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.325 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.325 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:48:58.325 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.325 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.325 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.325 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.325 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:48:58.325 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.325 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.326 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.326 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.326 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:48:58.326 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.326 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.326 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.326 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.326 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388311+0000", 2024-09-16T14:48:58.326 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2853e-05 2024-09-16T14:48:58.326 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.326 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.327 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.327 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388318+0000", 2024-09-16T14:48:58.327 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7549999999999997e-06 2024-09-16T14:48:58.327 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.327 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.327 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.327 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388322+0000", 2024-09-16T14:48:58.327 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5889999999999999e-06 2024-09-16T14:48:58.327 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.327 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.327 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.327 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.389160+0000", 2024-09-16T14:48:58.327 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000837877 2024-09-16T14:48:58.327 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.328 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.328 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.328 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.389230+0000", 2024-09-16T14:48:58.328 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0551999999999998e-05 2024-09-16T14:48:58.328 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.328 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.328 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.328 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.389371+0000", 2024-09-16T14:48:58.328 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014102099999999999 2024-09-16T14:48:58.328 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.328 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.328 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.328 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.390881+0000", 2024-09-16T14:48:58.329 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00151004 2024-09-16T14:48:58.329 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.329 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.329 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.329 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.396022+0000", 2024-09-16T14:48:58.329 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005140716 2024-09-16T14:48:58.329 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.329 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.329 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.329 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.396074+0000", 2024-09-16T14:48:58.329 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2089999999999998e-05 2024-09-16T14:48:58.329 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.329 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.330 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.330 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.396099+0000", 2024-09-16T14:48:58.330 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5038e-05 2024-09-16T14:48:58.330 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.330 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.330 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.330 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.330 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.330 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:58.330 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:48:58.330 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 35.901190913999997, 2024-09-16T14:48:58.330 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0072029349999999997, 2024-09-16T14:48:58.330 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.330 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.331 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.331 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.331 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.331 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 36 2024-09-16T14:48:58.331 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.331 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.331 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.331 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.331 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:48:58.331 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.331 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.331 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.331 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.331 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:48:58.332 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.332 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.332 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.332 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.332 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:48:58.332 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.332 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.332 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.332 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.332 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406695+0000", 2024-09-16T14:48:58.332 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1530000000000003e-06 2024-09-16T14:48:58.332 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.332 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.333 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.333 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406700+0000", 2024-09-16T14:48:58.333 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7079999999999996e-06 2024-09-16T14:48:58.333 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.333 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.333 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.333 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406702+0000", 2024-09-16T14:48:58.333 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7410000000000001e-06 2024-09-16T14:48:58.333 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.333 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.333 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.333 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409398+0000", 2024-09-16T14:48:58.333 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0026957420000000001 2024-09-16T14:48:58.333 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.334 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.334 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.334 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409441+0000", 2024-09-16T14:48:58.334 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3071999999999997e-05 2024-09-16T14:48:58.334 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.334 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.334 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.334 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409512+0000", 2024-09-16T14:48:58.334 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0871999999999995e-05 2024-09-16T14:48:58.334 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.334 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.334 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.334 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.410156+0000", 2024-09-16T14:48:58.334 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00064395800000000003 2024-09-16T14:48:58.335 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.335 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.335 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.335 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413850+0000", 2024-09-16T14:48:58.335 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036938959999999999 2024-09-16T14:48:58.335 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.335 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.335 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.335 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413872+0000", 2024-09-16T14:48:58.335 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2195e-05 2024-09-16T14:48:58.335 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.335 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.335 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.335 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413889+0000", 2024-09-16T14:48:58.336 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6948000000000001e-05 2024-09-16T14:48:58.336 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.336 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.336 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.336 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.336 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.336 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:58.336 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.406706+0000", 2024-09-16T14:48:58.336 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 35.901171335999997, 2024-09-16T14:48:58.336 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0072520049999999997, 2024-09-16T14:48:58.336 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.336 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.336 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.337 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.337 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.337 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 37 2024-09-16T14:48:58.337 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.337 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.337 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.337 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.337 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406706+0000", 2024-09-16T14:48:58.337 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.337 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.337 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.337 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.337 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406706+0000", 2024-09-16T14:48:58.337 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.338 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.338 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.338 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.338 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406705+0000", 2024-09-16T14:48:58.338 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.338 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.338 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.338 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.338 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406708+0000", 2024-09-16T14:48:58.339 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.57e-06 2024-09-16T14:48:58.339 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.339 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.339 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.339 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406712+0000", 2024-09-16T14:48:58.339 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.425e-06 2024-09-16T14:48:58.339 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.339 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.339 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.339 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406715+0000", 2024-09-16T14:48:58.339 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7700000000000002e-06 2024-09-16T14:48:58.340 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.340 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.340 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.340 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409721+0000", 2024-09-16T14:48:58.340 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003006327 2024-09-16T14:48:58.340 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.340 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.340 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.340 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409770+0000", 2024-09-16T14:48:58.340 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8022000000000002e-05 2024-09-16T14:48:58.340 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.340 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.340 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.341 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409853+0000", 2024-09-16T14:48:58.341 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3604999999999997e-05 2024-09-16T14:48:58.341 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.341 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.341 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.341 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.410391+0000", 2024-09-16T14:48:58.341 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00053785400000000004 2024-09-16T14:48:58.341 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.341 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.341 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.341 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413911+0000", 2024-09-16T14:48:58.341 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0035205240000000001 2024-09-16T14:48:58.341 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.341 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.342 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.342 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413939+0000", 2024-09-16T14:48:58.342 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7032000000000001e-05 2024-09-16T14:48:58.342 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.342 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.342 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.342 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413958+0000", 2024-09-16T14:48:58.342 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9147000000000002e-05 2024-09-16T14:48:58.342 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.342 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.342 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.342 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.342 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.342 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:58.343 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.620638+0000", 2024-09-16T14:48:58.343 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 35.687239233, 2024-09-16T14:48:58.343 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0078265469999999997, 2024-09-16T14:48:58.343 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.343 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.343 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.343 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.343 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.343 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 116 2024-09-16T14:48:58.343 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.343 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.343 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.343 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.343 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620638+0000", 2024-09-16T14:48:58.344 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.344 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.344 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.344 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.344 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620638+0000", 2024-09-16T14:48:58.344 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.344 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.344 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.344 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.345 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620637+0000", 2024-09-16T14:48:58.345 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.345 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.345 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.345 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.345 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620641+0000", 2024-09-16T14:48:58.345 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9360000000000001e-06 2024-09-16T14:48:58.345 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.345 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.345 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.345 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620647+0000", 2024-09-16T14:48:58.345 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3339999999999998e-06 2024-09-16T14:48:58.345 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.346 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.346 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.346 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620652+0000", 2024-09-16T14:48:58.346 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0950000000000003e-06 2024-09-16T14:48:58.346 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.346 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.346 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.346 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625724+0000", 2024-09-16T14:48:58.346 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050719370000000003 2024-09-16T14:48:58.346 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.346 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.346 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.346 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625768+0000", 2024-09-16T14:48:58.347 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4039000000000003e-05 2024-09-16T14:48:58.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.347 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.347 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.347 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625836+0000", 2024-09-16T14:48:58.347 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7181999999999995e-05 2024-09-16T14:48:58.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.347 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.347 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.347 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627424+0000", 2024-09-16T14:48:58.347 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015887119999999999 2024-09-16T14:48:58.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.347 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.348 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.348 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628421+0000", 2024-09-16T14:48:58.348 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00099648699999999989 2024-09-16T14:48:58.348 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.348 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.348 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.348 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628446+0000", 2024-09-16T14:48:58.348 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5307999999999999e-05 2024-09-16T14:48:58.348 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.348 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.348 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.348 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628464+0000", 2024-09-16T14:48:58.348 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8170999999999999e-05 2024-09-16T14:48:58.348 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.349 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.349 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.349 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.349 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.349 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:48:58.349 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:48:58.349 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 35.687219509000002, 2024-09-16T14:48:58.349 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0079223479999999992, 2024-09-16T14:48:58.349 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.349 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.349 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.349 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.349 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.349 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 117 2024-09-16T14:48:58.350 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.350 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.350 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.350 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.350 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:48:58.350 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.350 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.350 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.350 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.350 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:48:58.350 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.350 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.350 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.350 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.351 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:48:58.351 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.351 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.351 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.351 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.351 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620662+0000", 2024-09-16T14:48:58.351 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7609999999999998e-06 2024-09-16T14:48:58.351 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.351 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.351 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.351 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620668+0000", 2024-09-16T14:48:58.351 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.84e-06 2024-09-16T14:48:58.351 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.351 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.352 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.352 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620673+0000", 2024-09-16T14:48:58.352 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.468e-06 2024-09-16T14:48:58.352 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.352 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.352 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.352 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.626022+0000", 2024-09-16T14:48:58.352 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053484680000000003 2024-09-16T14:48:58.352 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.352 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.352 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.352 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.626071+0000", 2024-09-16T14:48:58.352 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9854000000000001e-05 2024-09-16T14:48:58.353 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.353 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.353 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.353 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.626145+0000", 2024-09-16T14:48:58.353 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.3159999999999997e-05 2024-09-16T14:48:58.353 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.353 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.353 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.353 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627426+0000", 2024-09-16T14:48:58.353 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001281847 2024-09-16T14:48:58.353 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.353 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.353 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.354 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628501+0000", 2024-09-16T14:48:58.354 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0010748019999999999 2024-09-16T14:48:58.354 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.354 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.354 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.354 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628541+0000", 2024-09-16T14:48:58.354 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9598999999999998e-05 2024-09-16T14:48:58.354 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.354 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.354 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.354 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628580+0000", 2024-09-16T14:48:58.354 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8977999999999998e-05 2024-09-16T14:48:58.354 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.354 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.355 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.355 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.355 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.355 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:58.355 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:48:58.355 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 34.915952615999998, 2024-09-16T14:48:58.355 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0081240189999999997, 2024-09-16T14:48:58.355 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.355 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.355 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.355 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.355 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.356 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 158 2024-09-16T14:48:58.356 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.356 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.356 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.356 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.356 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:48:58.356 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.356 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.356 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.356 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.356 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:48:58.356 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.356 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.357 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.357 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.357 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:48:58.357 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.357 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.357 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.357 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.357 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392007+0000", 2024-09-16T14:48:58.357 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3108000000000005e-05 2024-09-16T14:48:58.357 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.357 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.357 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.357 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392014+0000", 2024-09-16T14:48:58.358 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6170000000000003e-06 2024-09-16T14:48:58.358 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.358 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.358 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.358 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392017+0000", 2024-09-16T14:48:58.358 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3589999999999999e-06 2024-09-16T14:48:58.358 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.358 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.358 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.358 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394165+0000", 2024-09-16T14:48:58.358 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021481270000000001 2024-09-16T14:48:58.358 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.358 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.359 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.359 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394238+0000", 2024-09-16T14:48:58.359 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2366999999999993e-05 2024-09-16T14:48:58.359 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.359 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.359 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.359 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394347+0000", 2024-09-16T14:48:58.359 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000109907 2024-09-16T14:48:58.359 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.359 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.359 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.359 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396066+0000", 2024-09-16T14:48:58.360 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001718731 2024-09-16T14:48:58.360 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.360 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.360 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.360 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.399984+0000", 2024-09-16T14:48:58.360 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0039181850000000002 2024-09-16T14:48:58.360 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.360 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.360 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.360 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400027+0000", 2024-09-16T14:48:58.360 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2184000000000001e-05 2024-09-16T14:48:58.360 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.360 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.361 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.361 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400048+0000", 2024-09-16T14:48:58.361 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1897999999999999e-05 2024-09-16T14:48:58.361 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.361 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.361 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.361 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.361 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.361 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:58.361 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:48:58.361 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 34.915855821999997, 2024-09-16T14:48:58.361 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0081462830000000007, 2024-09-16T14:48:58.361 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.362 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.362 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.362 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.362 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.362 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 159 2024-09-16T14:48:58.362 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.362 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.362 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.362 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.362 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:48:58.362 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.362 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.362 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.362 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.363 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:48:58.363 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.363 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.363 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.363 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.363 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:48:58.363 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.363 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.363 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.363 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.363 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392023+0000", 2024-09-16T14:48:58.363 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:48:58.363 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.363 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392035+0000", 2024-09-16T14:48:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2207e-05 2024-09-16T14:48:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392040+0000", 2024-09-16T14:48:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7450000000000003e-06 2024-09-16T14:48:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394756+0000", 2024-09-16T14:48:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0027154010000000001 2024-09-16T14:48:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.365 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.365 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.365 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394799+0000", 2024-09-16T14:48:58.365 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3921999999999997e-05 2024-09-16T14:48:58.365 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.365 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.365 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.365 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394867+0000", 2024-09-16T14:48:58.365 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7770999999999999e-05 2024-09-16T14:48:58.365 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.365 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.365 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.365 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396922+0000", 2024-09-16T14:48:58.365 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0020549819999999999 2024-09-16T14:48:58.366 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.366 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.366 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.366 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400094+0000", 2024-09-16T14:48:58.366 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031720979999999999 2024-09-16T14:48:58.366 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.366 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.366 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.366 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400134+0000", 2024-09-16T14:48:58.366 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9449999999999997e-05 2024-09-16T14:48:58.366 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.366 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.366 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.366 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400167+0000", 2024-09-16T14:48:58.367 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3711000000000003e-05 2024-09-16T14:48:58.367 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.367 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.367 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.367 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.367 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.367 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:160 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:58.367 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:48:58.367 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 34.915833589000002, 2024-09-16T14:48:58.367 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082045659999999999, 2024-09-16T14:48:58.367 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.367 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.367 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.367 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.368 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.368 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 160 2024-09-16T14:48:58.368 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.368 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.368 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.368 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.368 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:48:58.368 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.368 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.368 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.368 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.368 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:48:58.368 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.369 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.369 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.369 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.369 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:48:58.369 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.369 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.369 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.369 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.369 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392045+0000", 2024-09-16T14:48:58.369 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.283e-06 2024-09-16T14:48:58.369 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.369 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.369 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.369 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392050+0000", 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1099999999999996e-06 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392052+0000", 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4660000000000002e-06 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395234+0000", 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031823260000000001 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.370 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395283+0000", 2024-09-16T14:48:58.371 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8347000000000003e-05 2024-09-16T14:48:58.371 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.371 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.371 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.371 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395364+0000", 2024-09-16T14:48:58.371 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0853999999999995e-05 2024-09-16T14:48:58.371 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.371 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.371 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.371 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396928+0000", 2024-09-16T14:48:58.371 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015642239999999999 2024-09-16T14:48:58.372 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.372 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.372 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.372 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400202+0000", 2024-09-16T14:48:58.372 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032747359999999999 2024-09-16T14:48:58.372 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.372 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.372 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.372 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400227+0000", 2024-09-16T14:48:58.372 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4079000000000001e-05 2024-09-16T14:48:58.372 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.372 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.372 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.373 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400248+0000", 2024-09-16T14:48:58.373 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1444e-05 2024-09-16T14:48:58.373 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.373 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.373 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.373 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.373 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.373 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:161 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:58.373 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:48:58.373 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 34.915822153999997, 2024-09-16T14:48:58.373 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082675720000000008, 2024-09-16T14:48:58.373 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.373 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.373 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.374 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.374 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.374 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 161 2024-09-16T14:48:58.374 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.374 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.374 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.374 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.374 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:48:58.374 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.374 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.374 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.374 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.374 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:48:58.374 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.375 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.375 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.375 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.375 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:48:58.375 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:48:58.375 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.375 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.375 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.375 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392122+0000", 2024-09-16T14:48:58.375 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7698000000000001e-05 2024-09-16T14:48:58.375 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.375 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.375 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.375 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392136+0000", 2024-09-16T14:48:58.376 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3464e-05 2024-09-16T14:48:58.376 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.376 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.376 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.376 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392140+0000", 2024-09-16T14:48:58.376 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0790000000000002e-06 2024-09-16T14:48:58.376 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.376 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.376 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.376 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395579+0000", 2024-09-16T14:48:58.376 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034393219999999999 2024-09-16T14:48:58.377 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.377 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.377 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.377 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395624+0000", 2024-09-16T14:48:58.377 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4946e-05 2024-09-16T14:48:58.377 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.377 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.377 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.377 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395692+0000", 2024-09-16T14:48:58.378 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7843999999999996e-05 2024-09-16T14:48:58.378 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.378 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.378 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.378 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397834+0000", 2024-09-16T14:48:58.378 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021421529999999999 2024-09-16T14:48:58.378 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.378 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.378 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.378 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400281+0000", 2024-09-16T14:48:58.378 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002447238 2024-09-16T14:48:58.378 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.378 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.379 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.379 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400305+0000", 2024-09-16T14:48:58.379 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3428999999999999e-05 2024-09-16T14:48:58.379 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.379 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.379 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.379 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400322+0000", 2024-09-16T14:48:58.379 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7632e-05 2024-09-16T14:48:58.379 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.379 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.379 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.379 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.379 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.380 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:58.380 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:48:58.380 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 34.915732875000003, 2024-09-16T14:48:58.380 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.008260814, 2024-09-16T14:48:58.380 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.380 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.380 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.380 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.380 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.380 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 162 2024-09-16T14:48:58.380 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.380 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.381 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.381 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.381 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:48:58.381 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.381 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.381 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.381 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.381 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:48:58.381 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.381 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.381 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.381 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.382 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:48:58.382 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.382 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.382 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.382 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.382 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392146+0000", 2024-09-16T14:48:58.382 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.526e-06 2024-09-16T14:48:58.382 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.382 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.382 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.382 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392151+0000", 2024-09-16T14:48:58.382 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2980000000000002e-06 2024-09-16T14:48:58.382 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.383 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.383 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.383 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392153+0000", 2024-09-16T14:48:58.383 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5409999999999999e-06 2024-09-16T14:48:58.383 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.383 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.383 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.383 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396103+0000", 2024-09-16T14:48:58.383 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0039497559999999996 2024-09-16T14:48:58.383 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.383 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.383 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.384 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396188+0000", 2024-09-16T14:48:58.384 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5038999999999997e-05 2024-09-16T14:48:58.384 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.384 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.384 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.384 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396315+0000", 2024-09-16T14:48:58.384 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000126895 2024-09-16T14:48:58.384 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.384 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.384 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.384 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397839+0000", 2024-09-16T14:48:58.384 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015246859999999999 2024-09-16T14:48:58.385 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.385 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.385 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.385 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400356+0000", 2024-09-16T14:48:58.385 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002516524 2024-09-16T14:48:58.385 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.385 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.385 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.385 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400379+0000", 2024-09-16T14:48:58.385 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2793e-05 2024-09-16T14:48:58.385 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.385 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.385 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.386 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400405+0000", 2024-09-16T14:48:58.386 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6140999999999999e-05 2024-09-16T14:48:58.386 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.386 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.386 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.386 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.386 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.386 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:58.386 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:48:58.386 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 34.915721484000002, 2024-09-16T14:48:58.386 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083183530000000006, 2024-09-16T14:48:58.387 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.387 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.387 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.387 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.387 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.387 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 163 2024-09-16T14:48:58.387 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.387 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.387 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.387 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.387 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:48:58.387 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.388 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.388 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.388 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.388 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:48:58.388 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.388 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.388 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.388 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.388 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392155+0000", 2024-09-16T14:48:58.388 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:48:58.388 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.388 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.389 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.389 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392159+0000", 2024-09-16T14:48:58.389 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0459999999999999e-06 2024-09-16T14:48:58.389 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.389 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.389 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.389 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392162+0000", 2024-09-16T14:48:58.389 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:48:58.389 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.390 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.390 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.390 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392165+0000", 2024-09-16T14:48:58.390 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.424e-06 2024-09-16T14:48:58.390 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.390 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.390 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.390 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396578+0000", 2024-09-16T14:48:58.390 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044136139999999997 2024-09-16T14:48:58.390 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.390 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.391 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.391 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396625+0000", 2024-09-16T14:48:58.391 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6486000000000002e-05 2024-09-16T14:48:58.391 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.391 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.391 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.391 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396694+0000", 2024-09-16T14:48:58.391 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9049999999999998e-05 2024-09-16T14:48:58.391 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.391 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.391 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.392 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397843+0000", 2024-09-16T14:48:58.392 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011485740000000001 2024-09-16T14:48:58.392 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.392 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.392 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.392 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400438+0000", 2024-09-16T14:48:58.392 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025950909999999999 2024-09-16T14:48:58.392 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.392 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.392 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.392 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400458+0000", 2024-09-16T14:48:58.393 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0248e-05 2024-09-16T14:48:58.393 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.393 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.393 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.393 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400474+0000", 2024-09-16T14:48:58.393 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5999999999999999e-05 2024-09-16T14:48:58.393 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.393 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.393 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.393 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.393 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.394 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:58.394 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:48:58.394 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 34.915709886999998, 2024-09-16T14:48:58.394 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083881519999999994, 2024-09-16T14:48:58.394 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.394 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.394 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.394 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.394 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.394 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 164 2024-09-16T14:48:58.394 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.394 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.394 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.395 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.395 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:48:58.395 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.395 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.395 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.395 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.395 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:48:58.395 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.395 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.395 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.395 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.395 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:48:58.396 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:48:58.396 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.396 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.396 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.396 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392212+0000", 2024-09-16T14:48:58.396 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5108999999999998e-05 2024-09-16T14:48:58.396 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.396 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.396 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.396 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392216+0000", 2024-09-16T14:48:58.396 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2549999999999999e-06 2024-09-16T14:48:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392220+0000", 2024-09-16T14:48:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3850000000000002e-06 2024-09-16T14:48:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396961+0000", 2024-09-16T14:48:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.004740976 2024-09-16T14:48:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397029+0000", 2024-09-16T14:48:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7889e-05 2024-09-16T14:48:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397131+0000", 2024-09-16T14:48:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000102315 2024-09-16T14:48:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400259+0000", 2024-09-16T14:48:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031285229999999998 2024-09-16T14:48:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400501+0000", 2024-09-16T14:48:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00024211999999999999 2024-09-16T14:48:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400530+0000", 2024-09-16T14:48:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8935999999999999e-05 2024-09-16T14:48:58.400 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.400 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.400 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.400 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400555+0000", 2024-09-16T14:48:58.400 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4861000000000001e-05 2024-09-16T14:48:58.400 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.400 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.400 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.400 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.400 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.400 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:48:58.401 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:48:58.401 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 34.915654121999999, 2024-09-16T14:48:58.401 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010593673, 2024-09-16T14:48:58.401 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.401 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.401 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.401 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:48:58.401 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:48:58.401 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 165 2024-09-16T14:48:58.401 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.401 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.401 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.402 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.402 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:48:58.402 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.402 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.402 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.402 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.402 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:48:58.402 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.402 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.402 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.402 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.402 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:48:58.403 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.403 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.403 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.403 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.403 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392226+0000", 2024-09-16T14:48:58.403 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2789999999999999e-06 2024-09-16T14:48:58.403 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.403 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.403 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.403 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392234+0000", 2024-09-16T14:48:58.403 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2369999999999992e-06 2024-09-16T14:48:58.403 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.404 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.404 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.404 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392237+0000", 2024-09-16T14:48:58.404 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8619999999999999e-06 2024-09-16T14:48:58.404 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.404 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.404 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.404 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397503+0000", 2024-09-16T14:48:58.404 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0052656220000000002 2024-09-16T14:48:58.404 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.404 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.405 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.405 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397558+0000", 2024-09-16T14:48:58.405 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5386999999999997e-05 2024-09-16T14:48:58.405 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.405 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.405 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:48:58.405 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397628+0000", 2024-09-16T14:48:58.405 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0119999999999999e-05 2024-09-16T14:48:58.405 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.405 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.405 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:48:58.406 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402533+0000", 2024-09-16T14:48:58.406 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049046200000000002 2024-09-16T14:48:58.406 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.406 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.406 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:48:58.406 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402778+0000", 2024-09-16T14:48:58.406 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000245348 2024-09-16T14:48:58.406 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.406 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.406 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.406 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402800+0000", 2024-09-16T14:48:58.406 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2277e-05 2024-09-16T14:48:58.407 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.407 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.407 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.407 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402817+0000", 2024-09-16T14:48:58.407 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6265000000000001e-05 2024-09-16T14:48:58.407 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.407 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.407 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.407 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.407 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.407 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:309 3.ds0 3:b9c0d236:::smithi04040151-19:head [append 0~737280] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:48:58.408 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:48:58.408 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.368462986000001, 2024-09-16T14:48:58.408 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01003656, 2024-09-16T14:48:58.408 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.408 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.408 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.408 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:48:58.408 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:58.409 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 309 2024-09-16T14:48:58.409 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.409 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.409 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.409 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.409 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:48:58.409 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.409 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.409 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.409 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.409 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:48:58.409 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.410 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.410 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.410 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.410 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939413+0000", 2024-09-16T14:48:58.410 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:58.410 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.410 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.410 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.410 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942226+0000", 2024-09-16T14:48:58.410 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0028130450000000001 2024-09-16T14:48:58.410 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.410 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.411 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.411 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942238+0000", 2024-09-16T14:48:58.411 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2639e-05 2024-09-16T14:48:58.411 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.411 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.411 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.411 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942245+0000", 2024-09-16T14:48:58.411 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3840000000000002e-06 2024-09-16T14:48:58.411 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.411 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.411 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.411 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942288+0000", 2024-09-16T14:48:58.412 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3186000000000003e-05 2024-09-16T14:48:58.412 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.412 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.412 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.412 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942368+0000", 2024-09-16T14:48:58.412 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0162999999999998e-05 2024-09-16T14:48:58.412 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.412 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.412 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:58.412 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943660+0000", 2024-09-16T14:48:58.413 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001291698 2024-09-16T14:48:58.413 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.413 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.413 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:58.413 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.947440+0000", 2024-09-16T14:48:58.413 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003780079 2024-09-16T14:48:58.413 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.413 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.413 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.413 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.949391+0000", 2024-09-16T14:48:58.413 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0019512990000000001 2024-09-16T14:48:58.414 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.414 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.414 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.414 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.949451+0000", 2024-09-16T14:48:58.414 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9404000000000002e-05 2024-09-16T14:48:58.414 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.414 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.414 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.414 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.414 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.414 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:312 3.ds0 3:b9c0d236:::smithi04040151-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:48:58.415 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:48:58.415 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.364483471, 2024-09-16T14:48:58.415 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084967020000000001, 2024-09-16T14:48:58.415 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.415 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:48:58.415 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.415 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:48:58.415 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:58.415 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 312 2024-09-16T14:48:58.415 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.415 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.415 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.416 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.416 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:48:58.416 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.416 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.416 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.416 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.416 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:48:58.416 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.416 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.416 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.416 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.417 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943393+0000", 2024-09-16T14:48:58.417 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.417 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.417 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.417 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.417 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943396+0000", 2024-09-16T14:48:58.417 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6079999999999998e-06 2024-09-16T14:48:58.417 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.417 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.417 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.417 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943402+0000", 2024-09-16T14:48:58.417 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6100000000000002e-06 2024-09-16T14:48:58.418 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.418 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.418 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.418 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943413+0000", 2024-09-16T14:48:58.418 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1178e-05 2024-09-16T14:48:58.418 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.418 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.418 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.418 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.946818+0000", 2024-09-16T14:48:58.418 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034043049999999998 2024-09-16T14:48:58.418 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.418 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.419 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:48:58.419 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.946922+0000", 2024-09-16T14:48:58.419 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000104076 2024-09-16T14:48:58.419 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.419 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.419 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.419 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.950275+0000", 2024-09-16T14:48:58.419 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0033530220000000002 2024-09-16T14:48:58.419 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.419 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.419 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.419 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.950357+0000", 2024-09-16T14:48:58.420 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2087000000000002e-05 2024-09-16T14:48:58.420 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.420 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.420 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.420 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.951890+0000", 2024-09-16T14:48:58.420 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001533333 2024-09-16T14:48:58.420 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.420 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.420 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.420 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.420 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.421 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:390 3.ds0 3:b855c7ac:::smithi04040151-47:head [append 0~0] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:48:58.421 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:53.864520+0000", 2024-09-16T14:48:58.421 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 4.4433572750000003, 2024-09-16T14:48:58.421 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089883559999999994, 2024-09-16T14:48:58.421 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.421 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.421 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.421 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:48:58.421 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:58.421 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 390 2024-09-16T14:48:58.421 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.422 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.422 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.422 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.422 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864520+0000", 2024-09-16T14:48:58.422 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.422 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.422 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.422 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.422 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864520+0000", 2024-09-16T14:48:58.422 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.422 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.422 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.423 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.423 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864519+0000", 2024-09-16T14:48:58.423 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.423 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.423 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.423 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:48:58.423 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0545e-05 2024-09-16T14:48:58.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.423 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.423 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.423 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864536+0000", 2024-09-16T14:48:58.424 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8069999999999996e-06 2024-09-16T14:48:58.424 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.424 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.424 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.424 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864541+0000", 2024-09-16T14:48:58.424 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0900000000000004e-06 2024-09-16T14:48:58.424 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.424 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.424 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.424 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.866852+0000", 2024-09-16T14:48:58.424 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0023115399999999999 2024-09-16T14:48:58.424 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.425 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.425 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.425 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.866946+0000", 2024-09-16T14:48:58.425 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3311000000000002e-05 2024-09-16T14:48:58.425 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.425 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.425 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:58.425 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.867413+0000", 2024-09-16T14:48:58.425 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00046786499999999998 2024-09-16T14:48:58.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.426 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.426 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:58.426 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.868091+0000", 2024-09-16T14:48:58.426 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00067759500000000002 2024-09-16T14:48:58.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.426 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.426 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.426 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.873468+0000", 2024-09-16T14:48:58.426 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053766930000000001 2024-09-16T14:48:58.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.427 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.427 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.427 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.873508+0000", 2024-09-16T14:48:58.427 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0314000000000001e-05 2024-09-16T14:48:58.427 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.427 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.427 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.427 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.427 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.427 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:459 3.ds0 3:b9c0d236:::smithi04040151-19:head [create,setxattr bk]W^gTaniZ\\TV]dqj_UT\\QgY_\\pYYUOcYdSe`m_ (1859) in=40b,setxattr XSPcap`qmQ\\ZXYl^_bQYNltYZfOtfbRjqmVk_mOa (419) in=40b,setxattr l_Qd^aXhnapg`O][bWecsVdjlWkdYc^tOnjZh]^m (1577) in=40b,setxattr facegtqpaQroppsVbsrZelUak]RbamP`srOTdms[ (350) in=40b,setxattr QllprTts[liem]moe]Pi`nNtcQko (1402) in=28b,setxattr sjm^Un[ahPriW\\TNkaYS\\NnPZfZ\\Zmq[rQ^TWYZm (1892) in=40b,setxattr kRdb__OpRdYV^sYmYmYdhes^Vbf]QaV`SVZkhRcY (209) in=40b,setxattr b[QTnoQc[_Yc^RmdY[ (1402) in=18b,setxattr ^iZkhfc`oO (1402) in=10b,setxattr d\\QUOkoScOhonabPS_YeNlheOOeOal]aQ^ZW]lc` (2805) in=40b,setxattr SioYcliigTgheX\\jnVqipcelfYhbZRjQRet]mQQU (1365) in=40b,setxattr jWcgaRWbr\\ZQ_XXr_eenPa]sXtbaPlmeshkgjOsm (1192) in=40b,setxattr WmVtOY (1402) in=6b,setxattr ]tk_t\\gnSfccZfYfnWnaqP (1402) in=22b,setxattr dS_dYomnsYPtQarmqoZQkO]Ofk_SfgjgitphTZPg (18524) in=40b,setxattr g_a^Zpt`YVahZiOW]V[jQ\\QccqSTo_ma[\\c\\[aft (454) in=40b,setxattr qmmdmgOR__hN\\UTjpR_gbY^s_pOVSdbTob`k (1402) in=36b,setxattr cW`PPOYrbfXV^hmaWhVNb`qVSXqjcdej`pY][]l^ (665) in=40b,setxattr jjo^VgcaQffrnVNPlnq_iTmcrlnTOenhaZV[_SZf (453) in=40b,setxattr n^nqj]bnVcl\\njtUkprb\\laSh^Rbmam_Z_NkeXqj (595) in=40b] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:48:58.427 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.993572+0000", 2024-09-16T14:48:58.427 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.314305269, 2024-09-16T14:48:58.428 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010337796999999999, 2024-09-16T14:48:58.428 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.428 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.428 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:48:58.428 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:48:58.428 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:58.428 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 459 2024-09-16T14:48:58.428 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.428 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.428 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.428 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.429 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993572+0000", 2024-09-16T14:48:58.429 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.429 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.429 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.429 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.429 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993572+0000", 2024-09-16T14:48:58.429 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.429 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.429 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.429 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.429 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993569+0000", 2024-09-16T14:48:58.430 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999976 2024-09-16T14:48:58.430 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.430 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.430 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.430 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993625+0000", 2024-09-16T14:48:58.430 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5105000000000002e-05 2024-09-16T14:48:58.430 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.430 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.430 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.430 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993647+0000", 2024-09-16T14:48:58.430 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1962000000000001e-05 2024-09-16T14:48:58.430 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.431 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.431 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.431 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993656+0000", 2024-09-16T14:48:58.431 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3179999999999996e-06 2024-09-16T14:48:58.431 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.431 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.431 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.431 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993717+0000", 2024-09-16T14:48:58.431 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.0738e-05 2024-09-16T14:48:58.431 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.431 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.432 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:48:58.432 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993841+0000", 2024-09-16T14:48:58.432 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000124537 2024-09-16T14:48:58.432 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.432 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.432 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:58.432 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.994891+0000", 2024-09-16T14:48:58.432 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0010499719999999999 2024-09-16T14:48:58.432 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.432 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.432 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:58.432 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.999326+0000", 2024-09-16T14:48:58.433 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044351240000000004 2024-09-16T14:48:58.433 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.433 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.433 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:48:58.433 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003847+0000", 2024-09-16T14:48:58.433 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045207629999999997 2024-09-16T14:48:58.433 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.433 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.433 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.433 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003910+0000", 2024-09-16T14:48:58.433 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2527999999999995e-05 2024-09-16T14:48:58.433 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.434 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.434 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.434 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.434 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.434 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.5s4 51/27 ECSubWrite(tid=60, reqid=client.4500.0:467, at_version=51'19, trim_to=0'0, roll_forward_to=51'16))", 2024-09-16T14:48:58.434 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.997576+0000", 2024-09-16T14:48:58.434 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.31030114399999997, 2024-09-16T14:48:58.434 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083578019999999992, 2024-09-16T14:48:58.434 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:48:58.434 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:48:58.434 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:48:58.434 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.435 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:48:58.435 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997576+0000", 2024-09-16T14:48:58.435 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.435 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.435 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.435 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:48:58.435 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997576+0000", 2024-09-16T14:48:58.435 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:48:58.435 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.435 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.435 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:48:58.435 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997575+0000", 2024-09-16T14:48:58.436 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.436 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.436 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.436 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:48:58.436 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997593+0000", 2024-09-16T14:48:58.436 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7645000000000001e-05 2024-09-16T14:48:58.436 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.436 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.436 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:48:58.436 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997608+0000", 2024-09-16T14:48:58.436 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5315000000000002e-05 2024-09-16T14:48:58.436 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.437 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.437 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.437 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997616+0000", 2024-09-16T14:48:58.437 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.6120000000000001e-06 2024-09-16T14:48:58.437 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.437 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.437 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:48:58.437 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997651+0000", 2024-09-16T14:48:58.437 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5256999999999998e-05 2024-09-16T14:48:58.437 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.437 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.437 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:48:58.438 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997660+0000", 2024-09-16T14:48:58.438 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7320000000000004e-06 2024-09-16T14:48:58.438 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.438 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.438 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:48:58.438 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.005913+0000", 2024-09-16T14:48:58.438 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082530840000000008 2024-09-16T14:48:58.438 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:48:58.438 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:48:58.438 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:48:58.438 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.005934+0000", 2024-09-16T14:48:58.438 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0812999999999999e-05 2024-09-16T14:48:58.439 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.439 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.439 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.439 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:48:58.439 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:48:58.439 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:48:58.441 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:48:58.441 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:58.439+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:58.541 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:48:58.540+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:48:58.555 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:58.555 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:58.555 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:58.564 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:48:58.642 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:58.641+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:58.707 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:58.707 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:58.707 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:58.707 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:58.707 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:58.715 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:48:58.742 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:58.742+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:58.819 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:295 3.9s0 3:96d2b183:::smithi04040151-29:head [copy-from ver 0] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T14:48:58.819 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:48:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.901228477, 2024-09-16T14:48:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010216596999999999, 2024-09-16T14:48:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 295 2024-09-16T14:48:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:48:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:48:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915430+0000", 2024-09-16T14:48:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999976 2024-09-16T14:48:58.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915441+0000", 2024-09-16T14:48:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0789e-05 2024-09-16T14:48:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915462+0000", 2024-09-16T14:48:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0845000000000001e-05 2024-09-16T14:48:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915473+0000", 2024-09-16T14:48:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0805000000000001e-05 2024-09-16T14:48:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915528+0000", 2024-09-16T14:48:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4772999999999999e-05 2024-09-16T14:48:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:48:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915537+0000", 2024-09-16T14:48:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6700000000000006e-06 2024-09-16T14:48:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.919813+0000", 2024-09-16T14:48:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042757840000000004 2024-09-16T14:48:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.919907+0000", 2024-09-16T14:48:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504999999999993e-05 2024-09-16T14:48:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925649+0000", 2024-09-16T14:48:58.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057427870000000001 2024-09-16T14:48:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:296 3.9s0 3:96d2b183:::smithi04040151-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-16T14:48:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:48:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.90117656, 2024-09-16T14:48:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010423623, 2024-09-16T14:48:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:58.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 296 2024-09-16T14:48:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:48:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:48:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915484+0000", 2024-09-16T14:48:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915489+0000", 2024-09-16T14:48:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6079999999999998e-06 2024-09-16T14:48:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915496+0000", 2024-09-16T14:48:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7589999999999997e-06 2024-09-16T14:48:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915506+0000", 2024-09-16T14:48:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0340000000000002e-06 2024-09-16T14:48:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915588+0000", 2024-09-16T14:48:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2198000000000001e-05 2024-09-16T14:48:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:48:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915594+0000", 2024-09-16T14:48:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:48:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920099+0000", 2024-09-16T14:48:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045048700000000002 2024-09-16T14:48:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:48:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920147+0000", 2024-09-16T14:48:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8239999999999999e-05 2024-09-16T14:48:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925726+0000", 2024-09-16T14:48:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055787529999999997 2024-09-16T14:48:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925816+0000", 2024-09-16T14:48:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0185000000000005e-05 2024-09-16T14:48:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925908+0000", 2024-09-16T14:48:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1948999999999997e-05 2024-09-16T14:48:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s6 37/27 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=37'32, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921399+0000", 2024-09-16T14:48:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.895261947, 2024-09-16T14:48:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071090340000000002, 2024-09-16T14:48:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921399+0000", 2024-09-16T14:48:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921399+0000", 2024-09-16T14:48:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921399+0000", 2024-09-16T14:48:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921420+0000", 2024-09-16T14:48:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1094000000000002e-05 2024-09-16T14:48:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921428+0000", 2024-09-16T14:48:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3450000000000006e-06 2024-09-16T14:48:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921436+0000", 2024-09-16T14:48:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5789999999999998e-06 2024-09-16T14:48:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921525+0000", 2024-09-16T14:48:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8943999999999994e-05 2024-09-16T14:48:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921535+0000", 2024-09-16T14:48:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0183000000000001e-05 2024-09-16T14:48:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928483+0000", 2024-09-16T14:48:58.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069473800000000004 2024-09-16T14:48:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928508+0000", 2024-09-16T14:48:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5856000000000001e-05 2024-09-16T14:48:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs3 37/27 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921846+0000", 2024-09-16T14:48:58.838 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.894815385999999, 2024-09-16T14:48:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071221770000000004, 2024-09-16T14:48:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921846+0000", 2024-09-16T14:48:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921846+0000", 2024-09-16T14:48:58.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921845+0000", 2024-09-16T14:48:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921862+0000", 2024-09-16T14:48:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6807999999999999e-05 2024-09-16T14:48:58.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921871+0000", 2024-09-16T14:48:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6810000000000004e-06 2024-09-16T14:48:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921878+0000", 2024-09-16T14:48:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.109e-06 2024-09-16T14:48:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921935+0000", 2024-09-16T14:48:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7082e-05 2024-09-16T14:48:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921944+0000", 2024-09-16T14:48:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7609999999999996e-06 2024-09-16T14:48:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928947+0000", 2024-09-16T14:48:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070028870000000002 2024-09-16T14:48:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928968+0000", 2024-09-16T14:48:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1251e-05 2024-09-16T14:48:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.844 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s6 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921894+0000", 2024-09-16T14:48:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.89476705, 2024-09-16T14:48:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0072329309999999997, 2024-09-16T14:48:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921894+0000", 2024-09-16T14:48:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921894+0000", 2024-09-16T14:48:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921894+0000", 2024-09-16T14:48:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921908+0000", 2024-09-16T14:48:58.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3976e-05 2024-09-16T14:48:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921918+0000", 2024-09-16T14:48:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0182e-05 2024-09-16T14:48:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921924+0000", 2024-09-16T14:48:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1419999999999998e-06 2024-09-16T14:48:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922184+0000", 2024-09-16T14:48:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000260315 2024-09-16T14:48:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922192+0000", 2024-09-16T14:48:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7179999999999996e-06 2024-09-16T14:48:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929113+0000", 2024-09-16T14:48:58.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069209529999999997 2024-09-16T14:48:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929127+0000", 2024-09-16T14:48:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3991999999999999e-05 2024-09-16T14:48:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as4 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:48:58.849 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:48:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.892621909000001, 2024-09-16T14:48:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088260140000000001, 2024-09-16T14:48:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:48:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:48:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:48:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924062+0000", 2024-09-16T14:48:58.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2892e-05 2024-09-16T14:48:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924070+0000", 2024-09-16T14:48:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8690000000000004e-06 2024-09-16T14:48:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924076+0000", 2024-09-16T14:48:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0739999999999998e-06 2024-09-16T14:48:58.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924119+0000", 2024-09-16T14:48:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2549999999999997e-05 2024-09-16T14:48:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924126+0000", 2024-09-16T14:48:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8960000000000003e-06 2024-09-16T14:48:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:58.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932835+0000", 2024-09-16T14:48:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087089200000000002 2024-09-16T14:48:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932865+0000", 2024-09-16T14:48:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0054000000000001e-05 2024-09-16T14:48:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es1 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:48:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:48:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.892507321, 2024-09-16T14:48:58.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087127249999999993, 2024-09-16T14:48:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:48:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:48:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:48:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924168+0000", 2024-09-16T14:48:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4800000000000001e-05 2024-09-16T14:48:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924176+0000", 2024-09-16T14:48:58.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.25e-06 2024-09-16T14:48:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924182+0000", 2024-09-16T14:48:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3640000000000004e-06 2024-09-16T14:48:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924229+0000", 2024-09-16T14:48:58.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6477999999999998e-05 2024-09-16T14:48:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924238+0000", 2024-09-16T14:48:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4429999999999992e-06 2024-09-16T14:48:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932848+0000", 2024-09-16T14:48:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086097910000000003 2024-09-16T14:48:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932867+0000", 2024-09-16T14:48:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8927999999999999e-05 2024-09-16T14:48:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.8s7 37/27 ECSubWrite(tid=108, reqid=osd.11.0:108, at_version=37'33, trim_to=0'0, roll_forward_to=35'31))", 2024-09-16T14:48:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.940492+0000", 2024-09-16T14:48:58.862 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.876169283999999, 2024-09-16T14:48:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061198069999999997, 2024-09-16T14:48:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940492+0000", 2024-09-16T14:48:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940492+0000", 2024-09-16T14:48:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940492+0000", 2024-09-16T14:48:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940508+0000", 2024-09-16T14:48:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6767999999999999e-05 2024-09-16T14:48:58.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940520+0000", 2024-09-16T14:48:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2078e-05 2024-09-16T14:48:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940527+0000", 2024-09-16T14:48:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8630000000000002e-06 2024-09-16T14:48:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940914+0000", 2024-09-16T14:48:58.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00038634200000000002 2024-09-16T14:48:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940922+0000", 2024-09-16T14:48:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7490000000000004e-06 2024-09-16T14:48:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.946588+0000", 2024-09-16T14:48:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0056654230000000002 2024-09-16T14:48:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.946612+0000", 2024-09-16T14:48:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4008000000000001e-05 2024-09-16T14:48:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.8s7 37/27 ECSubWrite(tid=110, reqid=osd.11.0:110, at_version=37'35, trim_to=0'0, roll_forward_to=35'31))", 2024-09-16T14:48:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.941394+0000", 2024-09-16T14:48:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.875267187, 2024-09-16T14:48:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0075660129999999999, 2024-09-16T14:48:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941394+0000", 2024-09-16T14:48:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941394+0000", 2024-09-16T14:48:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941393+0000", 2024-09-16T14:48:58.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941415+0000", 2024-09-16T14:48:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1162000000000002e-05 2024-09-16T14:48:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941426+0000", 2024-09-16T14:48:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1319e-05 2024-09-16T14:48:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941435+0000", 2024-09-16T14:48:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8769999999999999e-06 2024-09-16T14:48:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941762+0000", 2024-09-16T14:48:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032733399999999998 2024-09-16T14:48:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941772+0000", 2024-09-16T14:48:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9799999999999993e-06 2024-09-16T14:48:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.948937+0000", 2024-09-16T14:48:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071645550000000004 2024-09-16T14:48:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.948960+0000", 2024-09-16T14:48:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3529000000000001e-05 2024-09-16T14:48:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.871 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:394 3.9s0 3:93498a0b:::smithi04040151-32:head [append 0~491520] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:48:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:48:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.9470888400000002, 2024-09-16T14:48:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016461614999999999, 2024-09-16T14:48:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 394 2024-09-16T14:48:58.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:48:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:48:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869570+0000", 2024-09-16T14:48:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:48:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872427+0000", 2024-09-16T14:48:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028569070000000001 2024-09-16T14:48:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872450+0000", 2024-09-16T14:48:58.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2943e-05 2024-09-16T14:48:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872461+0000", 2024-09-16T14:48:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0882e-05 2024-09-16T14:48:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872521+0000", 2024-09-16T14:48:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9639000000000003e-05 2024-09-16T14:48:58.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:48:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872530+0000", 2024-09-16T14:48:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8810000000000002e-06 2024-09-16T14:48:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.878309+0000", 2024-09-16T14:48:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057788520000000001 2024-09-16T14:48:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:58.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.878434+0000", 2024-09-16T14:48:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012550899999999999 2024-09-16T14:48:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880126+0000", 2024-09-16T14:48:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016919019999999999 2024-09-16T14:48:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885887+0000", 2024-09-16T14:48:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057608959999999997 2024-09-16T14:48:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885957+0000", 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0504000000000001e-05 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886034+0000", 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6661000000000003e-05 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:395 3.9s0 3:93498a0b:::smithi04040151-32:head [append 0~770048] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:48:58.878 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:48:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.9423614249999996, 2024-09-16T14:48:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.018911853999999999, 2024-09-16T14:48:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 395 2024-09-16T14:48:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:48:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:48:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874298+0000", 2024-09-16T14:48:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:48:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875671+0000", 2024-09-16T14:48:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0013727069999999999 2024-09-16T14:48:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875689+0000", 2024-09-16T14:48:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7832000000000001e-05 2024-09-16T14:48:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875700+0000", 2024-09-16T14:48:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1267000000000001e-05 2024-09-16T14:48:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875758+0000", 2024-09-16T14:48:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8060999999999999e-05 2024-09-16T14:48:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:48:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875766+0000", 2024-09-16T14:48:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9580000000000007e-06 2024-09-16T14:48:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881004+0000", 2024-09-16T14:48:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052375169999999997 2024-09-16T14:48:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881098+0000", 2024-09-16T14:48:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3875999999999994e-05 2024-09-16T14:48:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882500+0000", 2024-09-16T14:48:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014019600000000001 2024-09-16T14:48:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886041+0000", 2024-09-16T14:48:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035415210000000002 2024-09-16T14:48:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893136+0000", 2024-09-16T14:48:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070949330000000003 2024-09-16T14:48:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893212+0000", 2024-09-16T14:48:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5739000000000001e-05 2024-09-16T14:48:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:396 3.9s0 3:93498a0b:::smithi04040151-32:head [setxattr _header (57) in=7b] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:48:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:48:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.9409442700000001, 2024-09-16T14:48:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017781597999999999, 2024-09-16T14:48:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:48:58.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 396 2024-09-16T14:48:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:48:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:48:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875715+0000", 2024-09-16T14:48:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999976 2024-09-16T14:48:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875721+0000", 2024-09-16T14:48:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6989999999999997e-06 2024-09-16T14:48:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875728+0000", 2024-09-16T14:48:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9040000000000003e-06 2024-09-16T14:48:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875734+0000", 2024-09-16T14:48:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4519999999999996e-06 2024-09-16T14:48:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875800+0000", 2024-09-16T14:48:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6538000000000006e-05 2024-09-16T14:48:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:48:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875804+0000", 2024-09-16T14:48:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.771e-06 2024-09-16T14:48:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883232+0000", 2024-09-16T14:48:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074282009999999997 2024-09-16T14:48:58.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883323+0000", 2024-09-16T14:48:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0740000000000002e-05 2024-09-16T14:48:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884057+0000", 2024-09-16T14:48:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00073422299999999999 2024-09-16T14:48:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893020+0000", 2024-09-16T14:48:58.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089623210000000005 2024-09-16T14:48:58.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:48:58.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893403+0000", 2024-09-16T14:48:58.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00038375799999999999 2024-09-16T14:48:58.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893499+0000", 2024-09-16T14:48:58.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5284000000000005e-05 2024-09-16T14:48:58.892 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.892 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.892 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.893 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:397 3.9s0 3:93498a0b:::smithi04040151-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:48:58.893 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:48:58.893 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.9409131449999997, 2024-09-16T14:48:58.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019838090999999999, 2024-09-16T14:48:58.893 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.893 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:58.893 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:58.893 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:58.893 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:58.893 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 397 2024-09-16T14:48:58.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.893 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:48:58.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:48:58.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:48:58.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875750+0000", 2024-09-16T14:48:58.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3939999999999999e-06 2024-09-16T14:48:58.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875755+0000", 2024-09-16T14:48:58.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.31e-06 2024-09-16T14:48:58.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875760+0000", 2024-09-16T14:48:58.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4769999999999997e-06 2024-09-16T14:48:58.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875824+0000", 2024-09-16T14:48:58.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3893000000000006e-05 2024-09-16T14:48:58.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:48:58.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875829+0000", 2024-09-16T14:48:58.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3489999999999999e-06 2024-09-16T14:48:58.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885652+0000", 2024-09-16T14:48:58.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0098232530000000005 2024-09-16T14:48:58.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:48:58.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885748+0000", 2024-09-16T14:48:58.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5366999999999997e-05 2024-09-16T14:48:58.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893876+0000", 2024-09-16T14:48:58.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081279479999999994 2024-09-16T14:48:58.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:58.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893967+0000", 2024-09-16T14:48:58.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0777000000000002e-05 2024-09-16T14:48:58.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895586+0000", 2024-09-16T14:48:58.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016197029999999999 2024-09-16T14:48:58.898 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.898 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.898 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.899 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(osd.3.17:4 3.9s0 3:9b721ec3:::smithi04040151-12:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:48:58.899 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:48:58.899 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.9326908510000003, 2024-09-16T14:48:58.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011299052, 2024-09-16T14:48:58.899 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.899 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:58.899 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:48:58.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:48:58.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883969+0000", 2024-09-16T14:48:58.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:48:58.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883976+0000", 2024-09-16T14:48:58.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3730000000000004e-06 2024-09-16T14:48:58.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883999+0000", 2024-09-16T14:48:58.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3390000000000001e-05 2024-09-16T14:48:58.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884007+0000", 2024-09-16T14:48:58.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1489999999999997e-06 2024-09-16T14:48:58.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886141+0000", 2024-09-16T14:48:58.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0021347240000000002 2024-09-16T14:48:58.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:58.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886230+0000", 2024-09-16T14:48:58.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8234999999999998e-05 2024-09-16T14:48:58.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895269+0000", 2024-09-16T14:48:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090399230000000001 2024-09-16T14:48:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 47/27 ECSubWriteReply(tid=27, last_complete=47'18, committed=1, applied=1))", 2024-09-16T14:48:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.886820+0000", 2024-09-16T14:48:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.9298416180000002, 2024-09-16T14:48:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064054309999999996, 2024-09-16T14:48:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886820+0000", 2024-09-16T14:48:58.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886820+0000", 2024-09-16T14:48:58.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886819+0000", 2024-09-16T14:48:58.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:58.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886825+0000", 2024-09-16T14:48:58.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8440000000000003e-06 2024-09-16T14:48:58.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886836+0000", 2024-09-16T14:48:58.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1331e-05 2024-09-16T14:48:58.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886844+0000", 2024-09-16T14:48:58.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.644e-06 2024-09-16T14:48:58.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893058+0000", 2024-09-16T14:48:58.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062146379999999998 2024-09-16T14:48:58.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893225+0000", 2024-09-16T14:48:58.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166868 2024-09-16T14:48:58.907 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.907 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.907 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.907 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpReadReply(3.9s0 47/27 ECSubReadReply(tid=29, attrs_read=0))", 2024-09-16T14:48:58.907 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.886991+0000", 2024-09-16T14:48:58.907 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.9296703669999999, 2024-09-16T14:48:58.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0076394690000000003, 2024-09-16T14:48:58.907 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.907 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:58.907 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886991+0000", 2024-09-16T14:48:58.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886991+0000", 2024-09-16T14:48:58.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886990+0000", 2024-09-16T14:48:58.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:58.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887086+0000", 2024-09-16T14:48:58.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5481000000000003e-05 2024-09-16T14:48:58.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887099+0000", 2024-09-16T14:48:58.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3713999999999999e-05 2024-09-16T14:48:58.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887105+0000", 2024-09-16T14:48:58.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0460000000000002e-06 2024-09-16T14:48:58.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894600+0000", 2024-09-16T14:48:58.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074949450000000003 2024-09-16T14:48:58.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894630+0000", 2024-09-16T14:48:58.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0026e-05 2024-09-16T14:48:58.911 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.911 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.911 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.911 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpReadReply(3.9s0 47/27 ECSubReadReply(tid=29, attrs_read=0))", 2024-09-16T14:48:58.911 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.887586+0000", 2024-09-16T14:48:58.911 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.9290756819999997, 2024-09-16T14:48:58.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069926040000000004, 2024-09-16T14:48:58.911 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.911 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:58.911 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887586+0000", 2024-09-16T14:48:58.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887586+0000", 2024-09-16T14:48:58.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887585+0000", 2024-09-16T14:48:58.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:58.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887722+0000", 2024-09-16T14:48:58.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000137458 2024-09-16T14:48:58.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887736+0000", 2024-09-16T14:48:58.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3586e-05 2024-09-16T14:48:58.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887742+0000", 2024-09-16T14:48:58.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2060000000000004e-06 2024-09-16T14:48:58.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894548+0000", 2024-09-16T14:48:58.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068061019999999996 2024-09-16T14:48:58.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894578+0000", 2024-09-16T14:48:58.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9989e-05 2024-09-16T14:48:58.915 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.915 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.915 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.915 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:440 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:48:58.915 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:48:58.915 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.94063753000000005, 2024-09-16T14:48:58.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01253772, 2024-09-16T14:48:58.915 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.915 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:58.915 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:58.916 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:58.916 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:58.916 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 440 2024-09-16T14:48:58.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.916 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:48:58.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:48:58.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876022+0000", 2024-09-16T14:48:58.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:48:58.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876033+0000", 2024-09-16T14:48:58.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1474000000000001e-05 2024-09-16T14:48:58.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876051+0000", 2024-09-16T14:48:58.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7628000000000001e-05 2024-09-16T14:48:58.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876060+0000", 2024-09-16T14:48:58.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339999999999996e-06 2024-09-16T14:48:58.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876128+0000", 2024-09-16T14:48:58.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7460999999999997e-05 2024-09-16T14:48:58.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:48:58.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876138+0000", 2024-09-16T14:48:58.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8649999999999996e-06 2024-09-16T14:48:58.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882141+0000", 2024-09-16T14:48:58.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060036550000000001 2024-09-16T14:48:58.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:58.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882233+0000", 2024-09-16T14:48:58.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1626000000000007e-05 2024-09-16T14:48:58.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.888561+0000", 2024-09-16T14:48:58.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063284769999999999 2024-09-16T14:48:58.921 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.921 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.921 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.921 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:441 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:48:58.921 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:48:58.921 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.94059047299999998, 2024-09-16T14:48:58.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.013066019999999999, 2024-09-16T14:48:58.921 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.922 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:58.922 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:58.922 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:58.922 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:58.922 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 441 2024-09-16T14:48:58.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.922 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:48:58.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:48:58.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876070+0000", 2024-09-16T14:48:58.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876074+0000", 2024-09-16T14:48:58.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1330000000000001e-06 2024-09-16T14:48:58.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876083+0000", 2024-09-16T14:48:58.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2519999999999999e-06 2024-09-16T14:48:58.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876088+0000", 2024-09-16T14:48:58.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5509999999999999e-06 2024-09-16T14:48:58.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876235+0000", 2024-09-16T14:48:58.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014688900000000001 2024-09-16T14:48:58.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:48:58.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876243+0000", 2024-09-16T14:48:58.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5490000000000001e-06 2024-09-16T14:48:58.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.884127+0000", 2024-09-16T14:48:58.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078848489999999993 2024-09-16T14:48:58.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:58.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.884238+0000", 2024-09-16T14:48:58.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111007 2024-09-16T14:48:58.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889137+0000", 2024-09-16T14:48:58.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048983739999999996 2024-09-16T14:48:58.927 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.927 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.927 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.927 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:442 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:48:58.927 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:48:58.927 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.94055181499999996, 2024-09-16T14:48:58.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014710668, 2024-09-16T14:48:58.927 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:58.928 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:48:58.928 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:48:58.928 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:48:58.928 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:48:58.928 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 442 2024-09-16T14:48:58.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.928 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:58.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:58.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:48:58.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:58.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:48:58.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:58.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:58.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:48:58.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:58.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:58.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876113+0000", 2024-09-16T14:48:58.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0759999999999996e-06 2024-09-16T14:48:58.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876119+0000", 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3339999999999998e-06 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876126+0000", 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8410000000000002e-06 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876273+0000", 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014714900000000001 2024-09-16T14:48:58.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:48:58.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876279+0000", 2024-09-16T14:48:58.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7819999999999999e-06 2024-09-16T14:48:58.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:58.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.885397+0000", 2024-09-16T14:48:58.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091178149999999996 2024-09-16T14:48:58.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:48:58.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.885492+0000", 2024-09-16T14:48:58.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5384999999999996e-05 2024-09-16T14:48:58.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:58.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:58.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:58.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.890820+0000", 2024-09-16T14:48:58.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053278559999999997 2024-09-16T14:48:58.932 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.932 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.932 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.932 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:58.932 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:58.932 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:58.933 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is set 2024-09-16T14:48:58.934 INFO:tasks.rados.rados.0.smithi040.stdout:191: finishing rollback tid 0 to smithi04040151-11 2024-09-16T14:48:58.934 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 19 v 31 (ObjNum 66 snap 4 seq_num 66) dirty exists 2024-09-16T14:48:58.934 INFO:tasks.rados.rados.0.smithi040.stdout:192: finishing rollback tid 0 to smithi04040151-8 2024-09-16T14:48:58.934 INFO:tasks.rados.rados.0.smithi040.stdout:192: finishing rollback tid 1 to smithi04040151-8 2024-09-16T14:48:58.934 INFO:tasks.rados.rados.0.smithi040.stdout:192: finishing rollback tid 2 to smithi04040151-8 2024-09-16T14:48:58.934 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 8 v 24 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-16T14:48:58.934 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 31 v 60 (ObjNum 55 snap 1 seq_num 55) dirty exists 2024-09-16T14:48:58.934 INFO:tasks.rados.rados.0.smithi040.stdout:196: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-16T14:48:58.935 INFO:teuthology.orchestra.run.smithi040.stdout:moved 1 / 146 (0.684932%) 2024-09-16T14:48:58.935 INFO:teuthology.orchestra.run.smithi040.stdout:avg 13.2727 2024-09-16T14:48:58.935 INFO:teuthology.orchestra.run.smithi040.stdout:stddev 1.71044 -> 1.54278 (expected baseline 3.47363) 2024-09-16T14:48:58.935 INFO:teuthology.orchestra.run.smithi040.stdout:min osd.2 with 10 -> 10 pgs (0.753425 -> 0.753425 * mean) 2024-09-16T14:48:58.935 INFO:teuthology.orchestra.run.smithi040.stdout:max osd.1 with 16 -> 16 pgs (1.20548 -> 1.20548 * mean) 2024-09-16T14:48:58.935 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:48:58.935 INFO:teuthology.orchestra.run.smithi040.stdout:oload 110 2024-09-16T14:48:58.935 INFO:teuthology.orchestra.run.smithi040.stdout:max_change 0.05 2024-09-16T14:48:58.936 INFO:teuthology.orchestra.run.smithi040.stdout:max_change_osds 4 2024-09-16T14:48:58.936 INFO:teuthology.orchestra.run.smithi040.stdout:average_utilization 0.0004 2024-09-16T14:48:58.936 INFO:teuthology.orchestra.run.smithi040.stdout:overload_utilization 0.0004 2024-09-16T14:48:58.936 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11 weight 1.0000 -> 0.9500 2024-09-16T14:48:58.936 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 weight 1.0000 -> 0.9500 2024-09-16T14:48:58.937 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:48:58.938 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-16T14:48:58.938 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-16T14:48:58.939 INFO:tasks.rados.rados.0.smithi040.stdout:194: finishing rollback tid 0 to smithi04040151-39 2024-09-16T14:48:58.940 INFO:tasks.rados.rados.0.smithi040.stdout:190: done (7 left) 2024-09-16T14:48:58.940 INFO:tasks.rados.rados.0.smithi040.stdout:192: done (6 left) 2024-09-16T14:48:58.940 INFO:tasks.rados.rados.0.smithi040.stdout:193: done (5 left) 2024-09-16T14:48:58.940 INFO:tasks.rados.rados.0.smithi040.stdout:196: done (4 left) 2024-09-16T14:48:58.940 INFO:tasks.rados.rados.0.smithi040.stdout:197: done (3 left) 2024-09-16T14:48:58.940 INFO:tasks.rados.rados.0.smithi040.stdout:198: read oid 5 snap -1 2024-09-16T14:48:58.940 INFO:tasks.rados.rados.0.smithi040.stdout:198: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-16T14:48:58.940 INFO:tasks.rados.rados.0.smithi040.stdout:199: rmattr oid 19 current snap is 13 2024-09-16T14:48:58.940 INFO:tasks.rados.rados.0.smithi040.stdout:191: finishing rollback tid 1 to smithi04040151-11 2024-09-16T14:48:58.940 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 11 v 54 (ObjNum 273 snap 0 seq_num 0) dirty dne 2024-09-16T14:48:58.941 INFO:tasks.rados.rados.0.smithi040.stdout:194: finishing rollback tid 1 to smithi04040151-39 2024-09-16T14:48:58.941 INFO:tasks.rados.rados.0.smithi040.stdout:194: finishing rollback tid 2 to smithi04040151-39 2024-09-16T14:48:58.941 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 39 v 20 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-16T14:48:58.941 INFO:tasks.rados.rados.0.smithi040.stdout:195: expect (ObjNum 82 snap 13 seq_num 82) 2024-09-16T14:48:58.941 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:48:58.932+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:48:58.959 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 19 v 36 (ObjNum 66 snap 4 seq_num 66) dirty exists 2024-09-16T14:48:58.959 INFO:tasks.rados.rados.0.smithi040.stdout:198: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-16T14:48:58.963 INFO:tasks.rados.rados.0.smithi040.stdout:191: done (4 left) 2024-09-16T14:48:58.963 INFO:tasks.rados.rados.0.smithi040.stdout:194: done (3 left) 2024-09-16T14:48:58.963 INFO:tasks.rados.rados.0.smithi040.stdout:195: done (2 left) 2024-09-16T14:48:58.963 INFO:tasks.rados.rados.0.smithi040.stdout:198: done (1 left) 2024-09-16T14:48:58.963 INFO:tasks.rados.rados.0.smithi040.stdout:199: done (0 left) 2024-09-16T14:48:58.963 INFO:tasks.rados.rados.0.smithi040.stdout:200: rollback oid 30 current snap is 13 2024-09-16T14:48:58.964 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 30 to 12 2024-09-16T14:48:58.964 INFO:tasks.rados.rados.0.smithi040.stdout:201: snap_create 2024-09-16T14:48:58.965 INFO:tasks.rados.rados.0.smithi040.stdout:200: finishing rollback tid 0 to smithi04040151-30 2024-09-16T14:48:58.967 INFO:tasks.rados.rados.0.smithi040.stdout:200: finishing rollback tid 1 to smithi04040151-30 2024-09-16T14:48:58.968 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 30 v 58 (ObjNum 193 snap 0 seq_num 1812545408) dirty dne 2024-09-16T14:48:59.034 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:59.033+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:48:59.041 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:59.041 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:59.041 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:59.041 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:59.050 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:48:59.136 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:48:59.135+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:48:59.153 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:59.153 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:59.153 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:59.161 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:48:59.236 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:48:59.235+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:48:59.247 INFO:teuthology.orchestra.run.smithi040.stdout:moved 0 / 146 (0%) 2024-09-16T14:48:59.248 INFO:teuthology.orchestra.run.smithi040.stdout:avg 13.2727 2024-09-16T14:48:59.248 INFO:teuthology.orchestra.run.smithi040.stdout:stddev 1.54278 -> 1.54278 (expected baseline 3.47363) 2024-09-16T14:48:59.248 INFO:teuthology.orchestra.run.smithi040.stdout:min osd.2 with 10 -> 10 pgs (0.753425 -> 0.753425 * mean) 2024-09-16T14:48:59.248 INFO:teuthology.orchestra.run.smithi040.stdout:max osd.1 with 16 -> 16 pgs (1.20548 -> 1.20548 * mean) 2024-09-16T14:48:59.248 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:48:59.248 INFO:teuthology.orchestra.run.smithi040.stdout:oload 1000 2024-09-16T14:48:59.248 INFO:teuthology.orchestra.run.smithi040.stdout:max_change 0.05 2024-09-16T14:48:59.248 INFO:teuthology.orchestra.run.smithi040.stdout:max_change_osds 4 2024-09-16T14:48:59.248 INFO:teuthology.orchestra.run.smithi040.stdout:average_utilization 0.0004 2024-09-16T14:48:59.248 INFO:teuthology.orchestra.run.smithi040.stdout:overload_utilization 0.0037 2024-09-16T14:48:59.249 INFO:teuthology.orchestra.run.smithi040.stderr:no change 2024-09-16T14:48:59.260 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T14:48:59.260 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-16T14:48:59.266 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:59.266 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:59.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:59.266 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:59.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.266 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=79, reqid=osd.3.0:79, at_version=34'18, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:48:59.266 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:59.266 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.356175757999999, 2024-09-16T14:48:59.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069050049999999997, 2024-09-16T14:48:59.266 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.266 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.266 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:59.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:59.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:48:59.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907863+0000", 2024-09-16T14:48:59.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5972000000000001e-05 2024-09-16T14:48:59.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907876+0000", 2024-09-16T14:48:59.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3230999999999999e-05 2024-09-16T14:48:59.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907900+0000", 2024-09-16T14:48:59.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3716000000000001e-05 2024-09-16T14:48:59.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907956+0000", 2024-09-16T14:48:59.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6307999999999997e-05 2024-09-16T14:48:59.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907965+0000", 2024-09-16T14:48:59.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1530000000000003e-06 2024-09-16T14:48:59.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914715+0000", 2024-09-16T14:48:59.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067495990000000002 2024-09-16T14:48:59.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914742+0000", 2024-09-16T14:48:59.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.728e-05 2024-09-16T14:48:59.270 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.270 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.270 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.270 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=80, reqid=osd.3.0:80, at_version=34'20, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:48:59.271 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:48:59.271 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.354759088000002, 2024-09-16T14:48:59.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065560849999999997, 2024-09-16T14:48:59.271 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.271 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.271 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:48:59.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:48:59.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909253+0000", 2024-09-16T14:48:59.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909278+0000", 2024-09-16T14:48:59.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4905e-05 2024-09-16T14:48:59.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909295+0000", 2024-09-16T14:48:59.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6713000000000001e-05 2024-09-16T14:48:59.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909304+0000", 2024-09-16T14:48:59.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6739999999999992e-06 2024-09-16T14:48:59.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909367+0000", 2024-09-16T14:48:59.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2700999999999995e-05 2024-09-16T14:48:59.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909391+0000", 2024-09-16T14:48:59.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3614000000000001e-05 2024-09-16T14:48:59.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915784+0000", 2024-09-16T14:48:59.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063938479999999997 2024-09-16T14:48:59.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915810+0000", 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5457999999999999e-05 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.354131499000001, 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007542137, 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:59.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:59.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:48:59.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909917+0000", 2024-09-16T14:48:59.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6513000000000001e-05 2024-09-16T14:48:59.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:48:59.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:48:59.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909941+0000", 2024-09-16T14:48:59.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1859999999999992e-06 2024-09-16T14:48:59.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910004+0000", 2024-09-16T14:48:59.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2706999999999994e-05 2024-09-16T14:48:59.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910012+0000", 2024-09-16T14:48:59.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.157e-06 2024-09-16T14:48:59.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917395+0000", 2024-09-16T14:48:59.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073827659999999998 2024-09-16T14:48:59.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917423+0000", 2024-09-16T14:48:59.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8568000000000002e-05 2024-09-16T14:48:59.279 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.279 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.279 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.279 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:48:59.279 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:48:59.279 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.349969259000002, 2024-09-16T14:48:59.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065747590000000003, 2024-09-16T14:48:59.280 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.280 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.280 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:48:59.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:48:59.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914043+0000", 2024-09-16T14:48:59.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914071+0000", 2024-09-16T14:48:59.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7633000000000001e-05 2024-09-16T14:48:59.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914086+0000", 2024-09-16T14:48:59.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4973999999999999e-05 2024-09-16T14:48:59.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914095+0000", 2024-09-16T14:48:59.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2119999999999992e-06 2024-09-16T14:48:59.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915626+0000", 2024-09-16T14:48:59.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001530998 2024-09-16T14:48:59.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915635+0000", 2024-09-16T14:48:59.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0140000000000004e-06 2024-09-16T14:48:59.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920590+0000", 2024-09-16T14:48:59.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049547740000000003 2024-09-16T14:48:59.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920618+0000", 2024-09-16T14:48:59.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8767000000000001e-05 2024-09-16T14:48:59.283 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.284 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.284 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.284 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:48:59.284 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:59.284 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.347646769000001, 2024-09-16T14:48:59.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006879947, 2024-09-16T14:48:59.284 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.284 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.284 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:48:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:59.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8702000000000002e-05 2024-09-16T14:48:59.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916394+0000", 2024-09-16T14:48:59.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6350000000000001e-06 2024-09-16T14:48:59.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916403+0000", 2024-09-16T14:48:59.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3950000000000001e-06 2024-09-16T14:48:59.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916459+0000", 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5889e-05 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916467+0000", 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2770000000000005e-06 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923224+0000", 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067573099999999999 2024-09-16T14:48:59.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.288 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.288 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923246+0000", 2024-09-16T14:48:59.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1882000000000002e-05 2024-09-16T14:48:59.288 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.288 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.288 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.288 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=82, reqid=osd.3.0:80, at_version=0'0, trim_to=0'0, roll_forward_to=34'20))", 2024-09-16T14:48:59.288 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:48:59.288 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.346072904, 2024-09-16T14:48:59.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005378954, 2024-09-16T14:48:59.288 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.289 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.289 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:48:59.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:48:59.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917939+0000", 2024-09-16T14:48:59.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917952+0000", 2024-09-16T14:48:59.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2278e-05 2024-09-16T14:48:59.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917963+0000", 2024-09-16T14:48:59.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.102e-05 2024-09-16T14:48:59.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917970+0000", 2024-09-16T14:48:59.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9589999999999998e-06 2024-09-16T14:48:59.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.918179+0000", 2024-09-16T14:48:59.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020897 2024-09-16T14:48:59.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.918188+0000", 2024-09-16T14:48:59.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9600000000000006e-06 2024-09-16T14:48:59.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923288+0000", 2024-09-16T14:48:59.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051001830000000003 2024-09-16T14:48:59.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923319+0000", 2024-09-16T14:48:59.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1131999999999999e-05 2024-09-16T14:48:59.292 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.292 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.292 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.293 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=37'32, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:59.293 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:48:59.293 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.343021155000001, 2024-09-16T14:48:59.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070991120000000003, 2024-09-16T14:48:59.293 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.293 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.293 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:48:59.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:48:59.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920991+0000", 2024-09-16T14:48:59.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921010+0000", 2024-09-16T14:48:59.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8529000000000002e-05 2024-09-16T14:48:59.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921018+0000", 2024-09-16T14:48:59.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4670000000000004e-06 2024-09-16T14:48:59.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921025+0000", 2024-09-16T14:48:59.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9330000000000004e-06 2024-09-16T14:48:59.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921076+0000", 2024-09-16T14:48:59.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0662999999999999e-05 2024-09-16T14:48:59.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921084+0000", 2024-09-16T14:48:59.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4780000000000008e-06 2024-09-16T14:48:59.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928073+0000", 2024-09-16T14:48:59.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069883940000000002 2024-09-16T14:48:59.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928091+0000", 2024-09-16T14:48:59.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7992999999999999e-05 2024-09-16T14:48:59.297 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.297 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.297 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.297 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s6 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:59.297 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:48:59.297 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.3420579, 2024-09-16T14:48:59.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059936520000000004, 2024-09-16T14:48:59.297 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.297 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.297 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:48:59.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:48:59.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:48:59.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921983+0000", 2024-09-16T14:48:59.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8493999999999999e-05 2024-09-16T14:48:59.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:48:59.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3110000000000006e-06 2024-09-16T14:48:59.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921999+0000", 2024-09-16T14:48:59.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5499999999999997e-06 2024-09-16T14:48:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922042+0000", 2024-09-16T14:48:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3387999999999999e-05 2024-09-16T14:48:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922053+0000", 2024-09-16T14:48:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0244999999999999e-05 2024-09-16T14:48:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.927921+0000", 2024-09-16T14:48:59.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0058679159999999999 2024-09-16T14:48:59.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.927949+0000", 2024-09-16T14:48:59.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8038000000000002e-05 2024-09-16T14:48:59.301 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.301 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.301 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.301 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:59.301 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:48:59.301 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.34202185, 2024-09-16T14:48:59.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070100509999999998, 2024-09-16T14:48:59.302 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.302 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.302 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:48:59.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:48:59.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921990+0000", 2024-09-16T14:48:59.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922012+0000", 2024-09-16T14:48:59.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1681000000000001e-05 2024-09-16T14:48:59.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922025+0000", 2024-09-16T14:48:59.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3101999999999999e-05 2024-09-16T14:48:59.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922033+0000", 2024-09-16T14:48:59.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1899999999999995e-06 2024-09-16T14:48:59.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922727+0000", 2024-09-16T14:48:59.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00069429699999999997 2024-09-16T14:48:59.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922737+0000", 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.285e-06 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928986+0000", 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062492679999999997 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929001+0000", 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5058999999999999e-05 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.306 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 37/27 ECSubWrite(tid=189, reqid=osd.7.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'12))", 2024-09-16T14:48:59.306 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922523+0000", 2024-09-16T14:48:59.306 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.341489985999999, 2024-09-16T14:48:59.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055324470000000002, 2024-09-16T14:48:59.306 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.306 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.306 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922523+0000", 2024-09-16T14:48:59.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922523+0000", 2024-09-16T14:48:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922522+0000", 2024-09-16T14:48:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922531+0000", 2024-09-16T14:48:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6079999999999996e-06 2024-09-16T14:48:59.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922540+0000", 2024-09-16T14:48:59.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-16T14:48:59.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922548+0000", 2024-09-16T14:48:59.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4089999999999997e-06 2024-09-16T14:48:59.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922589+0000", 2024-09-16T14:48:59.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0853999999999999e-05 2024-09-16T14:48:59.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922597+0000", 2024-09-16T14:48:59.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3270000000000001e-06 2024-09-16T14:48:59.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928016+0000", 2024-09-16T14:48:59.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054186269999999996 2024-09-16T14:48:59.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928055+0000", 2024-09-16T14:48:59.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9749999999999997e-05 2024-09-16T14:48:59.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.310 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.310 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.ds2 37/27 ECSubWrite(tid=190, reqid=osd.7.0:188, at_version=0'0, trim_to=0'0, roll_forward_to=37'20))", 2024-09-16T14:48:59.310 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:48:59.310 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.341263395, 2024-09-16T14:48:59.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062183510000000004, 2024-09-16T14:48:59.310 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.310 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.310 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:48:59.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:48:59.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:48:59.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922758+0000", 2024-09-16T14:48:59.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6209999999999993e-06 2024-09-16T14:48:59.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922767+0000", 2024-09-16T14:48:59.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4159999999999993e-06 2024-09-16T14:48:59.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922780+0000", 2024-09-16T14:48:59.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2992e-05 2024-09-16T14:48:59.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922834+0000", 2024-09-16T14:48:59.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3835999999999998e-05 2024-09-16T14:48:59.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922911+0000", 2024-09-16T14:48:59.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7534999999999995e-05 2024-09-16T14:48:59.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928945+0000", 2024-09-16T14:48:59.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060339210000000002 2024-09-16T14:48:59.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928968+0000", 2024-09-16T14:48:59.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2450999999999999e-05 2024-09-16T14:48:59.314 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.314 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.314 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.314 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s6 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:48:59.314 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923698+0000", 2024-09-16T14:48:59.315 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.340315141, 2024-09-16T14:48:59.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055372479999999998, 2024-09-16T14:48:59.315 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.315 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.315 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923698+0000", 2024-09-16T14:48:59.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923698+0000", 2024-09-16T14:48:59.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923697+0000", 2024-09-16T14:48:59.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923708+0000", 2024-09-16T14:48:59.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1524e-05 2024-09-16T14:48:59.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923720+0000", 2024-09-16T14:48:59.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1620000000000001e-05 2024-09-16T14:48:59.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923732+0000", 2024-09-16T14:48:59.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1711e-05 2024-09-16T14:48:59.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923796+0000", 2024-09-16T14:48:59.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4673999999999998e-05 2024-09-16T14:48:59.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923805+0000", 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3240000000000003e-06 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929203+0000", 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005398351 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929235+0000", 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1902000000000001e-05 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.318 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.319 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.319 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:48:59.319 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:48:59.319 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.338812785, 2024-09-16T14:48:59.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079457759999999999, 2024-09-16T14:48:59.319 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.319 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.319 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:48:59.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:48:59.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:48:59.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925222+0000", 2024-09-16T14:48:59.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1871000000000001e-05 2024-09-16T14:48:59.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925238+0000", 2024-09-16T14:48:59.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6719999999999999e-05 2024-09-16T14:48:59.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925244+0000", 2024-09-16T14:48:59.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3630000000000002e-06 2024-09-16T14:48:59.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925291+0000", 2024-09-16T14:48:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6944000000000003e-05 2024-09-16T14:48:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925299+0000", 2024-09-16T14:48:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:48:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933129+0000", 2024-09-16T14:48:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078297330000000002 2024-09-16T14:48:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933146+0000", 2024-09-16T14:48:59.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.664e-05 2024-09-16T14:48:59.323 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.323 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.323 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.323 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es5 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:48:59.323 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:48:59.323 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.33876115, 2024-09-16T14:48:59.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078358790000000005, 2024-09-16T14:48:59.323 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.324 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.324 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:48:59.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:48:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925251+0000", 2024-09-16T14:48:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925265+0000", 2024-09-16T14:48:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4078e-05 2024-09-16T14:48:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925273+0000", 2024-09-16T14:48:59.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1069999999999999e-06 2024-09-16T14:48:59.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925279+0000", 2024-09-16T14:48:59.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4409999999999998e-06 2024-09-16T14:48:59.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925330+0000", 2024-09-16T14:48:59.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1208999999999997e-05 2024-09-16T14:48:59.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925339+0000", 2024-09-16T14:48:59.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8109999999999992e-06 2024-09-16T14:48:59.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933064+0000", 2024-09-16T14:48:59.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077252149999999997 2024-09-16T14:48:59.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933088+0000", 2024-09-16T14:48:59.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3317999999999999e-05 2024-09-16T14:48:59.327 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.328 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.328 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.328 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 37/27 ECSubWrite(tid=125, reqid=osd.3.0:125, at_version=37'43, trim_to=0'0, roll_forward_to=37'41))", 2024-09-16T14:48:59.328 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.943728+0000", 2024-09-16T14:48:59.328 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.320285065, 2024-09-16T14:48:59.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005567626, 2024-09-16T14:48:59.328 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.328 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.328 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943728+0000", 2024-09-16T14:48:59.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943728+0000", 2024-09-16T14:48:59.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943727+0000", 2024-09-16T14:48:59.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943749+0000", 2024-09-16T14:48:59.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1634999999999999e-05 2024-09-16T14:48:59.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943760+0000", 2024-09-16T14:48:59.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1727e-05 2024-09-16T14:48:59.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943768+0000", 2024-09-16T14:48:59.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4529999999999996e-06 2024-09-16T14:48:59.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943814+0000", 2024-09-16T14:48:59.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6346e-05 2024-09-16T14:48:59.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943823+0000", 2024-09-16T14:48:59.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3230000000000004e-06 2024-09-16T14:48:59.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949278+0000", 2024-09-16T14:48:59.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054548649999999997 2024-09-16T14:48:59.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949295+0000", 2024-09-16T14:48:59.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7079e-05 2024-09-16T14:48:59.332 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.332 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.332 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.332 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.ds2 47/27 ECSubWrite(tid=208, reqid=client.4500.0:390, at_version=0'0, trim_to=0'0, roll_forward_to=47'27))", 2024-09-16T14:48:59.332 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.873647+0000", 2024-09-16T14:48:59.332 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.3903661610000002, 2024-09-16T14:48:59.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065883130000000002, 2024-09-16T14:48:59.333 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.333 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.333 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873647+0000", 2024-09-16T14:48:59.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873647+0000", 2024-09-16T14:48:59.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873646+0000", 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873660+0000", 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4056000000000001e-05 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873675+0000", 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.543e-05 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873683+0000", 2024-09-16T14:48:59.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0879999999999995e-06 2024-09-16T14:48:59.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873732+0000", 2024-09-16T14:48:59.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8738e-05 2024-09-16T14:48:59.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873742+0000", 2024-09-16T14:48:59.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8949999999999993e-06 2024-09-16T14:48:59.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880210+0000", 2024-09-16T14:48:59.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006468321 2024-09-16T14:48:59.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880235+0000", 2024-09-16T14:48:59.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.495e-05 2024-09-16T14:48:59.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.336 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.337 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 47/27 ECSubWrite(tid=45, reqid=client.4500.0:393, at_version=47'34, trim_to=0'0, roll_forward_to=43'30))", 2024-09-16T14:48:59.337 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.873840+0000", 2024-09-16T14:48:59.337 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.3901728379999998, 2024-09-16T14:48:59.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064368940000000003, 2024-09-16T14:48:59.337 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.337 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.337 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873840+0000", 2024-09-16T14:48:59.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873840+0000", 2024-09-16T14:48:59.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873839+0000", 2024-09-16T14:48:59.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873858+0000", 2024-09-16T14:48:59.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9228e-05 2024-09-16T14:48:59.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873870+0000", 2024-09-16T14:48:59.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1450999999999999e-05 2024-09-16T14:48:59.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873878+0000", 2024-09-16T14:48:59.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-16T14:48:59.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873936+0000", 2024-09-16T14:48:59.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7374999999999999e-05 2024-09-16T14:48:59.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873945+0000", 2024-09-16T14:48:59.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.662e-06 2024-09-16T14:48:59.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880251+0000", 2024-09-16T14:48:59.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063056099999999997 2024-09-16T14:48:59.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880277+0000", 2024-09-16T14:48:59.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6013999999999999e-05 2024-09-16T14:48:59.341 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.341 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.341 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.341 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.9s3 47/27 ECSubWrite(tid=28, reqid=client.4500.0:396, at_version=47'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-16T14:48:59.341 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.885200+0000", 2024-09-16T14:48:59.341 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.3788132160000002, 2024-09-16T14:48:59.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053724640000000004, 2024-09-16T14:48:59.341 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.341 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.341 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885200+0000", 2024-09-16T14:48:59.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885200+0000", 2024-09-16T14:48:59.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885199+0000", 2024-09-16T14:48:59.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885217+0000", 2024-09-16T14:48:59.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8190999999999999e-05 2024-09-16T14:48:59.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885232+0000", 2024-09-16T14:48:59.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4676e-05 2024-09-16T14:48:59.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885240+0000", 2024-09-16T14:48:59.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6240000000000008e-06 2024-09-16T14:48:59.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885296+0000", 2024-09-16T14:48:59.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5705000000000003e-05 2024-09-16T14:48:59.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885307+0000", 2024-09-16T14:48:59.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1601e-05 2024-09-16T14:48:59.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890555+0000", 2024-09-16T14:48:59.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052472229999999996 2024-09-16T14:48:59.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890572+0000", 2024-09-16T14:48:59.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7394999999999999e-05 2024-09-16T14:48:59.345 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.345 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.345 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.345 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 51/27 ECSubWrite(tid=165, reqid=osd.3.0:165, at_version=51'48, trim_to=0'0, roll_forward_to=50'46))", 2024-09-16T14:48:59.345 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.877962+0000", 2024-09-16T14:48:59.345 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.3860505869999999, 2024-09-16T14:48:59.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050691460000000001, 2024-09-16T14:48:59.346 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.346 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.346 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.877962+0000", 2024-09-16T14:48:59.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.877962+0000", 2024-09-16T14:48:59.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.877962+0000", 2024-09-16T14:48:59.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.877999+0000", 2024-09-16T14:48:59.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7058999999999999e-05 2024-09-16T14:48:59.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.878015+0000", 2024-09-16T14:48:59.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6370000000000001e-05 2024-09-16T14:48:59.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.878024+0000", 2024-09-16T14:48:59.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1880000000000008e-06 2024-09-16T14:48:59.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.878113+0000", 2024-09-16T14:48:59.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8469999999999998e-05 2024-09-16T14:48:59.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.878122+0000", 2024-09-16T14:48:59.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-16T14:48:59.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.883000+0000", 2024-09-16T14:48:59.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048776189999999997 2024-09-16T14:48:59.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.883031+0000", 2024-09-16T14:48:59.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1316999999999998e-05 2024-09-16T14:48:59.349 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.349 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.349 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.350 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 51/27 ECSubWrite(tid=171, reqid=osd.11.0:171, at_version=51'22, trim_to=0'0, roll_forward_to=49'20))", 2024-09-16T14:48:59.350 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.881321+0000", 2024-09-16T14:48:59.350 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.3826917459999999, 2024-09-16T14:48:59.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064808820000000003, 2024-09-16T14:48:59.350 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.350 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.350 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881321+0000", 2024-09-16T14:48:59.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881321+0000", 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881321+0000", 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881482+0000", 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000161318 2024-09-16T14:48:59.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881495+0000", 2024-09-16T14:48:59.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3210999999999999e-05 2024-09-16T14:48:59.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881504+0000", 2024-09-16T14:48:59.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.772e-06 2024-09-16T14:48:59.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881563+0000", 2024-09-16T14:48:59.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9608999999999998e-05 2024-09-16T14:48:59.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881575+0000", 2024-09-16T14:48:59.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1904e-05 2024-09-16T14:48:59.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.887760+0000", 2024-09-16T14:48:59.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061849319999999998 2024-09-16T14:48:59.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.887802+0000", 2024-09-16T14:48:59.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1706999999999999e-05 2024-09-16T14:48:59.354 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.354 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.354 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.354 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.354 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.354 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:59.355 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:48:59.356 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:48:59.354+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:48:59.456 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:59.454+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:59.458 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:59.458 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:59.458 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:59.458 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:59.467 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:48:59.556 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:48:59.556+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:48:59.608 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:59.608 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:59.608 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:48:59.608 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:48:59.608 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:59.616 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:48:59.658 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:48:59.657+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:48:59.716 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:48:59.721 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:59.721 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:48:59.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:48:59.721 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:48:59.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.721 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 34/27 ECSubWrite(tid=80, reqid=osd.3.0:80, at_version=34'20, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:48:59.721 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:48:59.721 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.809797366000002, 2024-09-16T14:48:59.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055013639999999999, 2024-09-16T14:48:59.721 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.721 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.722 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:48:59.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:48:59.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909414+0000", 2024-09-16T14:48:59.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909442+0000", 2024-09-16T14:48:59.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7662999999999999e-05 2024-09-16T14:48:59.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909452+0000", 2024-09-16T14:48:59.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0773e-05 2024-09-16T14:48:59.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909463+0000", 2024-09-16T14:48:59.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0144e-05 2024-09-16T14:48:59.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909522+0000", 2024-09-16T14:48:59.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9651000000000002e-05 2024-09-16T14:48:59.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909531+0000", 2024-09-16T14:48:59.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8529999999999998e-06 2024-09-16T14:48:59.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914874+0000", 2024-09-16T14:48:59.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053432829999999999 2024-09-16T14:48:59.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914916+0000", 2024-09-16T14:48:59.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1928000000000002e-05 2024-09-16T14:48:59.726 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.726 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.726 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.726 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:48:59.726 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:48:59.726 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.809728556, 2024-09-16T14:48:59.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079855010000000008, 2024-09-16T14:48:59.726 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.726 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.726 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:48:59.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:48:59.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909483+0000", 2024-09-16T14:48:59.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909521+0000", 2024-09-16T14:48:59.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7185999999999999e-05 2024-09-16T14:48:59.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909543+0000", 2024-09-16T14:48:59.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2802999999999998e-05 2024-09-16T14:48:59.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909551+0000", 2024-09-16T14:48:59.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5039999999999996e-06 2024-09-16T14:48:59.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909927+0000", 2024-09-16T14:48:59.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037637400000000001 2024-09-16T14:48:59.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:48:59.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2040000000000002e-06 2024-09-16T14:48:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917442+0000", 2024-09-16T14:48:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007508659 2024-09-16T14:48:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917469+0000", 2024-09-16T14:48:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7197999999999999e-05 2024-09-16T14:48:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.731 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:48:59.731 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:59.731 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.805656209999999, 2024-09-16T14:48:59.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067701159999999996, 2024-09-16T14:48:59.731 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.731 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.731 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:59.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:59.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:48:59.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913579+0000", 2024-09-16T14:48:59.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2829000000000001e-05 2024-09-16T14:48:59.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913591+0000", 2024-09-16T14:48:59.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2264e-05 2024-09-16T14:48:59.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913600+0000", 2024-09-16T14:48:59.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9959999999999999e-06 2024-09-16T14:48:59.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913648+0000", 2024-09-16T14:48:59.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7759000000000001e-05 2024-09-16T14:48:59.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913657+0000", 2024-09-16T14:48:59.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6709999999999994e-06 2024-09-16T14:48:59.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920296+0000", 2024-09-16T14:48:59.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066385120000000001 2024-09-16T14:48:59.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920326+0000", 2024-09-16T14:48:59.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0405000000000001e-05 2024-09-16T14:48:59.738 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.738 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.738 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.738 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 34/27 ECSubWrite(tid=74, reqid=client.4500.0:229, at_version=34'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:48:59.738 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:59.739 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.804840251999998, 2024-09-16T14:48:59.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060056090000000003, 2024-09-16T14:48:59.739 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.739 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.739 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:59.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:59.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:48:59.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914468+0000", 2024-09-16T14:48:59.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6398999999999996e-05 2024-09-16T14:48:59.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914483+0000", 2024-09-16T14:48:59.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5238e-05 2024-09-16T14:48:59.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914494+0000", 2024-09-16T14:48:59.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0192e-05 2024-09-16T14:48:59.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914622+0000", 2024-09-16T14:48:59.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012855599999999999 2024-09-16T14:48:59.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914635+0000", 2024-09-16T14:48:59.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2510000000000001e-05 2024-09-16T14:48:59.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920339+0000", 2024-09-16T14:48:59.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057041549999999998 2024-09-16T14:48:59.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920378+0000", 2024-09-16T14:48:59.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8918999999999998e-05 2024-09-16T14:48:59.744 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.744 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.744 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.744 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:48:59.744 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:59.744 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.802827476000001, 2024-09-16T14:48:59.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069011319999999999, 2024-09-16T14:48:59.744 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.744 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.744 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:59.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:59.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:48:59.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:59.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916410+0000", 2024-09-16T14:48:59.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5343000000000001e-05 2024-09-16T14:48:59.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916418+0000", 2024-09-16T14:48:59.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2360000000000004e-06 2024-09-16T14:48:59.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916428+0000", 2024-09-16T14:48:59.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6779999999999995e-06 2024-09-16T14:48:59.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916492+0000", 2024-09-16T14:48:59.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4219000000000003e-05 2024-09-16T14:48:59.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916502+0000", 2024-09-16T14:48:59.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6690000000000001e-06 2024-09-16T14:48:59.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923254+0000", 2024-09-16T14:48:59.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067518070000000003 2024-09-16T14:48:59.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923286+0000", 2024-09-16T14:48:59.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2367999999999999e-05 2024-09-16T14:48:59.748 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.748 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.748 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.748 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 34/27 ECSubWrite(tid=75, reqid=client.4500.0:229, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-09-16T14:48:59.749 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.920873+0000", 2024-09-16T14:48:59.749 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.798339062, 2024-09-16T14:48:59.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049917640000000001, 2024-09-16T14:48:59.749 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.749 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.749 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920873+0000", 2024-09-16T14:48:59.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920873+0000", 2024-09-16T14:48:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920872+0000", 2024-09-16T14:48:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920884+0000", 2024-09-16T14:48:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1501999999999999e-05 2024-09-16T14:48:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920898+0000", 2024-09-16T14:48:59.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4391e-05 2024-09-16T14:48:59.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920916+0000", 2024-09-16T14:48:59.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8057e-05 2024-09-16T14:48:59.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920965+0000", 2024-09-16T14:48:59.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8962000000000003e-05 2024-09-16T14:48:59.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920975+0000", 2024-09-16T14:48:59.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3989999999999993e-06 2024-09-16T14:48:59.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925839+0000", 2024-09-16T14:48:59.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048646330000000002 2024-09-16T14:48:59.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.753 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.753 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925865+0000", 2024-09-16T14:48:59.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5703000000000001e-05 2024-09-16T14:48:59.753 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.753 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.753 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.753 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/27 ECSubWrite(tid=169, reqid=osd.7.0:168, at_version=0'0, trim_to=0'0, roll_forward_to=34'8))", 2024-09-16T14:48:59.753 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.921046+0000", 2024-09-16T14:48:59.753 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.798166337000001, 2024-09-16T14:48:59.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048734829999999996, 2024-09-16T14:48:59.753 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.754 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.754 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921046+0000", 2024-09-16T14:48:59.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921046+0000", 2024-09-16T14:48:59.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921045+0000", 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921056+0000", 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0733e-05 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921067+0000", 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1048e-05 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.755 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921076+0000", 2024-09-16T14:48:59.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:48:59.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921135+0000", 2024-09-16T14:48:59.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8681000000000003e-05 2024-09-16T14:48:59.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.756 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.921144+0000", 2024-09-16T14:48:59.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9399999999999997e-06 2024-09-16T14:48:59.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.756 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925903+0000", 2024-09-16T14:48:59.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047588860000000004 2024-09-16T14:48:59.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.925920+0000", 2024-09-16T14:48:59.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6147999999999999e-05 2024-09-16T14:48:59.757 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.757 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.757 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.758 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s7 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:48:59.758 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:48:59.758 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.797523092, 2024-09-16T14:48:59.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006972692, 2024-09-16T14:48:59.758 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.758 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.758 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:48:59.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:48:59.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:48:59.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921709+0000", 2024-09-16T14:48:59.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0411000000000002e-05 2024-09-16T14:48:59.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921719+0000", 2024-09-16T14:48:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2650000000000002e-06 2024-09-16T14:48:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921726+0000", 2024-09-16T14:48:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8770000000000004e-06 2024-09-16T14:48:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921773+0000", 2024-09-16T14:48:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7435999999999998e-05 2024-09-16T14:48:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921782+0000", 2024-09-16T14:48:59.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6039999999999993e-06 2024-09-16T14:48:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928635+0000", 2024-09-16T14:48:59.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068533129999999998 2024-09-16T14:48:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928662+0000", 2024-09-16T14:48:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7089000000000001e-05 2024-09-16T14:48:59.766 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:48:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:48:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.797026603999999, 2024-09-16T14:48:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064783530000000001, 2024-09-16T14:48:59.768 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.769 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:48:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.770 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:48:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.771 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922185+0000", 2024-09-16T14:48:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.772 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922205+0000", 2024-09-16T14:48:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9871e-05 2024-09-16T14:48:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.773 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922219+0000", 2024-09-16T14:48:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4266e-05 2024-09-16T14:48:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922229+0000", 2024-09-16T14:48:59.774 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5619999999999993e-06 2024-09-16T14:48:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922270+0000", 2024-09-16T14:48:59.775 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0917000000000001e-05 2024-09-16T14:48:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922278+0000", 2024-09-16T14:48:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3890000000000005e-06 2024-09-16T14:48:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928630+0000", 2024-09-16T14:48:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063524439999999996 2024-09-16T14:48:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928664+0000", 2024-09-16T14:48:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3651e-05 2024-09-16T14:48:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s5 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:48:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:48:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.7955782, 2024-09-16T14:48:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069615850000000002, 2024-09-16T14:48:59.779 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:48:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:48:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.780 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923633+0000", 2024-09-16T14:48:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923644+0000", 2024-09-16T14:48:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.08e-05 2024-09-16T14:48:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923657+0000", 2024-09-16T14:48:59.781 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2836999999999999e-05 2024-09-16T14:48:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923664+0000", 2024-09-16T14:48:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1470000000000004e-06 2024-09-16T14:48:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923885+0000", 2024-09-16T14:48:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022155800000000001 2024-09-16T14:48:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923894+0000", 2024-09-16T14:48:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9260000000000006e-06 2024-09-16T14:48:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930571+0000", 2024-09-16T14:48:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066770420000000002 2024-09-16T14:48:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930596+0000", 2024-09-16T14:48:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4383e-05 2024-09-16T14:48:59.783 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as3 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:48:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:48:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.794831964, 2024-09-16T14:48:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062471519999999997, 2024-09-16T14:48:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.784 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:48:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:48:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.785 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:48:59.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924405+0000", 2024-09-16T14:48:59.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5147000000000002e-05 2024-09-16T14:48:59.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.786 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.786 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.786 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924414+0000", 2024-09-16T14:48:59.786 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4810000000000006e-06 2024-09-16T14:48:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924421+0000", 2024-09-16T14:48:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0899999999999999e-06 2024-09-16T14:48:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924468+0000", 2024-09-16T14:48:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.727e-05 2024-09-16T14:48:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.787 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924477+0000", 2024-09-16T14:48:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7360000000000007e-06 2024-09-16T14:48:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930601+0000", 2024-09-16T14:48:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061242529999999996 2024-09-16T14:48:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930628+0000", 2024-09-16T14:48:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6517e-05 2024-09-16T14:48:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.788 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:48:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:48:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.79458176, 2024-09-16T14:48:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006019328, 2024-09-16T14:48:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.789 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:48:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:48:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924630+0000", 2024-09-16T14:48:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924650+0000", 2024-09-16T14:48:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0024e-05 2024-09-16T14:48:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924660+0000", 2024-09-16T14:48:59.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9219999999999992e-06 2024-09-16T14:48:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924668+0000", 2024-09-16T14:48:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6820000000000003e-06 2024-09-16T14:48:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924711+0000", 2024-09-16T14:48:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3009000000000001e-05 2024-09-16T14:48:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924720+0000", 2024-09-16T14:48:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9290000000000004e-06 2024-09-16T14:48:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930634+0000", 2024-09-16T14:48:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059144810000000001 2024-09-16T14:48:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930650+0000", 2024-09-16T14:48:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5688999999999998e-05 2024-09-16T14:48:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.793 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.5s7 37/27 ECSubWrite(tid=48, reqid=osd.5.0:47, at_version=0'0, trim_to=0'0, roll_forward_to=37'11))", 2024-09-16T14:48:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925599+0000", 2024-09-16T14:48:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.793613173000001, 2024-09-16T14:48:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050916670000000002, 2024-09-16T14:48:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925599+0000", 2024-09-16T14:48:59.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925599+0000", 2024-09-16T14:48:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925598+0000", 2024-09-16T14:48:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925608+0000", 2024-09-16T14:48:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0034000000000001e-05 2024-09-16T14:48:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925619+0000", 2024-09-16T14:48:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0575e-05 2024-09-16T14:48:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925624+0000", 2024-09-16T14:48:59.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3909999999999997e-06 2024-09-16T14:48:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925664+0000", 2024-09-16T14:48:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9379999999999999e-05 2024-09-16T14:48:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925670+0000", 2024-09-16T14:48:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6619999999999999e-06 2024-09-16T14:48:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930669+0000", 2024-09-16T14:48:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049987199999999999 2024-09-16T14:48:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930691+0000", 2024-09-16T14:48:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1695000000000002e-05 2024-09-16T14:48:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 37/27 ECSubWrite(tid=123, reqid=osd.3.0:123, at_version=37'38, trim_to=0'0, roll_forward_to=37'34))", 2024-09-16T14:48:59.798 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.935961+0000", 2024-09-16T14:48:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.783251191, 2024-09-16T14:48:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052566519999999997, 2024-09-16T14:48:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.935961+0000", 2024-09-16T14:48:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.935961+0000", 2024-09-16T14:48:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.935961+0000", 2024-09-16T14:48:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936022+0000", 2024-09-16T14:48:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1205999999999996e-05 2024-09-16T14:48:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936037+0000", 2024-09-16T14:48:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5478e-05 2024-09-16T14:48:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936045+0000", 2024-09-16T14:48:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7060000000000004e-06 2024-09-16T14:48:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936604+0000", 2024-09-16T14:48:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00055949199999999998 2024-09-16T14:48:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.936612+0000", 2024-09-16T14:48:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8040000000000002e-06 2024-09-16T14:48:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941192+0000", 2024-09-16T14:48:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045797370000000004 2024-09-16T14:48:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941218+0000", 2024-09-16T14:48:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5826e-05 2024-09-16T14:48:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.8s6 37/27 ECSubWrite(tid=110, reqid=osd.11.0:110, at_version=37'35, trim_to=0'0, roll_forward_to=35'31))", 2024-09-16T14:48:59.803 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.941403+0000", 2024-09-16T14:48:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.777809391, 2024-09-16T14:48:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053034789999999998, 2024-09-16T14:48:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941403+0000", 2024-09-16T14:48:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941403+0000", 2024-09-16T14:48:59.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941402+0000", 2024-09-16T14:48:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941417+0000", 2024-09-16T14:48:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4578e-05 2024-09-16T14:48:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941431+0000", 2024-09-16T14:48:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3729e-05 2024-09-16T14:48:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941439+0000", 2024-09-16T14:48:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0199999999999994e-06 2024-09-16T14:48:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941487+0000", 2024-09-16T14:48:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8936e-05 2024-09-16T14:48:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941496+0000", 2024-09-16T14:48:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5320000000000006e-06 2024-09-16T14:48:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.946682+0000", 2024-09-16T14:48:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051858779999999997 2024-09-16T14:48:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.946706+0000", 2024-09-16T14:48:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4541000000000001e-05 2024-09-16T14:48:59.807 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 37/27 ECSubWrite(tid=125, reqid=osd.3.0:125, at_version=37'43, trim_to=0'0, roll_forward_to=37'41))", 2024-09-16T14:48:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.943746+0000", 2024-09-16T14:48:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.775465863999999, 2024-09-16T14:48:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054472219999999998, 2024-09-16T14:48:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943746+0000", 2024-09-16T14:48:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943746+0000", 2024-09-16T14:48:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943746+0000", 2024-09-16T14:48:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:48:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943767+0000", 2024-09-16T14:48:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1325e-05 2024-09-16T14:48:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943779+0000", 2024-09-16T14:48:59.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1865000000000001e-05 2024-09-16T14:48:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943787+0000", 2024-09-16T14:48:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7449999999999995e-06 2024-09-16T14:48:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943828+0000", 2024-09-16T14:48:59.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0985000000000001e-05 2024-09-16T14:48:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943836+0000", 2024-09-16T14:48:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5459999999999992e-06 2024-09-16T14:48:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949139+0000", 2024-09-16T14:48:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053031220000000004 2024-09-16T14:48:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949194+0000", 2024-09-16T14:48:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4360000000000001e-05 2024-09-16T14:48:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.813 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "pg_update_log_missing(3.5s7 epoch 51/27 rep_tid 54 entries 51'14 (0'0) error 3:a7111aa9:::smithi04040151-45:head by client.4500.0:449 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 37'11)", 2024-09-16T14:48:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.879195+0000", 2024-09-16T14:48:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.840017655, 2024-09-16T14:48:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051427, 2024-09-16T14:48:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.814 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879195+0000", 2024-09-16T14:48:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879195+0000", 2024-09-16T14:48:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879194+0000", 2024-09-16T14:48:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:48:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879203+0000", 2024-09-16T14:48:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1130000000000001e-06 2024-09-16T14:48:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879215+0000", 2024-09-16T14:48:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2289e-05 2024-09-16T14:48:59.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879226+0000", 2024-09-16T14:48:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0721e-05 2024-09-16T14:48:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879309+0000", 2024-09-16T14:48:59.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3679999999999998e-05 2024-09-16T14:48:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.884337+0000", 2024-09-16T14:48:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050281279999999998 2024-09-16T14:48:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.8s6 51/27 ECSubWrite(tid=169, reqid=osd.11.0:169, at_version=51'52, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:48:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.880753+0000", 2024-09-16T14:48:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.838458964, 2024-09-16T14:48:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005355032, 2024-09-16T14:48:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.820 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880753+0000", 2024-09-16T14:48:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880753+0000", 2024-09-16T14:48:59.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880753+0000", 2024-09-16T14:48:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880794+0000", 2024-09-16T14:48:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1371000000000001e-05 2024-09-16T14:48:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880806+0000", 2024-09-16T14:48:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1382e-05 2024-09-16T14:48:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880814+0000", 2024-09-16T14:48:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3760000000000008e-06 2024-09-16T14:48:59.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880876+0000", 2024-09-16T14:48:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1509999999999999e-05 2024-09-16T14:48:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880887+0000", 2024-09-16T14:48:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1090999999999999e-05 2024-09-16T14:48:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.886067+0000", 2024-09-16T14:48:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051807950000000002 2024-09-16T14:48:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.886108+0000", 2024-09-16T14:48:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0967999999999998e-05 2024-09-16T14:48:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.825 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.8s6 51/27 ECSubWrite(tid=173, reqid=osd.11.0:173, at_version=51'54, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:48:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.881283+0000", 2024-09-16T14:48:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.837929672, 2024-09-16T14:48:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057129700000000004, 2024-09-16T14:48:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881283+0000", 2024-09-16T14:48:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881283+0000", 2024-09-16T14:48:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881282+0000", 2024-09-16T14:48:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881333+0000", 2024-09-16T14:48:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0413e-05 2024-09-16T14:48:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881345+0000", 2024-09-16T14:48:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1933000000000001e-05 2024-09-16T14:48:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881353+0000", 2024-09-16T14:48:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0469999999999994e-06 2024-09-16T14:48:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882512+0000", 2024-09-16T14:48:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001159806 2024-09-16T14:48:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882521+0000", 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9639999999999992e-06 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.886958+0000", 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044365170000000001 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.886996+0000", 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7808999999999997e-05 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.830 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 51/27 ECSubWrite(tid=47, reqid=osd.1.0:47, at_version=51'36, trim_to=0'0, roll_forward_to=47'34))", 2024-09-16T14:48:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.889168+0000", 2024-09-16T14:48:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.830044682, 2024-09-16T14:48:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047161149999999999, 2024-09-16T14:48:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:48:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:48:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:48:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:48:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889168+0000", 2024-09-16T14:48:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:48:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889168+0000", 2024-09-16T14:48:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:48:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:48:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889167+0000", 2024-09-16T14:48:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:48:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:48:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889194+0000", 2024-09-16T14:48:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6358000000000001e-05 2024-09-16T14:48:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:48:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889204+0000", 2024-09-16T14:48:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0009e-05 2024-09-16T14:48:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:48:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889212+0000", 2024-09-16T14:48:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4009999999999997e-06 2024-09-16T14:48:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:48:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889273+0000", 2024-09-16T14:48:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0844000000000002e-05 2024-09-16T14:48:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:48:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889283+0000", 2024-09-16T14:48:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8989999999999996e-06 2024-09-16T14:48:59.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:48:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.893843+0000", 2024-09-16T14:48:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045604180000000001 2024-09-16T14:48:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:48:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:48:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:48:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.893884+0000", 2024-09-16T14:48:59.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0686999999999999e-05 2024-09-16T14:48:59.836 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.836 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.836 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.836 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:48:59.836 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:48:59.836 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:59.840 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:48:59.841 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:48:59.842 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:48:59.836+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:48:59.842 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:48:59.850 INFO:tasks.rados.rados.0.smithi040.stdout:200: done (1 left) 2024-09-16T14:48:59.850 INFO:tasks.rados.rados.0.smithi040.stdout:201: done (0 left) 2024-09-16T14:48:59.850 INFO:tasks.rados.rados.0.smithi040.stdout:202: snap_remove snap 13 2024-09-16T14:48:59.853 INFO:teuthology.orchestra.run.smithi040.stdout:moved 3 / 146 (2.05479%) 2024-09-16T14:48:59.853 INFO:teuthology.orchestra.run.smithi040.stdout:avg 13.2727 2024-09-16T14:48:59.853 INFO:teuthology.orchestra.run.smithi040.stdout:stddev 1.54278 -> 1.35452 (expected baseline 3.47363) 2024-09-16T14:48:59.853 INFO:teuthology.orchestra.run.smithi040.stdout:min osd.2 with 10 -> 10 pgs (0.753425 -> 0.753425 * mean) 2024-09-16T14:48:59.853 INFO:teuthology.orchestra.run.smithi040.stdout:max osd.1 with 16 -> 14 pgs (1.20548 -> 1.05479 * mean) 2024-09-16T14:48:59.853 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:48:59.853 INFO:teuthology.orchestra.run.smithi040.stdout:oload 110 2024-09-16T14:48:59.853 INFO:teuthology.orchestra.run.smithi040.stdout:max_change 1 2024-09-16T14:48:59.853 INFO:teuthology.orchestra.run.smithi040.stdout:max_change_osds 4 2024-09-16T14:48:59.854 INFO:teuthology.orchestra.run.smithi040.stdout:average_utilization 0.0004 2024-09-16T14:48:59.854 INFO:teuthology.orchestra.run.smithi040.stdout:overload_utilization 0.0004 2024-09-16T14:48:59.854 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11 weight 0.9500 -> 0.8139 2024-09-16T14:48:59.854 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1 weight 0.9500 -> 0.8342 2024-09-16T14:48:59.861 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:48:59.861 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:48:59.869 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-16T14:48:59.870 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-16T14:48:59.878 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:48:59.878 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:48:59.896 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:48:59.896 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:48:59.913 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:48:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:48:59.930 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:48:59.930 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:48:59.938 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:48:59.937+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:48:59.946 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:48:59.947 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:48:59.948 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:48:59.948 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:48:59.948 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:48:59.948 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:48:59.957 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:48:59.963 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:48:59.964 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:48:59.980 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:48:59.980 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:48:59.997 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:48:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:00.015 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:00.015 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:00.031 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:00.032 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:00.039 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:00.038+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:00.100 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:00.101 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:00.110 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:49:00.140 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:00.139+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:00.157 INFO:teuthology.orchestra.run.smithi040.stdout:moved 0 / 146 (0%) 2024-09-16T14:49:00.157 INFO:teuthology.orchestra.run.smithi040.stdout:avg 13.2727 2024-09-16T14:49:00.158 INFO:teuthology.orchestra.run.smithi040.stdout:stddev 1.35452 -> 1.35452 (expected baseline 3.47363) 2024-09-16T14:49:00.158 INFO:teuthology.orchestra.run.smithi040.stdout:min osd.2 with 10 -> 10 pgs (0.753425 -> 0.753425 * mean) 2024-09-16T14:49:00.158 INFO:teuthology.orchestra.run.smithi040.stdout:max osd.10 with 15 -> 15 pgs (1.13014 -> 1.13014 * mean) 2024-09-16T14:49:00.158 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:49:00.158 INFO:teuthology.orchestra.run.smithi040.stdout:oload 1000 2024-09-16T14:49:00.158 INFO:teuthology.orchestra.run.smithi040.stdout:max_change 3 2024-09-16T14:49:00.158 INFO:teuthology.orchestra.run.smithi040.stdout:max_change_osds 4 2024-09-16T14:49:00.158 INFO:teuthology.orchestra.run.smithi040.stdout:average_utilization 0.0004 2024-09-16T14:49:00.158 INFO:teuthology.orchestra.run.smithi040.stdout:overload_utilization 0.0037 2024-09-16T14:49:00.158 INFO:teuthology.orchestra.run.smithi040.stderr:no change 2024-09-16T14:49:00.212 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:00.212 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:00.220 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:49:00.240 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:00.239+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:00.336 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:00.336 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:00.341 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:00.340+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:00.344 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:49:00.442 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:00.441+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:00.446 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:00.446 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:00.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:49:00.543 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:00.542+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:00.557 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:00.557 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:00.565 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:49:00.644 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:00.643+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:00.677 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:00.677 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:00.686 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:49:00.745 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:00.744+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:00.833 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:00.833 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:00.841 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:49:00.850 INFO:tasks.rados.rados.0.smithi040.stdout:202: done (0 left) 2024-09-16T14:49:00.850 INFO:tasks.rados.rados.0.smithi040.stdout:203: setattr oid 43 current snap is 14 2024-09-16T14:49:00.851 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:00.849+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:00.851 INFO:tasks.rados.rados.0.smithi040.stdout:204: delete oid 30 current snap is 14 2024-09-16T14:49:00.861 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 43 v 63 (ObjNum 62 snap 3 seq_num 62) dirty exists 2024-09-16T14:49:00.861 INFO:tasks.rados.rados.0.smithi040.stdout:203: done (1 left) 2024-09-16T14:49:00.861 INFO:tasks.rados.rados.0.smithi040.stdout:204: done (0 left) 2024-09-16T14:49:00.861 INFO:tasks.rados.rados.0.smithi040.stdout:205: copy_from oid 36 from oid 15 current snap is 14 2024-09-16T14:49:00.861 INFO:tasks.rados.rados.0.smithi040.stdout:206: snap_remove snap 11 2024-09-16T14:49:00.939 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:49:00.946 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:00.946+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:00.957 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:00.957 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:00.966 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:49:01.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:01.052+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:01.116 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:01.116 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:01.137 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:49:01.149 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:01.147+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:01.250 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:01.249+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:01.290 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:01.290 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:01.299 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:49:01.350 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:01.349+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:01.405 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:01.405 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:01.414 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:49:01.458 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:01.457+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:01.521 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:01.521 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:01.530 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:49:01.571 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:01.570+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:01.653 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:01.652+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:01.676 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:01.676 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:01.685 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:49:01.770 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:01.769+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:01.832 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:01.832 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:01.841 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:49:01.852 INFO:tasks.rados.rados.0.smithi040.stdout:205: finishing copy_from racing read to smithi04040151-36 2024-09-16T14:49:01.852 INFO:tasks.rados.rados.0.smithi040.stdout:205: finishing copy_from to smithi04040151-36 2024-09-16T14:49:01.853 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 36 v 2 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-16T14:49:01.853 INFO:tasks.rados.rados.0.smithi040.stdout:205: done (1 left) 2024-09-16T14:49:01.853 INFO:tasks.rados.rados.0.smithi040.stdout:206: done (0 left) 2024-09-16T14:49:01.853 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 26 current snap is 14 2024-09-16T14:49:01.853 INFO:tasks.rados.rados.0.smithi040.stdout:207: seq_num 87 ranges {1196032=704512,1900544=688128,2588672=81920} 2024-09-16T14:49:01.857 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is unset 2024-09-16T14:49:01.858 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:01.855+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:01.864 INFO:tasks.rados.rados.0.smithi040.stdout:207: writing smithi04040151-26 from 1196032 to 1900544 tid 1 2024-09-16T14:49:01.871 INFO:tasks.rados.rados.0.smithi040.stdout:207: writing smithi04040151-26 from 1900544 to 2588672 tid 2 2024-09-16T14:49:01.872 INFO:tasks.rados.rados.0.smithi040.stdout:207: writing smithi04040151-26 from 2588672 to 2670592 tid 3 2024-09-16T14:49:01.873 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 27 current snap is 14 2024-09-16T14:49:01.873 INFO:tasks.rados.rados.0.smithi040.stdout:208: seq_num 88 ranges {442368=720896,1163264=442368,1605632=327680} 2024-09-16T14:49:01.880 INFO:tasks.rados.rados.0.smithi040.stdout:208: writing smithi04040151-27 from 442368 to 1163264 tid 1 2024-09-16T14:49:01.883 INFO:tasks.rados.rados.0.smithi040.stdout:208: writing smithi04040151-27 from 1163264 to 1605632 tid 2 2024-09-16T14:49:01.886 INFO:tasks.rados.rados.0.smithi040.stdout:208: writing smithi04040151-27 from 1605632 to 1933312 tid 3 2024-09-16T14:49:01.886 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 19 current snap is 14 2024-09-16T14:49:01.886 INFO:tasks.rados.rados.0.smithi040.stdout:207: finishing write tid 1 to smithi04040151-26 2024-09-16T14:49:01.886 INFO:tasks.rados.rados.0.smithi040.stdout:209: seq_num 89 ranges {1425408=737280,2162688=573440,2736128=196608} 2024-09-16T14:49:01.898 INFO:tasks.rados.rados.0.smithi040.stdout:209: writing smithi04040151-19 from 1425408 to 2162688 tid 1 2024-09-16T14:49:01.901 INFO:tasks.rados.rados.0.smithi040.stdout:209: writing smithi04040151-19 from 2162688 to 2736128 tid 2 2024-09-16T14:49:01.903 INFO:tasks.rados.rados.0.smithi040.stdout:209: writing smithi04040151-19 from 2736128 to 2932736 tid 3 2024-09-16T14:49:01.903 INFO:tasks.rados.rados.0.smithi040.stdout:210: snap_create 2024-09-16T14:49:01.904 INFO:tasks.rados.rados.0.smithi040.stdout:207: finishing write tid 2 to smithi04040151-26 2024-09-16T14:49:01.904 INFO:tasks.rados.rados.0.smithi040.stdout:207: finishing write tid 3 to smithi04040151-26 2024-09-16T14:49:01.904 INFO:tasks.rados.rados.0.smithi040.stdout:207: finishing write tid 4 to smithi04040151-26 2024-09-16T14:49:01.904 INFO:tasks.rados.rados.0.smithi040.stdout:207: finishing write tid 5 to smithi04040151-26 2024-09-16T14:49:01.904 INFO:tasks.rados.rados.0.smithi040.stdout:207: oid 26 updating version 0 to 68 2024-09-16T14:49:01.904 INFO:tasks.rados.rados.0.smithi040.stdout:207: oid 26 updating version 68 to 69 2024-09-16T14:49:01.904 INFO:tasks.rados.rados.0.smithi040.stdout:207: oid 26 version 69 is already newer than 67 2024-09-16T14:49:01.904 INFO:tasks.rados.rados.0.smithi040.stdout:207: oid 26 version 69 is already newer than 66 2024-09-16T14:49:01.904 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 26 v 69 (ObjNum 87 snap 14 seq_num 87) dirty exists 2024-09-16T14:49:01.904 INFO:tasks.rados.rados.0.smithi040.stdout:207: left oid 26 (ObjNum 87 snap 14 seq_num 87) 2024-09-16T14:49:01.904 INFO:tasks.rados.rados.0.smithi040.stdout:208: finishing write tid 1 to smithi04040151-27 2024-09-16T14:49:01.904 INFO:tasks.rados.rados.0.smithi040.stdout:208: finishing write tid 2 to smithi04040151-27 2024-09-16T14:49:01.904 INFO:tasks.rados.rados.0.smithi040.stdout:208: finishing write tid 3 to smithi04040151-27 2024-09-16T14:49:01.905 INFO:tasks.rados.rados.0.smithi040.stdout:208: finishing write tid 4 to smithi04040151-27 2024-09-16T14:49:01.905 INFO:tasks.rados.rados.0.smithi040.stdout:208: finishing write tid 5 to smithi04040151-27 2024-09-16T14:49:01.905 INFO:tasks.rados.rados.0.smithi040.stdout:208: oid 27 updating version 0 to 26 2024-09-16T14:49:01.905 INFO:tasks.rados.rados.0.smithi040.stdout:208: oid 27 version 26 is already newer than 25 2024-09-16T14:49:01.905 INFO:tasks.rados.rados.0.smithi040.stdout:208: oid 27 updating version 26 to 28 2024-09-16T14:49:01.905 INFO:tasks.rados.rados.0.smithi040.stdout:208: oid 27 version 28 is already newer than 27 2024-09-16T14:49:01.905 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 27 v 28 (ObjNum 88 snap 14 seq_num 88) dirty exists 2024-09-16T14:49:01.905 INFO:tasks.rados.rados.0.smithi040.stdout:208: left oid 27 (ObjNum 88 snap 14 seq_num 88) 2024-09-16T14:49:01.905 INFO:tasks.rados.rados.0.smithi040.stdout:209: finishing write tid 1 to smithi04040151-19 2024-09-16T14:49:01.907 INFO:tasks.rados.rados.0.smithi040.stdout:209: finishing write tid 2 to smithi04040151-19 2024-09-16T14:49:01.909 INFO:tasks.rados.rados.0.smithi040.stdout:209: finishing write tid 3 to smithi04040151-19 2024-09-16T14:49:01.909 INFO:tasks.rados.rados.0.smithi040.stdout:209: finishing write tid 4 to smithi04040151-19 2024-09-16T14:49:01.912 INFO:tasks.rados.rados.0.smithi040.stdout:209: finishing write tid 5 to smithi04040151-19 2024-09-16T14:49:01.912 INFO:tasks.rados.rados.0.smithi040.stdout:209: oid 19 updating version 0 to 42 2024-09-16T14:49:01.913 INFO:tasks.rados.rados.0.smithi040.stdout:209: oid 19 version 42 is already newer than 40 2024-09-16T14:49:01.913 INFO:tasks.rados.rados.0.smithi040.stdout:209: oid 19 version 42 is already newer than 39 2024-09-16T14:49:01.913 INFO:tasks.rados.rados.0.smithi040.stdout:209: oid 19 version 42 is already newer than 41 2024-09-16T14:49:01.913 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 19 v 42 (ObjNum 89 snap 14 seq_num 89) dirty exists 2024-09-16T14:49:01.913 INFO:tasks.rados.rados.0.smithi040.stdout:209: left oid 19 (ObjNum 89 snap 14 seq_num 89) 2024-09-16T14:49:01.955 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:01.954+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:01.976 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:01.977 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:01.985 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:49:02.043 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:49:02.069 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:02.068+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:02.133 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:02.142 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:49:02.157 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:02.157+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:02.192 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:02.192 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:02.209 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:02.209 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:02.225 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:02.225 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:02.243 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:02.243 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:02.254 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:02.254 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:02.254 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:02.254 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:02.254 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.255 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:303 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:02.255 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:02.255 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 17.319431303999998, 2024-09-16T14:49:02.255 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014763235, 2024-09-16T14:49:02.255 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.255 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:02.255 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.255 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.255 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.255 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 303 2024-09-16T14:49:02.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.255 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.255 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.255 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.256 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:02.256 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.256 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.256 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.256 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.256 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:02.256 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.256 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.256 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.256 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.256 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932467+0000", 2024-09-16T14:49:02.257 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:02.257 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.257 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.257 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.257 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932474+0000", 2024-09-16T14:49:02.257 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3189999999999997e-06 2024-09-16T14:49:02.257 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.257 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.257 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.257 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932490+0000", 2024-09-16T14:49:02.257 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:49:02.257 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.257 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.258 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.258 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932498+0000", 2024-09-16T14:49:02.258 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7789999999999996e-06 2024-09-16T14:49:02.258 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.258 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.258 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.258 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932544+0000", 2024-09-16T14:49:02.258 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6486000000000002e-05 2024-09-16T14:49:02.258 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.259 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.259 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:02.259 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932550+0000", 2024-09-16T14:49:02.259 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.3149999999999999e-06 2024-09-16T14:49:02.259 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.259 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.259 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.259 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.939655+0000", 2024-09-16T14:49:02.259 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071050039999999998 2024-09-16T14:49:02.259 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.259 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.260 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.260 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.939762+0000", 2024-09-16T14:49:02.260 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000106885 2024-09-16T14:49:02.260 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.260 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.260 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.260 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.947232+0000", 2024-09-16T14:49:02.260 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074701619999999998 2024-09-16T14:49:02.260 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.260 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.260 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.260 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.260 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.261 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:304 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:02.261 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:02.261 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 17.319393562999998, 2024-09-16T14:49:02.261 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015391726, 2024-09-16T14:49:02.261 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.261 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:02.261 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.261 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.261 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.261 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 304 2024-09-16T14:49:02.261 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.261 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.261 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.262 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.262 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:02.262 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.262 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.262 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.262 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.262 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:02.262 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.262 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.262 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.263 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.263 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:02.263 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.263 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.263 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.263 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.263 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932510+0000", 2024-09-16T14:49:02.263 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.1069999999999998e-06 2024-09-16T14:49:02.263 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.263 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.263 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.263 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932516+0000", 2024-09-16T14:49:02.263 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4699999999999999e-06 2024-09-16T14:49:02.263 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.264 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.264 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.264 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932522+0000", 2024-09-16T14:49:02.264 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4750000000000001e-06 2024-09-16T14:49:02.264 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.264 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.264 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.265 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932610+0000", 2024-09-16T14:49:02.265 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8623999999999997e-05 2024-09-16T14:49:02.265 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.265 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.265 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:02.265 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932616+0000", 2024-09-16T14:49:02.265 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.9649999999999996e-06 2024-09-16T14:49:02.265 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.265 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.265 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.265 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940205+0000", 2024-09-16T14:49:02.266 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0075882570000000002 2024-09-16T14:49:02.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.266 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.266 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.266 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940285+0000", 2024-09-16T14:49:02.266 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0783000000000002e-05 2024-09-16T14:49:02.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.266 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.266 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.266 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.947898+0000", 2024-09-16T14:49:02.266 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0076124820000000003 2024-09-16T14:49:02.266 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.266 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.266 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.267 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.267 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:305 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:02.267 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:02.267 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 17.319312773, 2024-09-16T14:49:02.267 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015660833999999998, 2024-09-16T14:49:02.267 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.267 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:02.267 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.267 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.267 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.267 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 305 2024-09-16T14:49:02.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.267 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.268 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.268 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.268 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:02.268 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.268 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.268 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.268 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.268 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:02.268 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.268 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.268 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.268 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.268 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932586+0000", 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932591+0000", 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2470000000000002e-06 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932602+0000", 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1396000000000001e-05 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.269 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932608+0000", 2024-09-16T14:49:02.270 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.7169999999999996e-06 2024-09-16T14:49:02.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.270 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.270 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.270 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932646+0000", 2024-09-16T14:49:02.270 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.7978999999999997e-05 2024-09-16T14:49:02.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.270 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.270 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:02.270 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932652+0000", 2024-09-16T14:49:02.271 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0310000000000004e-06 2024-09-16T14:49:02.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.271 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.271 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.271 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941247+0000", 2024-09-16T14:49:02.271 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085954180000000005 2024-09-16T14:49:02.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.271 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.271 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.271 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941327+0000", 2024-09-16T14:49:02.271 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9354e-05 2024-09-16T14:49:02.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.272 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.272 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.272 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.948248+0000", 2024-09-16T14:49:02.272 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0069212400000000004 2024-09-16T14:49:02.272 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.272 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.272 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.272 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.272 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:362 3.2s0 3:4ee78709:::smithi04040151-40:head [copy-from ver 6] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T14:49:02.273 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:47.938030+0000", 2024-09-16T14:49:02.273 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.313869813, 2024-09-16T14:49:02.273 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0092047480000000004, 2024-09-16T14:49:02.273 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.273 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:02.273 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.273 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.274 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.274 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 362 2024-09-16T14:49:02.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.274 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.274 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.274 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.274 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938030+0000", 2024-09-16T14:49:02.274 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.275 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.275 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.275 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938030+0000", 2024-09-16T14:49:02.275 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.275 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.275 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.275 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938025+0000", 2024-09-16T14:49:02.275 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999952 2024-09-16T14:49:02.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.275 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.275 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.275 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938040+0000", 2024-09-16T14:49:02.276 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4786e-05 2024-09-16T14:49:02.276 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.276 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.276 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.276 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938062+0000", 2024-09-16T14:49:02.276 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1725e-05 2024-09-16T14:49:02.276 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.276 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.276 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.276 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938072+0000", 2024-09-16T14:49:02.276 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0516000000000001e-05 2024-09-16T14:49:02.276 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.277 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.277 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.277 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938126+0000", 2024-09-16T14:49:02.277 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.3168000000000003e-05 2024-09-16T14:49:02.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.277 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.277 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:02.277 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938131+0000", 2024-09-16T14:49:02.277 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:49:02.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.277 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.277 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.277 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.939892+0000", 2024-09-16T14:49:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001761295 2024-09-16T14:49:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.939971+0000", 2024-09-16T14:49:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8417999999999994e-05 2024-09-16T14:49:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.944590+0000", 2024-09-16T14:49:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0046191219999999998 2024-09-16T14:49:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.278 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.945941+0000", 2024-09-16T14:49:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0013506029999999999 2024-09-16T14:49:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.947193+0000", 2024-09-16T14:49:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001252555 2024-09-16T14:49:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.947235+0000", 2024-09-16T14:49:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.1569999999999997e-05 2024-09-16T14:49:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.279 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.280 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.280 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.280 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:375 3.7s0 3:e91f0d6c:::smithi04040151-31:head [append 0~0] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:02.280 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.960177+0000", 2024-09-16T14:49:02.280 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 13.291722917, 2024-09-16T14:49:02.280 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.009691926, 2024-09-16T14:49:02.280 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.280 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:02.280 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.280 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.280 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.280 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 375 2024-09-16T14:49:02.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.281 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.281 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.281 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960177+0000", 2024-09-16T14:49:02.281 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.281 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.281 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960177+0000", 2024-09-16T14:49:02.281 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.281 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.281 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960176+0000", 2024-09-16T14:49:02.282 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.282 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.282 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960180+0000", 2024-09-16T14:49:02.282 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.4640000000000002e-06 2024-09-16T14:49:02.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.282 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.282 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960524+0000", 2024-09-16T14:49:02.282 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034462499999999999 2024-09-16T14:49:02.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.282 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.283 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960535+0000", 2024-09-16T14:49:02.283 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0971e-05 2024-09-16T14:49:02.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.283 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.283 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.963989+0000", 2024-09-16T14:49:02.283 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0034539449999999999 2024-09-16T14:49:02.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.283 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.283 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.964073+0000", 2024-09-16T14:49:02.283 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.3725000000000002e-05 2024-09-16T14:49:02.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.284 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.284 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.964387+0000", 2024-09-16T14:49:02.284 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00031368200000000002 2024-09-16T14:49:02.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.284 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.284 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.284 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.969143+0000", 2024-09-16T14:49:02.284 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0047564260000000002 2024-09-16T14:49:02.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.284 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.284 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:02.284 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.969786+0000", 2024-09-16T14:49:02.284 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00064274600000000003 2024-09-16T14:49:02.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.285 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.285 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.285 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.969869+0000", 2024-09-16T14:49:02.285 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2804000000000002e-05 2024-09-16T14:49:02.285 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.285 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.285 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.285 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.285 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.285 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:403 3.2s0 3:421e4b92:::smithi04040151-11:head [copy-from ver 2] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:02.285 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:02.285 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.3683348760000005, 2024-09-16T14:49:02.285 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015904270000000002, 2024-09-16T14:49:02.285 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.286 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:02.286 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.286 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.286 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.286 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 403 2024-09-16T14:49:02.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.286 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.286 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.286 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.286 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:02.286 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.286 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.286 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.287 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:02.287 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.287 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.287 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.287 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883563+0000", 2024-09-16T14:49:02.287 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:02.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.287 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.287 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.287 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883573+0000", 2024-09-16T14:49:02.288 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0583e-05 2024-09-16T14:49:02.288 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.288 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.288 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.288 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883593+0000", 2024-09-16T14:49:02.288 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0052e-05 2024-09-16T14:49:02.288 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.288 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.288 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.288 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883603+0000", 2024-09-16T14:49:02.288 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4409999999999999e-06 2024-09-16T14:49:02.288 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.288 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.289 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.289 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883667+0000", 2024-09-16T14:49:02.289 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4671000000000005e-05 2024-09-16T14:49:02.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.289 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.289 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883775+0000", 2024-09-16T14:49:02.289 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000108068 2024-09-16T14:49:02.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.289 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.289 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.896809+0000", 2024-09-16T14:49:02.289 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01303355 2024-09-16T14:49:02.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.290 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.290 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.290 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897717+0000", 2024-09-16T14:49:02.290 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00090801599999999999 2024-09-16T14:49:02.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.290 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.290 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:02.290 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.899408+0000", 2024-09-16T14:49:02.290 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0016911459999999999 2024-09-16T14:49:02.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.290 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.290 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.290 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.899469+0000", 2024-09-16T14:49:02.290 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0968999999999999e-05 2024-09-16T14:49:02.291 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.291 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.291 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.291 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.291 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:404 3.2s0 3:421e4b92:::smithi04040151-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:02.291 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:02.291 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.3682815789999996, 2024-09-16T14:49:02.291 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0139831, 2024-09-16T14:49:02.291 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.291 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:02.291 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.291 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.291 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.292 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 404 2024-09-16T14:49:02.292 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.292 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.292 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.292 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.292 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:02.292 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.292 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.292 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.292 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.292 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:02.292 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.292 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.293 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.293 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.293 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883617+0000", 2024-09-16T14:49:02.293 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:02.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.293 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.293 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.293 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883622+0000", 2024-09-16T14:49:02.293 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7489999999999998e-06 2024-09-16T14:49:02.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.293 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.293 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.293 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883631+0000", 2024-09-16T14:49:02.293 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6489999999999997e-06 2024-09-16T14:49:02.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.294 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.294 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883639+0000", 2024-09-16T14:49:02.294 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2120000000000003e-06 2024-09-16T14:49:02.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.294 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.294 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883930+0000", 2024-09-16T14:49:02.294 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029138200000000002 2024-09-16T14:49:02.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.294 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883961+0000", 2024-09-16T14:49:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0190999999999999e-05 2024-09-16T14:49:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897411+0000", 2024-09-16T14:49:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013450037999999999 2024-09-16T14:49:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897506+0000", 2024-09-16T14:49:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5785000000000005e-05 2024-09-16T14:49:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.296 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.296 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897601+0000", 2024-09-16T14:49:02.296 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4868e-05 2024-09-16T14:49:02.296 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.296 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.296 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.296 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:460 3.2s0 3:421e4b92:::smithi04040151-11:head [append 0~0] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:02.296 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.993559+0000", 2024-09-16T14:49:02.296 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.2583402679999995, 2024-09-16T14:49:02.296 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0095830099999999994, 2024-09-16T14:49:02.296 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.296 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:02.297 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.297 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.297 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.297 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 460 2024-09-16T14:49:02.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.297 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.297 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.297 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993559+0000", 2024-09-16T14:49:02.297 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.297 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993559+0000", 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993557+0000", 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993569+0000", 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1644999999999999e-05 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.298 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.299 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993588+0000", 2024-09-16T14:49:02.299 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8793000000000001e-05 2024-09-16T14:49:02.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.299 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.299 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993597+0000", 2024-09-16T14:49:02.299 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3710000000000006e-06 2024-09-16T14:49:02.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.299 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.299 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993650+0000", 2024-09-16T14:49:02.299 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.3001999999999998e-05 2024-09-16T14:49:02.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.300 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.300 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993768+0000", 2024-09-16T14:49:02.300 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000117947 2024-09-16T14:49:02.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.300 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.300 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994233+0000", 2024-09-16T14:49:02.300 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00046465800000000001 2024-09-16T14:49:02.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.300 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.300 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997829+0000", 2024-09-16T14:49:02.300 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0035960779999999999 2024-09-16T14:49:02.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.301 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:02.301 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003062+0000", 2024-09-16T14:49:02.301 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0052332100000000003 2024-09-16T14:49:02.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.301 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.301 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003142+0000", 2024-09-16T14:49:02.301 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0585000000000002e-05 2024-09-16T14:49:02.301 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.301 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.301 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.302 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:461 3.2s0 3:421e4b92:::smithi04040151-11:head [rollback 10] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:02.302 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:02.302 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.2582915359999998, 2024-09-16T14:49:02.302 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011419016000000001, 2024-09-16T14:49:02.302 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.302 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:02.302 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.302 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.302 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.302 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 461 2024-09-16T14:49:02.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.302 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993611+0000", 2024-09-16T14:49:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6440000000000002e-06 2024-09-16T14:49:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993619+0000", 2024-09-16T14:49:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.199e-06 2024-09-16T14:49:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.305 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.305 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993624+0000", 2024-09-16T14:49:02.305 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.8790000000000002e-06 2024-09-16T14:49:02.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.305 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.305 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.305 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994734+0000", 2024-09-16T14:49:02.305 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0011103339999999999 2024-09-16T14:49:02.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.305 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.306 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.306 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994830+0000", 2024-09-16T14:49:02.306 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6125000000000005e-05 2024-09-16T14:49:02.306 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.306 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.306 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.306 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995335+0000", 2024-09-16T14:49:02.306 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00050461799999999997 2024-09-16T14:49:02.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.307 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.307 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.001584+0000", 2024-09-16T14:49:02.307 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0062489609999999999 2024-09-16T14:49:02.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.307 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:02.307 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004966+0000", 2024-09-16T14:49:02.307 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.003382674 2024-09-16T14:49:02.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.308 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.308 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.308 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.005027+0000", 2024-09-16T14:49:02.308 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0974000000000003e-05 2024-09-16T14:49:02.308 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.308 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.308 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.308 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.308 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:462 3.bs0 3:d9d8585e:::smithi04040151-8:head [append 0~0] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:02.308 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.993639+0000", 2024-09-16T14:49:02.308 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.2582607000000001, 2024-09-16T14:49:02.309 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010145506, 2024-09-16T14:49:02.309 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.309 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:02.309 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.309 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.309 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.309 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 462 2024-09-16T14:49:02.309 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.309 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.309 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.309 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.309 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993639+0000", 2024-09-16T14:49:02.310 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.310 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.310 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.310 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993639+0000", 2024-09-16T14:49:02.310 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.310 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.310 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.310 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993639+0000", 2024-09-16T14:49:02.311 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.311 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.311 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993641+0000", 2024-09-16T14:49:02.311 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4789999999999999e-06 2024-09-16T14:49:02.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.311 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.311 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993651+0000", 2024-09-16T14:49:02.311 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4059999999999994e-06 2024-09-16T14:49:02.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.311 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.312 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993655+0000", 2024-09-16T14:49:02.312 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.5369999999999999e-06 2024-09-16T14:49:02.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.312 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.312 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993720+0000", 2024-09-16T14:49:02.312 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5048000000000005e-05 2024-09-16T14:49:02.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.312 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.312 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993816+0000", 2024-09-16T14:49:02.312 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5568000000000004e-05 2024-09-16T14:49:02.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.313 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.313 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994512+0000", 2024-09-16T14:49:02.313 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00069592099999999995 2024-09-16T14:49:02.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.313 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.313 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.000585+0000", 2024-09-16T14:49:02.313 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060736160000000004 2024-09-16T14:49:02.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.313 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:02.313 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003712+0000", 2024-09-16T14:49:02.314 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0031268239999999998 2024-09-16T14:49:02.314 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.314 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.314 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.314 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003785+0000", 2024-09-16T14:49:02.314 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2495000000000003e-05 2024-09-16T14:49:02.314 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.314 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.314 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.314 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.314 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.314 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:463 3.bs0 3:d9d8585e:::smithi04040151-8:head [rollback 10] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:02.314 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.993666+0000", 2024-09-16T14:49:02.315 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.2582335459999996, 2024-09-16T14:49:02.315 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010553527, 2024-09-16T14:49:02.315 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.315 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:02.315 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.315 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.315 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.315 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 463 2024-09-16T14:49:02.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.315 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.316 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.316 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993666+0000", 2024-09-16T14:49:02.316 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.316 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.316 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.317 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993666+0000", 2024-09-16T14:49:02.317 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.317 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.317 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993666+0000", 2024-09-16T14:49:02.317 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.318 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.318 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993668+0000", 2024-09-16T14:49:02.318 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5629999999999999e-06 2024-09-16T14:49:02.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.319 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.319 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993675+0000", 2024-09-16T14:49:02.319 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.139e-06 2024-09-16T14:49:02.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.319 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.319 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993688+0000", 2024-09-16T14:49:02.319 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3237e-05 2024-09-16T14:49:02.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.319 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.319 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995227+0000", 2024-09-16T14:49:02.320 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0015395949999999999 2024-09-16T14:49:02.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.320 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.320 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995317+0000", 2024-09-16T14:49:02.320 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0026999999999997e-05 2024-09-16T14:49:02.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.320 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.320 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995832+0000", 2024-09-16T14:49:02.320 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00051462399999999998 2024-09-16T14:49:02.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.321 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.321 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002431+0000", 2024-09-16T14:49:02.321 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0065994540000000003 2024-09-16T14:49:02.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.321 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.321 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:02.321 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004155+0000", 2024-09-16T14:49:02.321 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001723961 2024-09-16T14:49:02.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.321 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.321 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.321 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004220+0000", 2024-09-16T14:49:02.321 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4319999999999994e-05 2024-09-16T14:49:02.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.322 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.325 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:464 3.bs0 3:d9d8585e:::smithi04040151-8:head [append 0~0] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.993720+0000", 2024-09-16T14:49:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.2581797180000001, 2024-09-16T14:49:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010696298, 2024-09-16T14:49:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 464 2024-09-16T14:49:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993720+0000", 2024-09-16T14:49:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993720+0000", 2024-09-16T14:49:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993720+0000", 2024-09-16T14:49:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993722+0000", 2024-09-16T14:49:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:49:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993730+0000", 2024-09-16T14:49:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3669999999999999e-06 2024-09-16T14:49:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993737+0000", 2024-09-16T14:49:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.985e-06 2024-09-16T14:49:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997488+0000", 2024-09-16T14:49:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0037506879999999999 2024-09-16T14:49:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997578+0000", 2024-09-16T14:49:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0210000000000005e-05 2024-09-16T14:49:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.998012+0000", 2024-09-16T14:49:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000434201 2024-09-16T14:49:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002434+0000", 2024-09-16T14:49:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044225080000000003 2024-09-16T14:49:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004356+0000", 2024-09-16T14:49:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0019215720000000001 2024-09-16T14:49:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004416+0000", 2024-09-16T14:49:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0316999999999999e-05 2024-09-16T14:49:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:465 3.7s0 3:e91f0d6c:::smithi04040151-31:head [create,setxattr bk]W^gTaniZ\\TV]dqj_UT\\QgY_\\pYYUOcYdSe`m_ (1859) in=40b,setxattr XSPcap`qmQ\\ZXYl^_bQYNltYZfOtfbRjqmVk_mOa (419) in=40b,setxattr l_Qd^aXhnapg`O][bWecsVdjlWkdYc^tOnjZh]^m (1577) in=40b,setxattr facegtqpaQroppsVbsrZelUak]RbamP`srOTdms[ (350) in=40b,setxattr QllprTts[liem]moe]Pi`nNtcQko (1402) in=28b,setxattr sjm^Un[ahPriW\\TNkaYS\\NnPZfZ\\Zmq[rQ^TWYZm (1892) in=40b,setxattr kRdb__OpRdYV^sYmYmYdhes^Vbf]QaV`SVZkhRcY (209) in=40b,setxattr b[QTnoQc[_Yc^RmdY[ (1402) in=18b,setxattr ^iZkhfc`oO (1402) in=10b,setxattr d\\QUOkoScOhonabPS_YeNlheOOeOal]aQ^ZW]lc` (2805) in=40b,setxattr SioYcliigTgheX\\jnVqipcelfYhbZRjQRet]mQQU (1365) in=40b,setxattr jWcgaRWbr\\ZQ_XXr_eenPa]sXtbaPlmeshkgjOsm (1192) in=40b,setxattr WmVtOY (1402) in=6b,setxattr ]tk_t\\gnSfccZfYfnWnaqP (1402) in=22b,setxattr dS_dYomnsYPtQarmqoZQkO]Ofk_SfgjgitphTZPg (18524) in=40b,setxattr g_a^Zpt`YVahZiOW]V[jQ\\QccqSTo_ma[\\c\\[aft (454) in=40b,setxattr qmmdmgOR__hN\\UTjpR_gbY^s_pOVSdbTob`k (1402) in=36b,setxattr cW`PPOYrbfXV^hmaWhVNb`qVSXqjcdej`pY][]l^ (665) in=40b,setxattr jjo^VgcaQffrnVNPlnq_iTmcrlnTOenhaZV[_SZf (453) in=40b,setxattr n^nqj]bnVcl\\njtUkprb\\laSh^Rbmam_Z_NkeXqj (595) in=40b] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.995572+0000", 2024-09-16T14:49:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.2563274120000001, 2024-09-16T14:49:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0092652180000000004, 2024-09-16T14:49:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 465 2024-09-16T14:49:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995572+0000", 2024-09-16T14:49:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995572+0000", 2024-09-16T14:49:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995571+0000", 2024-09-16T14:49:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995588+0000", 2024-09-16T14:49:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7146000000000002e-05 2024-09-16T14:49:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995599+0000", 2024-09-16T14:49:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0489000000000001e-05 2024-09-16T14:49:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995605+0000", 2024-09-16T14:49:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6189999999999996e-06 2024-09-16T14:49:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995654+0000", 2024-09-16T14:49:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.8857999999999999e-05 2024-09-16T14:49:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995779+0000", 2024-09-16T14:49:02.339 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012446000000000001 2024-09-16T14:49:02.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.339 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.339 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.996775+0000", 2024-09-16T14:49:02.339 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00099642999999999997 2024-09-16T14:49:02.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002430+0000", 2024-09-16T14:49:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056549249999999999 2024-09-16T14:49:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004762+0000", 2024-09-16T14:49:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0023316589999999998 2024-09-16T14:49:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.340 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.341 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004838+0000", 2024-09-16T14:49:02.341 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5910999999999999e-05 2024-09-16T14:49:02.341 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.341 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.341 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:469 3.2s0 3:4ee78709:::smithi04040151-40:head [read 0~1474560,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.995831+0000", 2024-09-16T14:49:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.2560682630000004, 2024-09-16T14:49:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090063799999999996, 2024-09-16T14:49:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 469 2024-09-16T14:49:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.342 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995831+0000", 2024-09-16T14:49:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995831+0000", 2024-09-16T14:49:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.343 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995831+0000", 2024-09-16T14:49:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995836+0000", 2024-09-16T14:49:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9439999999999996e-06 2024-09-16T14:49:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.344 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995845+0000", 2024-09-16T14:49:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5189999999999998e-06 2024-09-16T14:49:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995852+0000", 2024-09-16T14:49:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6370000000000002e-06 2024-09-16T14:49:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995970+0000", 2024-09-16T14:49:02.345 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011843100000000001 2024-09-16T14:49:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.996056+0000", 2024-09-16T14:49:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5445999999999995e-05 2024-09-16T14:49:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004838+0000", 2024-09-16T14:49:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0087821040000000006 2024-09-16T14:49:02.346 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:470 3.2s0 3:4ee78709:::smithi04040151-40:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.2560370719999998, 2024-09-16T14:49:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011056359999999999, 2024-09-16T14:49:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.347 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 470 2024-09-16T14:49:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.348 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995862+0000", 2024-09-16T14:49:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995865+0000", 2024-09-16T14:49:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2440000000000002e-06 2024-09-16T14:49:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995872+0000", 2024-09-16T14:49:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5640000000000002e-06 2024-09-16T14:49:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995877+0000", 2024-09-16T14:49:02.350 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1680000000000003e-06 2024-09-16T14:49:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.996768+0000", 2024-09-16T14:49:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00089129699999999999 2024-09-16T14:49:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.996867+0000", 2024-09-16T14:49:02.351 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8259999999999998e-05 2024-09-16T14:49:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.006919+0000", 2024-09-16T14:49:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010052284 2024-09-16T14:49:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:471 3.2s0 3:4ee78709:::smithi04040151-40:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:02.352 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.2560183460000003, 2024-09-16T14:49:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012013398999999999, 2024-09-16T14:49:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 471 2024-09-16T14:49:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:02.354 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995884+0000", 2024-09-16T14:49:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:49:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995891+0000", 2024-09-16T14:49:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.72e-06 2024-09-16T14:49:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.355 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995896+0000", 2024-09-16T14:49:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1889999999999998e-06 2024-09-16T14:49:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997239+0000", 2024-09-16T14:49:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0013430460000000001 2024-09-16T14:49:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.356 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:02.357 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997352+0000", 2024-09-16T14:49:02.357 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011287500000000001 2024-09-16T14:49:02.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.357 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.357 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007895+0000", 2024-09-16T14:49:02.358 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010543113999999999 2024-09-16T14:49:02.358 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.358 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.358 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.358 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s4 52/27 ECSubWrite(tid=183, reqid=osd.11.0:183, at_version=52'60, trim_to=0'0, roll_forward_to=51'56))", 2024-09-16T14:49:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 3.3646294750000001, 2024-09-16T14:49:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011901715, 2024-09-16T14:49:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.359 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887292+0000", 2024-09-16T14:49:02.360 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1920999999999999e-05 2024-09-16T14:49:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887303+0000", 2024-09-16T14:49:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1472e-05 2024-09-16T14:49:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887311+0000", 2024-09-16T14:49:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.361 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.362 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887360+0000", 2024-09-16T14:49:02.362 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9002999999999998e-05 2024-09-16T14:49:02.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.362 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.362 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.362 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887369+0000", 2024-09-16T14:49:02.362 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9299999999999992e-06 2024-09-16T14:49:02.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.362 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.899136+0000", 2024-09-16T14:49:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011767123000000001 2024-09-16T14:49:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.899172+0000", 2024-09-16T14:49:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6161000000000002e-05 2024-09-16T14:49:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.363 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es6 52/27 ECSubWrite(tid=182, reqid=osd.11.0:182, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887409+0000", 2024-09-16T14:49:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 3.3644907879999999, 2024-09-16T14:49:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010651061999999999, 2024-09-16T14:49:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.364 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887409+0000", 2024-09-16T14:49:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887409+0000", 2024-09-16T14:49:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.365 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887408+0000", 2024-09-16T14:49:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887448+0000", 2024-09-16T14:49:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.998e-05 2024-09-16T14:49:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887460+0000", 2024-09-16T14:49:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1931e-05 2024-09-16T14:49:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.366 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887468+0000", 2024-09-16T14:49:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1240000000000005e-06 2024-09-16T14:49:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887710+0000", 2024-09-16T14:49:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000241272 2024-09-16T14:49:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887722+0000", 2024-09-16T14:49:02.367 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2515e-05 2024-09-16T14:49:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.898028+0000", 2024-09-16T14:49:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010305365 2024-09-16T14:49:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.898060+0000", 2024-09-16T14:49:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2391999999999998e-05 2024-09-16T14:49:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds5 52/27 ECSubWrite(tid=214, reqid=osd.7.0:214, at_version=52'35, trim_to=0'0, roll_forward_to=51'31))", 2024-09-16T14:49:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 3.3638503110000002, 2024-09-16T14:49:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010846751, 2024-09-16T14:49:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.369 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888072+0000", 2024-09-16T14:49:02.370 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3008000000000001e-05 2024-09-16T14:49:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888082+0000", 2024-09-16T14:49:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0129000000000001e-05 2024-09-16T14:49:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888090+0000", 2024-09-16T14:49:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7510000000000008e-06 2024-09-16T14:49:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.371 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889906+0000", 2024-09-16T14:49:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001816603 2024-09-16T14:49:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889931+0000", 2024-09-16T14:49:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4165e-05 2024-09-16T14:49:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.898870+0000", 2024-09-16T14:49:02.372 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089394299999999999 2024-09-16T14:49:02.373 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.373 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.373 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.373 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.898896+0000", 2024-09-16T14:49:02.373 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6188999999999999e-05 2024-09-16T14:49:02.373 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.373 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.373 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s3 52/27 ECSubWrite(tid=29, reqid=osd.4.0:29, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.889955+0000", 2024-09-16T14:49:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 3.361944346, 2024-09-16T14:49:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0095517620000000001, 2024-09-16T14:49:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.374 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889955+0000", 2024-09-16T14:49:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889955+0000", 2024-09-16T14:49:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889955+0000", 2024-09-16T14:49:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.375 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889980+0000", 2024-09-16T14:49:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4765000000000001e-05 2024-09-16T14:49:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.891022+0000", 2024-09-16T14:49:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0010423800000000001 2024-09-16T14:49:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.376 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.891037+0000", 2024-09-16T14:49:02.377 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4501000000000001e-05 2024-09-16T14:49:02.377 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.377 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.377 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:02.377 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.891261+0000", 2024-09-16T14:49:02.377 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00022435599999999999 2024-09-16T14:49:02.377 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.377 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.377 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.891270+0000", 2024-09-16T14:49:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:49:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.899486+0000", 2024-09-16T14:49:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0082159580000000006 2024-09-16T14:49:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:02.378 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.899507+0000", 2024-09-16T14:49:02.379 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1311e-05 2024-09-16T14:49:02.380 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.380 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.380 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.380 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:02.380 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:02.380 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:02.381 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:02.382 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:02.383 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:49:02.383 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:02.383 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:02.384 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:02.384 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:02.384 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:02.385 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:02.385 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:02.385 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:02.386 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:02.386 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:02.386 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:02.386 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:02.387 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:02.387 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:02.380+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:02.482 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:02.481+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:02.500 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:02.500 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:02.500 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:02.500 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:02.509 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:49:02.583 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:02.581+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:02.611 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:02.611 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:02.611 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:02.612 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:02.612 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:02.620 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:49:02.683 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:02.682+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:02.724 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:02.724 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:02.725 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:02.725 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:02.725 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.725 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/27 ECSubWrite(tid=76, reqid=client.4500.0:227, at_version=33'16, trim_to=0'0, roll_forward_to=30'10))", 2024-09-16T14:49:02.725 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:39.917588+0000", 2024-09-16T14:49:02.725 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 22.804044741999999, 2024-09-16T14:49:02.726 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049844299999999998, 2024-09-16T14:49:02.726 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.726 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.726 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.726 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.727 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.727 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917588+0000", 2024-09-16T14:49:02.727 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.727 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.727 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.727 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.728 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917588+0000", 2024-09-16T14:49:02.728 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.728 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.728 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.728 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.728 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917588+0000", 2024-09-16T14:49:02.729 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.729 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.729 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.729 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.729 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917606+0000", 2024-09-16T14:49:02.729 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8737999999999999e-05 2024-09-16T14:49:02.729 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.730 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.732 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.732 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917616+0000", 2024-09-16T14:49:02.732 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6679999999999996e-06 2024-09-16T14:49:02.733 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.733 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.733 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.733 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917621+0000", 2024-09-16T14:49:02.733 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2820000000000004e-06 2024-09-16T14:49:02.734 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.734 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.734 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.734 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917655+0000", 2024-09-16T14:49:02.734 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.4029000000000002e-05 2024-09-16T14:49:02.734 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.735 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.735 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.735 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917665+0000", 2024-09-16T14:49:02.735 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3889999999999994e-06 2024-09-16T14:49:02.735 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.735 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.736 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.736 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.922545+0000", 2024-09-16T14:49:02.736 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048804490000000002 2024-09-16T14:49:02.736 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.736 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.737 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.737 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.922573+0000", 2024-09-16T14:49:02.737 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7424e-05 2024-09-16T14:49:02.737 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.737 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.737 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.738 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.738 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.738 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s3 37/27 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=37'32, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:02.738 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.920644+0000", 2024-09-16T14:49:02.738 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.800988930999999, 2024-09-16T14:49:02.738 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0056995509999999997, 2024-09-16T14:49:02.739 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.739 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.739 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.739 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.739 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.740 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920644+0000", 2024-09-16T14:49:02.740 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.740 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.740 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.740 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.740 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920644+0000", 2024-09-16T14:49:02.741 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.741 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.741 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.741 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.741 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920644+0000", 2024-09-16T14:49:02.741 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:02.741 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.742 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.742 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.743 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920668+0000", 2024-09-16T14:49:02.743 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3832000000000002e-05 2024-09-16T14:49:02.743 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.743 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.743 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.743 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920680+0000", 2024-09-16T14:49:02.743 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2021999999999999e-05 2024-09-16T14:49:02.743 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.743 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.744 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.744 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920689+0000", 2024-09-16T14:49:02.744 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2760000000000006e-06 2024-09-16T14:49:02.744 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.744 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.744 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.744 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920734+0000", 2024-09-16T14:49:02.744 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4943e-05 2024-09-16T14:49:02.744 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.744 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.744 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.744 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920744+0000", 2024-09-16T14:49:02.744 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9480000000000003e-06 2024-09-16T14:49:02.745 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.745 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.745 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.745 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.926318+0000", 2024-09-16T14:49:02.745 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0055746700000000003 2024-09-16T14:49:02.745 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.745 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.745 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.745 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.926344+0000", 2024-09-16T14:49:02.745 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5072e-05 2024-09-16T14:49:02.745 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.745 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.745 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.746 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.746 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.746 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs2 37/27 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:02.746 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:02.746 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.800929453999998, 2024-09-16T14:49:02.746 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010231874, 2024-09-16T14:49:02.746 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.746 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.746 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.746 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.746 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.746 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:02.746 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.746 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.747 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.747 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.747 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:02.747 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.747 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.747 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.747 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.747 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:02.747 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.747 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.747 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.747 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.747 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920725+0000", 2024-09-16T14:49:02.747 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2198999999999999e-05 2024-09-16T14:49:02.748 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.748 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.748 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.748 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920735+0000", 2024-09-16T14:49:02.748 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0122000000000001e-05 2024-09-16T14:49:02.748 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.748 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.748 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.748 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920742+0000", 2024-09-16T14:49:02.748 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7179999999999999e-06 2024-09-16T14:49:02.749 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.749 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.749 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.749 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921079+0000", 2024-09-16T14:49:02.749 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00033676000000000002 2024-09-16T14:49:02.749 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.749 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.749 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.749 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921088+0000", 2024-09-16T14:49:02.749 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9290000000000004e-06 2024-09-16T14:49:02.749 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.749 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.749 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.749 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930916+0000", 2024-09-16T14:49:02.750 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0098279749999999992 2024-09-16T14:49:02.750 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.750 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.750 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.750 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930935+0000", 2024-09-16T14:49:02.750 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9510000000000001e-05 2024-09-16T14:49:02.750 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.750 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.750 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.750 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.750 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.750 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.fs7 37/27 ECSubWrite(tid=189, reqid=osd.7.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'12))", 2024-09-16T14:49:02.750 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:02.750 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.800741755000001, 2024-09-16T14:49:02.751 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010007947999999999, 2024-09-16T14:49:02.751 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.751 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.751 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.751 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.751 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.751 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:02.751 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.751 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.751 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.751 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.751 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:02.751 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.752 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.752 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.752 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.752 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920890+0000", 2024-09-16T14:49:02.752 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:02.752 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.752 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.752 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.752 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920900+0000", 2024-09-16T14:49:02.752 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9839999999999996e-06 2024-09-16T14:49:02.752 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.752 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.752 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.753 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920910+0000", 2024-09-16T14:49:02.753 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4220000000000006e-06 2024-09-16T14:49:02.753 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.753 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.753 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.753 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920917+0000", 2024-09-16T14:49:02.753 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5349999999999999e-06 2024-09-16T14:49:02.753 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.753 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.753 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.753 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920967+0000", 2024-09-16T14:49:02.753 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9348000000000003e-05 2024-09-16T14:49:02.753 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.754 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.754 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.754 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920976+0000", 2024-09-16T14:49:02.754 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8810000000000002e-06 2024-09-16T14:49:02.754 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.754 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.754 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.754 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930871+0000", 2024-09-16T14:49:02.754 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0098950500000000007 2024-09-16T14:49:02.754 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.754 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.755 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.755 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930899+0000", 2024-09-16T14:49:02.755 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.847e-05 2024-09-16T14:49:02.755 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.755 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.755 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.755 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.755 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.755 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s3 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:02.756 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:02.756 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.799744712999999, 2024-09-16T14:49:02.756 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011669621999999999, 2024-09-16T14:49:02.756 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.756 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.756 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.756 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.756 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.756 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:02.756 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.756 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.756 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.756 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.757 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:02.757 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.757 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.757 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.757 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.757 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921887+0000", 2024-09-16T14:49:02.757 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:02.757 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.757 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.757 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.757 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921911+0000", 2024-09-16T14:49:02.757 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4012999999999999e-05 2024-09-16T14:49:02.757 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921923+0000", 2024-09-16T14:49:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1642e-05 2024-09-16T14:49:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921932+0000", 2024-09-16T14:49:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0210000000000005e-06 2024-09-16T14:49:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921988+0000", 2024-09-16T14:49:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5822999999999997e-05 2024-09-16T14:49:02.759 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.759 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.759 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.759 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921997+0000", 2024-09-16T14:49:02.759 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7900000000000005e-06 2024-09-16T14:49:02.759 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.759 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.759 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.759 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933539+0000", 2024-09-16T14:49:02.759 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011542248 2024-09-16T14:49:02.759 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.759 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.759 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.760 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933558+0000", 2024-09-16T14:49:02.760 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8964999999999999e-05 2024-09-16T14:49:02.760 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.760 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.760 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.760 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.760 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.760 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds6 37/27 ECSubWrite(tid=190, reqid=osd.7.0:188, at_version=0'0, trim_to=0'0, roll_forward_to=37'20))", 2024-09-16T14:49:02.760 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:02.760 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.799663113000001, 2024-09-16T14:49:02.760 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0090799120000000007, 2024-09-16T14:49:02.761 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.761 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.761 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.761 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.761 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.761 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:02.761 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.762 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.762 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.762 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.762 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:02.762 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.762 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.763 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.763 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.763 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921969+0000", 2024-09-16T14:49:02.763 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:02.763 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.763 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.763 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.763 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921983+0000", 2024-09-16T14:49:02.763 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4403000000000001e-05 2024-09-16T14:49:02.763 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.763 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.763 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.763 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921996+0000", 2024-09-16T14:49:02.763 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.259e-05 2024-09-16T14:49:02.764 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.764 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.764 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.764 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922005+0000", 2024-09-16T14:49:02.764 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9080000000000001e-06 2024-09-16T14:49:02.764 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.764 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.764 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.764 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922073+0000", 2024-09-16T14:49:02.764 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8127999999999996e-05 2024-09-16T14:49:02.764 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.764 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.764 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.765 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922085+0000", 2024-09-16T14:49:02.765 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1712999999999999e-05 2024-09-16T14:49:02.765 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.765 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.765 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.765 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931031+0000", 2024-09-16T14:49:02.765 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089464669999999996 2024-09-16T14:49:02.765 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.765 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.765 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.765 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931050+0000", 2024-09-16T14:49:02.765 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8471999999999999e-05 2024-09-16T14:49:02.766 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.766 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.766 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.766 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.766 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.766 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.6s2 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:49:02.766 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.923513+0000", 2024-09-16T14:49:02.766 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.798120158, 2024-09-16T14:49:02.766 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0075220499999999997, 2024-09-16T14:49:02.766 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.766 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.766 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.767 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.767 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.767 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923513+0000", 2024-09-16T14:49:02.767 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.767 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.767 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.767 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.767 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923513+0000", 2024-09-16T14:49:02.767 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.767 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.767 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.767 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.768 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923512+0000", 2024-09-16T14:49:02.768 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:02.768 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.768 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.768 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.768 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923534+0000", 2024-09-16T14:49:02.768 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1892999999999999e-05 2024-09-16T14:49:02.768 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.768 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.768 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.768 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923551+0000", 2024-09-16T14:49:02.768 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7127999999999999e-05 2024-09-16T14:49:02.768 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.768 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.769 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.769 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923559+0000", 2024-09-16T14:49:02.769 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6400000000000003e-06 2024-09-16T14:49:02.769 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.769 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.769 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.769 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923615+0000", 2024-09-16T14:49:02.769 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5560000000000003e-05 2024-09-16T14:49:02.769 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.769 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.770 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.770 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923627+0000", 2024-09-16T14:49:02.770 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2296999999999999e-05 2024-09-16T14:49:02.770 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.770 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.770 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.770 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931010+0000", 2024-09-16T14:49:02.770 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.007382258 2024-09-16T14:49:02.770 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.770 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.770 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.770 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931035+0000", 2024-09-16T14:49:02.770 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5194e-05 2024-09-16T14:49:02.770 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.771 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.771 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.771 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.771 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.771 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as1 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:02.771 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:02.771 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.797567864000001, 2024-09-16T14:49:02.771 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0095052339999999996, 2024-09-16T14:49:02.771 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.771 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.771 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.771 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.771 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.771 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:02.772 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.772 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.772 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.772 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.772 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:02.772 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.772 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.772 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.772 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.772 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:02.772 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:02.772 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.772 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.772 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.773 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924089+0000", 2024-09-16T14:49:02.773 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3768999999999999e-05 2024-09-16T14:49:02.773 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.773 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.773 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.773 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924101+0000", 2024-09-16T14:49:02.773 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2294e-05 2024-09-16T14:49:02.773 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.773 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.773 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.773 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924109+0000", 2024-09-16T14:49:02.773 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7389999999999999e-06 2024-09-16T14:49:02.773 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.774 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.774 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.774 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924163+0000", 2024-09-16T14:49:02.774 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.4052999999999999e-05 2024-09-16T14:49:02.774 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.774 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.774 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.774 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924172+0000", 2024-09-16T14:49:02.774 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7189999999999996e-06 2024-09-16T14:49:02.774 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.774 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.774 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933561+0000", 2024-09-16T14:49:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0093884659999999998 2024-09-16T14:49:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933570+0000", 2024-09-16T14:49:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4120000000000007e-06 2024-09-16T14:49:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.es3 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:02.775 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:02.776 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.797510428999999, 2024-09-16T14:49:02.776 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0094028089999999998, 2024-09-16T14:49:02.776 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.776 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.776 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.776 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.776 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.776 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:02.776 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.776 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.776 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.776 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.776 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:02.777 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.777 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.777 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.777 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.777 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:02.777 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.777 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.777 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.777 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.777 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924142+0000", 2024-09-16T14:49:02.778 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0162000000000001e-05 2024-09-16T14:49:02.778 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.778 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.778 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.778 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924150+0000", 2024-09-16T14:49:02.778 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0919999999999998e-06 2024-09-16T14:49:02.779 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.779 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.779 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.779 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924157+0000", 2024-09-16T14:49:02.779 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6069999999999996e-06 2024-09-16T14:49:02.779 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.779 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.779 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.779 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924215+0000", 2024-09-16T14:49:02.779 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8192000000000001e-05 2024-09-16T14:49:02.779 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.779 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.779 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.780 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924225+0000", 2024-09-16T14:49:02.780 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9760000000000007e-06 2024-09-16T14:49:02.780 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.780 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.780 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.780 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933494+0000", 2024-09-16T14:49:02.780 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0092692720000000003 2024-09-16T14:49:02.780 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.780 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.780 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.780 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933525+0000", 2024-09-16T14:49:02.780 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0976999999999998e-05 2024-09-16T14:49:02.780 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.781 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.781 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.781 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.781 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.781 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs2 37/27 ECSubWrite(tid=118, reqid=osd.3.0:116, at_version=0'0, trim_to=0'0, roll_forward_to=37'14))", 2024-09-16T14:49:02.781 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.932325+0000", 2024-09-16T14:49:02.781 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.789308132999999, 2024-09-16T14:49:02.781 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0047342620000000004, 2024-09-16T14:49:02.781 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.781 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.781 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.781 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.781 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.781 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.932325+0000", 2024-09-16T14:49:02.782 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.782 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.782 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.782 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.782 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.932325+0000", 2024-09-16T14:49:02.782 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.782 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.782 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.782 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.782 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.932325+0000", 2024-09-16T14:49:02.782 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.782 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.782 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.783 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.783 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.932330+0000", 2024-09-16T14:49:02.783 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9000000000000003e-06 2024-09-16T14:49:02.783 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.783 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.783 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.783 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.932336+0000", 2024-09-16T14:49:02.783 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0889999999999999e-06 2024-09-16T14:49:02.783 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.783 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.783 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.783 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.932343+0000", 2024-09-16T14:49:02.783 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5490000000000001e-06 2024-09-16T14:49:02.783 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.784 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.784 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.784 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.932469+0000", 2024-09-16T14:49:02.784 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000125945 2024-09-16T14:49:02.784 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.784 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.784 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.784 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.932475+0000", 2024-09-16T14:49:02.784 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.1890000000000004e-06 2024-09-16T14:49:02.784 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.784 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.784 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.784 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.937000+0000", 2024-09-16T14:49:02.785 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045244220000000002 2024-09-16T14:49:02.785 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.785 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.785 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.785 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.937059+0000", 2024-09-16T14:49:02.785 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9422000000000001e-05 2024-09-16T14:49:02.785 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.785 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.785 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.785 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.785 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.785 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.7s6 37/27 ECSubWrite(tid=119, reqid=osd.3.0:119, at_version=37'39, trim_to=0'0, roll_forward_to=36'37))", 2024-09-16T14:49:02.785 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.933089+0000", 2024-09-16T14:49:02.786 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.788543628999999, 2024-09-16T14:49:02.786 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048817859999999999, 2024-09-16T14:49:02.786 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.786 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.786 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.786 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.786 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.786 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933089+0000", 2024-09-16T14:49:02.786 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.786 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.786 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.786 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.786 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933089+0000", 2024-09-16T14:49:02.786 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.787 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.787 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.787 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.787 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933089+0000", 2024-09-16T14:49:02.787 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:02.787 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.787 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.787 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.787 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933123+0000", 2024-09-16T14:49:02.787 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3973999999999997e-05 2024-09-16T14:49:02.787 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.787 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.788 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.788 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933135+0000", 2024-09-16T14:49:02.788 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2459999999999999e-05 2024-09-16T14:49:02.788 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.788 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.788 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.788 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933141+0000", 2024-09-16T14:49:02.788 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.7989999999999999e-06 2024-09-16T14:49:02.788 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.788 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.789 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.789 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933187+0000", 2024-09-16T14:49:02.789 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6100999999999998e-05 2024-09-16T14:49:02.789 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.789 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.789 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.789 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933198+0000", 2024-09-16T14:49:02.789 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1015999999999999e-05 2024-09-16T14:49:02.789 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.789 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.789 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.789 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.937948+0000", 2024-09-16T14:49:02.789 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0047503679999999996 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.937971+0000", 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2785e-05 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:384 3.3s0 3:c9d06351:::smithi04040151-33:head [delete] snapc 12=[12,10,c] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:53.856763+0000", 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 8.8648695340000003, 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0073332120000000004, 2024-09-16T14:49:02.790 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.791 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:02.791 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:02.791 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:02.791 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:02.791 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 384 2024-09-16T14:49:02.791 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.791 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.791 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.791 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.791 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856763+0000", 2024-09-16T14:49:02.791 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.791 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.791 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.791 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.792 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856763+0000", 2024-09-16T14:49:02.792 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.792 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.792 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.792 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.792 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856762+0000", 2024-09-16T14:49:02.792 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:02.792 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.792 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.792 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.792 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856770+0000", 2024-09-16T14:49:02.792 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4400000000000005e-06 2024-09-16T14:49:02.792 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.793 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.793 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.793 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856786+0000", 2024-09-16T14:49:02.793 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5608e-05 2024-09-16T14:49:02.793 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.793 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.793 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.793 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856794+0000", 2024-09-16T14:49:02.793 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9910000000000002e-06 2024-09-16T14:49:02.793 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.793 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.793 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.794 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856845+0000", 2024-09-16T14:49:02.794 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0862999999999997e-05 2024-09-16T14:49:02.794 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.794 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.794 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:02.794 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856855+0000", 2024-09-16T14:49:02.794 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9539999999999999e-06 2024-09-16T14:49:02.794 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.794 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.794 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.794 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.858580+0000", 2024-09-16T14:49:02.794 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001725096 2024-09-16T14:49:02.794 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.794 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.795 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:02.795 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.858688+0000", 2024-09-16T14:49:02.795 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000108879 2024-09-16T14:49:02.795 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.795 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.796 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.796 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.859596+0000", 2024-09-16T14:49:02.796 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00090781499999999997 2024-09-16T14:49:02.796 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.796 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.796 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.796 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.860493+0000", 2024-09-16T14:49:02.796 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00089657300000000001 2024-09-16T14:49:02.796 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.797 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.797 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:02.797 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864053+0000", 2024-09-16T14:49:02.797 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0035596339999999999 2024-09-16T14:49:02.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.797 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.797 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.797 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864097+0000", 2024-09-16T14:49:02.797 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4058999999999999e-05 2024-09-16T14:49:02.797 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.797 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.797 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.797 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.798 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s3 51/27 ECSubWrite(tid=169, reqid=osd.11.0:169, at_version=51'52, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:02.798 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.880653+0000", 2024-09-16T14:49:02.798 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 4.8409802309999996, 2024-09-16T14:49:02.798 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005579428, 2024-09-16T14:49:02.798 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.798 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.798 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.798 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.798 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.798 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880653+0000", 2024-09-16T14:49:02.798 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.798 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.798 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.798 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.799 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880653+0000", 2024-09-16T14:49:02.799 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.799 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.799 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.799 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880652+0000", 2024-09-16T14:49:02.799 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:02.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.799 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.799 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.799 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880672+0000", 2024-09-16T14:49:02.799 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9930999999999999e-05 2024-09-16T14:49:02.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.799 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880684+0000", 2024-09-16T14:49:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1871999999999999e-05 2024-09-16T14:49:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880691+0000", 2024-09-16T14:49:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4780000000000002e-06 2024-09-16T14:49:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880739+0000", 2024-09-16T14:49:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8136999999999997e-05 2024-09-16T14:49:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.801 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.801 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.801 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880749+0000", 2024-09-16T14:49:02.801 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5529999999999998e-06 2024-09-16T14:49:02.801 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.801 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.801 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.801 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.886204+0000", 2024-09-16T14:49:02.801 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005455321 2024-09-16T14:49:02.801 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.801 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.801 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.801 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.886232+0000", 2024-09-16T14:49:02.801 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7895e-05 2024-09-16T14:49:02.802 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.802 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.802 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.802 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.802 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.802 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as1 51/27 ECSubWrite(tid=171, reqid=osd.11.0:171, at_version=51'22, trim_to=0'0, roll_forward_to=49'20))", 2024-09-16T14:49:02.802 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.880756+0000", 2024-09-16T14:49:02.802 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 4.8408768110000002, 2024-09-16T14:49:02.802 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0055646999999999997, 2024-09-16T14:49:02.802 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.802 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.802 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.802 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.802 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.803 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880756+0000", 2024-09-16T14:49:02.803 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.803 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.803 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.803 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.803 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880756+0000", 2024-09-16T14:49:02.803 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.803 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.803 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.803 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.803 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880756+0000", 2024-09-16T14:49:02.803 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.804 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.804 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.804 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.804 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880773+0000", 2024-09-16T14:49:02.804 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7295999999999998e-05 2024-09-16T14:49:02.804 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.804 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.804 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.804 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880782+0000", 2024-09-16T14:49:02.804 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6640000000000004e-06 2024-09-16T14:49:02.804 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.804 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.805 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.805 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880788+0000", 2024-09-16T14:49:02.805 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8419999999999999e-06 2024-09-16T14:49:02.805 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.805 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.805 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.805 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880828+0000", 2024-09-16T14:49:02.805 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9419000000000003e-05 2024-09-16T14:49:02.805 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.805 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.806 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.806 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880834+0000", 2024-09-16T14:49:02.806 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8250000000000001e-06 2024-09-16T14:49:02.806 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.806 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.806 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.806 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.886297+0000", 2024-09-16T14:49:02.806 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005463514 2024-09-16T14:49:02.806 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.806 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.806 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.807 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.886321+0000", 2024-09-16T14:49:02.807 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3558999999999999e-05 2024-09-16T14:49:02.807 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.807 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.807 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.807 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.807 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.807 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s3 51/27 ECSubWrite(tid=173, reqid=osd.11.0:173, at_version=51'54, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:02.808 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.881482+0000", 2024-09-16T14:49:02.808 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 4.8401513339999997, 2024-09-16T14:49:02.808 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0051394190000000001, 2024-09-16T14:49:02.808 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.808 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.808 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.808 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.808 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.808 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881482+0000", 2024-09-16T14:49:02.808 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.808 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.809 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.809 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881482+0000", 2024-09-16T14:49:02.809 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.809 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.809 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881481+0000", 2024-09-16T14:49:02.809 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.809 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.809 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881510+0000", 2024-09-16T14:49:02.810 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.9377999999999999e-05 2024-09-16T14:49:02.810 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.810 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.810 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.810 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881525+0000", 2024-09-16T14:49:02.810 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5064e-05 2024-09-16T14:49:02.810 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.810 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.810 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.810 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881533+0000", 2024-09-16T14:49:02.810 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5859999999999999e-06 2024-09-16T14:49:02.810 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.810 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.810 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.811 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881568+0000", 2024-09-16T14:49:02.811 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.4589999999999999e-05 2024-09-16T14:49:02.811 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.811 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.811 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.811 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881574+0000", 2024-09-16T14:49:02.811 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.0299999999999999e-06 2024-09-16T14:49:02.811 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.811 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.811 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.811 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.886597+0000", 2024-09-16T14:49:02.812 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050232000000000002 2024-09-16T14:49:02.812 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.812 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.812 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.812 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.886621+0000", 2024-09-16T14:49:02.812 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4090000000000001e-05 2024-09-16T14:49:02.812 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.812 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.812 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.812 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.812 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.812 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s3 51/27 ECSubWrite(tid=175, reqid=osd.11.0:173, at_version=0'0, trim_to=0'0, roll_forward_to=51'54))", 2024-09-16T14:49:02.812 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.892410+0000", 2024-09-16T14:49:02.813 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 4.8292225609999999, 2024-09-16T14:49:02.813 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053568419999999997, 2024-09-16T14:49:02.813 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.813 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.813 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.813 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.813 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.813 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892410+0000", 2024-09-16T14:49:02.813 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.813 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.813 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.813 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.813 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892410+0000", 2024-09-16T14:49:02.814 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.814 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.814 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.814 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892410+0000", 2024-09-16T14:49:02.814 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.814 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.814 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.814 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892420+0000", 2024-09-16T14:49:02.814 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9820000000000003e-06 2024-09-16T14:49:02.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.814 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.814 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.815 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892430+0000", 2024-09-16T14:49:02.815 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0603e-05 2024-09-16T14:49:02.815 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.815 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.815 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.815 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892439+0000", 2024-09-16T14:49:02.815 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2749999999999995e-06 2024-09-16T14:49:02.815 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.816 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.816 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.816 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892473+0000", 2024-09-16T14:49:02.816 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.4694999999999999e-05 2024-09-16T14:49:02.816 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.816 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.816 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.816 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892480+0000", 2024-09-16T14:49:02.816 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3339999999999998e-06 2024-09-16T14:49:02.816 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.816 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.816 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.816 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.897742+0000", 2024-09-16T14:49:02.816 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0052626440000000003 2024-09-16T14:49:02.817 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.817 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.817 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.817 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.897767+0000", 2024-09-16T14:49:02.817 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4842e-05 2024-09-16T14:49:02.817 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.817 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.817 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.817 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.817 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.817 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds6 51/27 ECSubWrite(tid=210, reqid=osd.7.0:209, at_version=0'0, trim_to=0'0, roll_forward_to=51'29))", 2024-09-16T14:49:02.817 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.893020+0000", 2024-09-16T14:49:02.817 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 4.8286129730000003, 2024-09-16T14:49:02.817 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0047727450000000001, 2024-09-16T14:49:02.818 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.818 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.818 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.818 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.818 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.818 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893020+0000", 2024-09-16T14:49:02.818 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.818 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.818 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.818 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893020+0000", 2024-09-16T14:49:02.818 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893019+0000", 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893028+0000", 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5599999999999994e-06 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893038+0000", 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0019e-05 2024-09-16T14:49:02.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.820 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.820 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.820 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893043+0000", 2024-09-16T14:49:02.820 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0309999999999998e-06 2024-09-16T14:49:02.820 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.820 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.820 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.820 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893100+0000", 2024-09-16T14:49:02.820 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.7127999999999999e-05 2024-09-16T14:49:02.820 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.820 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.820 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.820 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893109+0000", 2024-09-16T14:49:02.821 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4999999999999999e-06 2024-09-16T14:49:02.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.821 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.821 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.821 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.897775+0000", 2024-09-16T14:49:02.821 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0046666090000000004 2024-09-16T14:49:02.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.821 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.821 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.821 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.897793+0000", 2024-09-16T14:49:02.821 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7563000000000001e-05 2024-09-16T14:49:02.821 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.821 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.821 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as1 51/27 ECSubWrite(tid=177, reqid=osd.11.0:171, at_version=0'0, trim_to=0'0, roll_forward_to=51'22))", 2024-09-16T14:49:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.893088+0000", 2024-09-16T14:49:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 4.8285445830000002, 2024-09-16T14:49:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.004697863, 2024-09-16T14:49:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893088+0000", 2024-09-16T14:49:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.823 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.823 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.823 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893088+0000", 2024-09-16T14:49:02.823 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.823 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.823 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.823 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.823 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893088+0000", 2024-09-16T14:49:02.823 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.823 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.823 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.823 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.823 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893097+0000", 2024-09-16T14:49:02.823 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8219999999999996e-06 2024-09-16T14:49:02.824 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.824 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.824 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.824 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893105+0000", 2024-09-16T14:49:02.824 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8499999999999994e-06 2024-09-16T14:49:02.824 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.824 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.824 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.824 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893111+0000", 2024-09-16T14:49:02.824 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.5910000000000001e-06 2024-09-16T14:49:02.824 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.824 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.824 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.824 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893148+0000", 2024-09-16T14:49:02.825 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6553000000000001e-05 2024-09-16T14:49:02.825 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.825 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.825 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.825 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893156+0000", 2024-09-16T14:49:02.825 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5339999999999999e-06 2024-09-16T14:49:02.825 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.825 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.825 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.825 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.897759+0000", 2024-09-16T14:49:02.825 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0046030630000000001 2024-09-16T14:49:02.825 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.825 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.825 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.826 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.897786+0000", 2024-09-16T14:49:02.826 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6889e-05 2024-09-16T14:49:02.826 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.826 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.826 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.826 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.826 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.826 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as1 52/27 ECSubWrite(tid=187, reqid=osd.11.0:187, at_version=52'24, trim_to=0'0, roll_forward_to=51'22))", 2024-09-16T14:49:02.826 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:58.903387+0000", 2024-09-16T14:49:02.826 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.8182463520000001, 2024-09-16T14:49:02.826 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0059598029999999996, 2024-09-16T14:49:02.826 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.826 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.826 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903387+0000", 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903387+0000", 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903386+0000", 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.827 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.828 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.828 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.828 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903399+0000", 2024-09-16T14:49:02.828 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3086e-05 2024-09-16T14:49:02.828 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.828 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.828 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.828 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903405+0000", 2024-09-16T14:49:02.828 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8159999999999999e-06 2024-09-16T14:49:02.828 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.828 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.828 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.828 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903411+0000", 2024-09-16T14:49:02.828 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.84e-06 2024-09-16T14:49:02.829 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.829 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.829 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.829 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903467+0000", 2024-09-16T14:49:02.829 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5878000000000003e-05 2024-09-16T14:49:02.829 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.829 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.829 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.829 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903476+0000", 2024-09-16T14:49:02.829 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8119999999999997e-06 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909323+0000", 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005847316 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909346+0000", 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.338e-05 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.830 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.831 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs2 52/27 ECSubWrite(tid=192, reqid=osd.3.0:192, at_version=52'28, trim_to=0'0, roll_forward_to=51'24))", 2024-09-16T14:49:02.831 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:58.906213+0000", 2024-09-16T14:49:02.831 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.8154197229999998, 2024-09-16T14:49:02.831 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049290319999999999, 2024-09-16T14:49:02.831 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:02.831 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:02.831 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:02.831 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.831 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:02.831 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.906213+0000", 2024-09-16T14:49:02.831 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.831 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.831 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:02.832 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.906213+0000", 2024-09-16T14:49:02.832 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:02.832 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.832 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.832 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:02.832 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.906212+0000", 2024-09-16T14:49:02.832 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:02.832 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.832 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.832 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:02.832 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.906230+0000", 2024-09-16T14:49:02.832 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7674000000000001e-05 2024-09-16T14:49:02.832 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.832 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.833 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:02.833 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.906243+0000", 2024-09-16T14:49:02.833 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2432000000000001e-05 2024-09-16T14:49:02.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.833 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.833 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:02.833 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.906250+0000", 2024-09-16T14:49:02.833 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.498e-06 2024-09-16T14:49:02.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.833 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.833 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:02.833 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.906761+0000", 2024-09-16T14:49:02.834 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00051108600000000001 2024-09-16T14:49:02.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.834 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.834 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:02.834 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.906768+0000", 2024-09-16T14:49:02.834 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3690000000000003e-06 2024-09-16T14:49:02.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.834 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.834 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:02.834 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.911124+0000", 2024-09-16T14:49:02.835 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0043566500000000001 2024-09-16T14:49:02.835 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:02.835 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:02.835 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:02.835 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.911142+0000", 2024-09-16T14:49:02.835 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8006999999999999e-05 2024-09-16T14:49:02.836 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.836 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.836 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.836 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:02.836 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:02.836 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:02.838 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:49:02.839 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:02.837+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:02.894 INFO:tasks.rados.rados.0.smithi040.stdout:207: done (3 left) 2024-09-16T14:49:02.894 INFO:tasks.rados.rados.0.smithi040.stdout:208: done (2 left) 2024-09-16T14:49:02.894 INFO:tasks.rados.rados.0.smithi040.stdout:209: done (1 left) 2024-09-16T14:49:02.894 INFO:tasks.rados.rados.0.smithi040.stdout:210: done (0 left) 2024-09-16T14:49:02.895 INFO:tasks.rados.rados.0.smithi040.stdout:211: rollback oid 3 current snap is 15 2024-09-16T14:49:02.895 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 3 to 10 2024-09-16T14:49:02.895 INFO:tasks.rados.rados.0.smithi040.stdout:212: setattr oid 37 current snap is 15 2024-09-16T14:49:02.896 INFO:tasks.rados.rados.0.smithi040.stdout:213: read oid 4 snap 10 2024-09-16T14:49:02.896 INFO:tasks.rados.rados.0.smithi040.stdout:213: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-16T14:49:02.896 INFO:tasks.rados.rados.0.smithi040.stdout:214: snap_create 2024-09-16T14:49:02.902 INFO:tasks.rados.rados.0.smithi040.stdout:211: finishing rollback tid 0 to smithi04040151-3 2024-09-16T14:49:02.903 INFO:tasks.rados.rados.0.smithi040.stdout:211: finishing rollback tid 1 to smithi04040151-3 2024-09-16T14:49:02.904 INFO:tasks.rados.rados.0.smithi040.stdout:211: finishing rollback tid 2 to smithi04040151-3 2024-09-16T14:49:02.904 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 3 v 44 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-16T14:49:02.904 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 37 v 70 (ObjNum 33 snap 0 seq_num 209864231) dirty exists 2024-09-16T14:49:02.906 INFO:tasks.rados.rados.0.smithi040.stdout:213: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-16T14:49:02.941 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:02.941+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:02.984 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:02.984 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:02.984 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:02.984 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:02.992 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:49:03.076 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:03.075+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:03.134 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:03.134 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:03.134 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:03.134 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:03.135 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:03.143 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:49:03.144 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:03.143+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:03.245 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:03.244+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:03.287 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:03.288 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:03.288 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:03.288 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:03.288 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.288 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s4 33/27 ECSubWrite(tid=76, reqid=client.4500.0:227, at_version=33'16, trim_to=0'0, roll_forward_to=30'10))", 2024-09-16T14:49:03.288 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:39.917607+0000", 2024-09-16T14:49:03.288 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 23.367893681999998, 2024-09-16T14:49:03.288 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048754200000000001, 2024-09-16T14:49:03.288 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.288 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.288 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.289 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.289 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.289 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917607+0000", 2024-09-16T14:49:03.289 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.289 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.289 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.289 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.289 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917607+0000", 2024-09-16T14:49:03.289 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.289 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.289 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.289 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.289 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917606+0000", 2024-09-16T14:49:03.290 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:03.290 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.290 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.290 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.290 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917631+0000", 2024-09-16T14:49:03.290 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4905e-05 2024-09-16T14:49:03.290 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.290 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.290 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.290 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917646+0000", 2024-09-16T14:49:03.290 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4640999999999999e-05 2024-09-16T14:49:03.290 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.290 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.290 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.291 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917654+0000", 2024-09-16T14:49:03.291 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0779999999999996e-06 2024-09-16T14:49:03.291 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.291 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.291 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.291 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917697+0000", 2024-09-16T14:49:03.291 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3245999999999999e-05 2024-09-16T14:49:03.291 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.291 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.291 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.291 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.917705+0000", 2024-09-16T14:49:03.291 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0350000000000001e-06 2024-09-16T14:49:03.291 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.292 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.292 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.292 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.922460+0000", 2024-09-16T14:49:03.292 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0047549569999999998 2024-09-16T14:49:03.292 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.292 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.292 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.292 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:39.922482+0000", 2024-09-16T14:49:03.292 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2280999999999999e-05 2024-09-16T14:49:03.292 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.292 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.292 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.292 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.293 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.293 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s4 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:03.293 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921852+0000", 2024-09-16T14:49:03.293 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 19.363648675, 2024-09-16T14:49:03.293 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0065836089999999998, 2024-09-16T14:49:03.293 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.293 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.293 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.293 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.293 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.293 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921852+0000", 2024-09-16T14:49:03.294 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.294 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.294 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.294 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.294 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921852+0000", 2024-09-16T14:49:03.294 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.294 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.294 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.294 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.294 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921851+0000", 2024-09-16T14:49:03.294 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:03.294 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.294 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.294 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.295 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921890+0000", 2024-09-16T14:49:03.295 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8668000000000003e-05 2024-09-16T14:49:03.295 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.295 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.295 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.295 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921906+0000", 2024-09-16T14:49:03.295 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6772000000000001e-05 2024-09-16T14:49:03.295 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.295 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.295 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.295 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921920+0000", 2024-09-16T14:49:03.295 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3525e-05 2024-09-16T14:49:03.295 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.296 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.296 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.296 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923071+0000", 2024-09-16T14:49:03.296 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001150812 2024-09-16T14:49:03.296 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.296 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.296 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.296 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923079+0000", 2024-09-16T14:49:03.296 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6119999999999999e-06 2024-09-16T14:49:03.296 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.296 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.296 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.296 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.928412+0000", 2024-09-16T14:49:03.297 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053327419999999997 2024-09-16T14:49:03.297 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.297 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.297 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.297 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.928435+0000", 2024-09-16T14:49:03.297 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3419999999999999e-05 2024-09-16T14:49:03.297 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.297 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.297 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.297 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.297 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.297 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.6s7 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:49:03.297 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.923528+0000", 2024-09-16T14:49:03.298 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 19.361972915999999, 2024-09-16T14:49:03.298 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049476429999999998, 2024-09-16T14:49:03.298 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.298 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.298 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.298 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.298 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.298 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923528+0000", 2024-09-16T14:49:03.298 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.298 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.298 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.298 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.298 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923528+0000", 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923527+0000", 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923536+0000", 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5999999999999996e-06 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.299 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923551+0000", 2024-09-16T14:49:03.300 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4459e-05 2024-09-16T14:49:03.300 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.300 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.300 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.300 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923560+0000", 2024-09-16T14:49:03.300 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5519999999999993e-06 2024-09-16T14:49:03.300 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.300 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.300 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.300 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923614+0000", 2024-09-16T14:49:03.300 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3538000000000001e-05 2024-09-16T14:49:03.300 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923623+0000", 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9299999999999992e-06 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.928446+0000", 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048234369999999999 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.928475+0000", 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8904999999999999e-05 2024-09-16T14:49:03.301 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.302 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.302 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.302 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.302 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.302 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s3 37/27 ECSubWrite(tid=34, reqid=osd.1.0:33, at_version=0'0, trim_to=0'0, roll_forward_to=37'24))", 2024-09-16T14:49:03.302 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.931191+0000", 2024-09-16T14:49:03.302 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 19.354309894, 2024-09-16T14:49:03.302 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050404129999999997, 2024-09-16T14:49:03.302 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.302 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.302 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.302 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.302 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.303 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931191+0000", 2024-09-16T14:49:03.303 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.303 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.303 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.303 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.303 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931191+0000", 2024-09-16T14:49:03.303 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.303 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.303 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.303 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.303 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931190+0000", 2024-09-16T14:49:03.303 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.303 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.304 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.304 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.304 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931199+0000", 2024-09-16T14:49:03.304 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5289999999999997e-06 2024-09-16T14:49:03.304 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.304 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.304 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.304 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931211+0000", 2024-09-16T14:49:03.304 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1600000000000001e-05 2024-09-16T14:49:03.304 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.304 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.304 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.304 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931217+0000", 2024-09-16T14:49:03.305 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3790000000000003e-06 2024-09-16T14:49:03.305 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.305 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.305 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.305 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931260+0000", 2024-09-16T14:49:03.305 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2577999999999998e-05 2024-09-16T14:49:03.305 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.305 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.305 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.305 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931268+0000", 2024-09-16T14:49:03.305 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0679999999999997e-06 2024-09-16T14:49:03.305 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.305 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.305 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.936212+0000", 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049436840000000003 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.936231+0000", 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9198000000000002e-05 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:449 3.5s0 3:a7111aa9:::smithi04040151-45:head [stat,delete] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:03.306 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.409114067, 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013571421, 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 449 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.307 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.308 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.308 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:03.308 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.308 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.308 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.308 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.308 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876385+0000", 2024-09-16T14:49:03.308 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:03.308 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.308 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.308 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.308 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876396+0000", 2024-09-16T14:49:03.308 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1094000000000001e-05 2024-09-16T14:49:03.308 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.309 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.309 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.309 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876412+0000", 2024-09-16T14:49:03.309 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6563e-05 2024-09-16T14:49:03.309 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.309 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.309 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.309 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876421+0000", 2024-09-16T14:49:03.309 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5189999999999992e-06 2024-09-16T14:49:03.309 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.309 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.309 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.309 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876477+0000", 2024-09-16T14:49:03.310 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.6679999999999999e-05 2024-09-16T14:49:03.310 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.310 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.310 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:03.310 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876487+0000", 2024-09-16T14:49:03.310 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4509999999999998e-06 2024-09-16T14:49:03.310 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.310 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.310 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.310 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.878486+0000", 2024-09-16T14:49:03.310 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001999635 2024-09-16T14:49:03.310 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.310 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.310 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.311 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.878592+0000", 2024-09-16T14:49:03.311 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000105398 2024-09-16T14:49:03.311 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.311 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.311 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.311 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.889958+0000", 2024-09-16T14:49:03.311 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01136595 2024-09-16T14:49:03.311 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.311 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.311 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.311 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.311 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.311 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 51/27 ECSubWrite(tid=169, reqid=osd.11.0:169, at_version=51'52, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:03.312 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.881078+0000", 2024-09-16T14:49:03.312 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.4044221820000002, 2024-09-16T14:49:03.312 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049463199999999997, 2024-09-16T14:49:03.312 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.312 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.312 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.312 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.312 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.312 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881078+0000", 2024-09-16T14:49:03.312 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.312 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.313 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.313 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.313 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881078+0000", 2024-09-16T14:49:03.313 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.313 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.313 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.313 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.313 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881078+0000", 2024-09-16T14:49:03.313 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.313 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.313 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.313 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.313 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881095+0000", 2024-09-16T14:49:03.313 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7686999999999999e-05 2024-09-16T14:49:03.314 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.314 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.314 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.314 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881106+0000", 2024-09-16T14:49:03.314 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0105000000000001e-05 2024-09-16T14:49:03.314 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.314 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.314 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.314 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881114+0000", 2024-09-16T14:49:03.314 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9810000000000003e-06 2024-09-16T14:49:03.314 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.314 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.314 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.314 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881152+0000", 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8738000000000001e-05 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881161+0000", 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4449999999999996e-06 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.886004+0000", 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048437360000000004 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.315 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.886025+0000", 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0160999999999999e-05 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 51/27 ECSubWrite(tid=173, reqid=osd.11.0:173, at_version=51'54, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.881541+0000", 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.4039599420000002, 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049332719999999998, 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.316 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.317 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881541+0000", 2024-09-16T14:49:03.317 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.317 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.317 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.317 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.317 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881541+0000", 2024-09-16T14:49:03.317 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.317 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.317 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.317 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.317 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881540+0000", 2024-09-16T14:49:03.317 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.317 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.317 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.318 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.318 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881554+0000", 2024-09-16T14:49:03.318 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4416000000000001e-05 2024-09-16T14:49:03.318 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.318 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.318 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.318 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881573+0000", 2024-09-16T14:49:03.318 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8136000000000001e-05 2024-09-16T14:49:03.318 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.318 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.318 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.318 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881580+0000", 2024-09-16T14:49:03.318 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9450000000000004e-06 2024-09-16T14:49:03.318 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881612+0000", 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2700999999999997e-05 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.881619+0000", 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8789999999999997e-06 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.886456+0000", 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048364100000000002 2024-09-16T14:49:03.319 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.320 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.320 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.320 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.886474+0000", 2024-09-16T14:49:03.320 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8304999999999998e-05 2024-09-16T14:49:03.320 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.320 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.320 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.320 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.320 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.320 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 51/27 ECSubWrite(tid=175, reqid=osd.11.0:173, at_version=0'0, trim_to=0'0, roll_forward_to=51'54))", 2024-09-16T14:49:03.320 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.892545+0000", 2024-09-16T14:49:03.320 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.3929556590000001, 2024-09-16T14:49:03.320 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0052329689999999996, 2024-09-16T14:49:03.320 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.321 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.321 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.321 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.321 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.321 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892545+0000", 2024-09-16T14:49:03.321 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.321 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.321 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.321 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.321 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892545+0000", 2024-09-16T14:49:03.321 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.321 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.321 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.321 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.322 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892544+0000", 2024-09-16T14:49:03.322 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.322 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.322 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.322 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.322 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892553+0000", 2024-09-16T14:49:03.322 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4100000000000008e-06 2024-09-16T14:49:03.322 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.322 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.322 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.322 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892562+0000", 2024-09-16T14:49:03.322 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6390000000000004e-06 2024-09-16T14:49:03.322 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.322 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.323 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.323 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892567+0000", 2024-09-16T14:49:03.323 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9490000000000004e-06 2024-09-16T14:49:03.323 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.323 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.323 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.323 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892603+0000", 2024-09-16T14:49:03.323 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5290999999999998e-05 2024-09-16T14:49:03.323 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.323 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.323 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.323 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.892608+0000", 2024-09-16T14:49:03.323 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:49:03.323 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.324 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.324 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.324 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.897756+0000", 2024-09-16T14:49:03.324 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0051476849999999999 2024-09-16T14:49:03.324 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.324 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.324 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.324 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.897778+0000", 2024-09-16T14:49:03.324 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.209e-05 2024-09-16T14:49:03.324 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.324 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.324 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.324 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.324 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.325 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds1 51/27 ECSubWrite(tid=210, reqid=osd.7.0:209, at_version=0'0, trim_to=0'0, roll_forward_to=51'29))", 2024-09-16T14:49:03.325 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.893021+0000", 2024-09-16T14:49:03.325 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.3924791509999999, 2024-09-16T14:49:03.325 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048356500000000004, 2024-09-16T14:49:03.325 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.325 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.325 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.325 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.325 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.325 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893021+0000", 2024-09-16T14:49:03.325 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.325 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.325 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.325 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.326 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893021+0000", 2024-09-16T14:49:03.326 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.326 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.326 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.326 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.326 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893021+0000", 2024-09-16T14:49:03.326 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.326 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.326 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.326 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.326 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893029+0000", 2024-09-16T14:49:03.326 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7020000000000001e-06 2024-09-16T14:49:03.326 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.326 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893037+0000", 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8230000000000001e-06 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893042+0000", 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1919999999999998e-06 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893086+0000", 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4274000000000003e-05 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.327 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.893095+0000", 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4499999999999993e-06 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.897836+0000", 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0047402690000000001 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.897857+0000", 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1387999999999999e-05 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.328 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.329 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.329 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.329 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.329 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 51/27 ECSubWrite(tid=179, reqid=osd.11.0:176, at_version=0'0, trim_to=0'0, roll_forward_to=51'56))", 2024-09-16T14:49:03.329 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.900995+0000", 2024-09-16T14:49:03.329 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.3845058410000002, 2024-09-16T14:49:03.329 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0046389659999999996, 2024-09-16T14:49:03.329 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.329 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.330 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.330 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.330 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.330 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.900995+0000", 2024-09-16T14:49:03.330 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.330 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.330 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.330 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.330 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.900995+0000", 2024-09-16T14:49:03.330 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.330 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.330 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.330 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.330 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.900994+0000", 2024-09-16T14:49:03.331 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.331 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.331 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.331 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.331 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.901003+0000", 2024-09-16T14:49:03.331 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1220000000000001e-06 2024-09-16T14:49:03.331 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.331 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.331 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.331 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.901015+0000", 2024-09-16T14:49:03.331 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2262999999999999e-05 2024-09-16T14:49:03.331 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.331 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.332 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.332 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.901025+0000", 2024-09-16T14:49:03.332 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6050000000000004e-06 2024-09-16T14:49:03.332 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.332 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.332 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.332 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.901068+0000", 2024-09-16T14:49:03.332 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2617000000000002e-05 2024-09-16T14:49:03.332 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.332 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.332 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.332 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.901076+0000", 2024-09-16T14:49:03.332 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6999999999999997e-06 2024-09-16T14:49:03.333 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.333 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.333 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.333 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.905612+0000", 2024-09-16T14:49:03.333 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045359069999999996 2024-09-16T14:49:03.333 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.333 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.333 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.333 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.905634+0000", 2024-09-16T14:49:03.333 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.128e-05 2024-09-16T14:49:03.333 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.333 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.333 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.333 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.334 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.334 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:466 3.5s0 3:a29e72cc:::smithi04040151-39:head [append 0~0] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:03.334 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995540+0000", 2024-09-16T14:49:03.334 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.289960486, 2024-09-16T14:49:03.334 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084481160000000003, 2024-09-16T14:49:03.334 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.334 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.334 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.334 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:03.334 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:03.334 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 466 2024-09-16T14:49:03.334 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.334 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.335 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.335 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.335 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995540+0000", 2024-09-16T14:49:03.335 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.335 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.335 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.335 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.335 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995540+0000", 2024-09-16T14:49:03.335 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.335 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.335 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.335 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.336 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995538+0000", 2024-09-16T14:49:03.336 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:03.336 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.336 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.336 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.336 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995548+0000", 2024-09-16T14:49:03.336 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1570000000000006e-06 2024-09-16T14:49:03.336 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.336 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.336 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.336 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995560+0000", 2024-09-16T14:49:03.336 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2527999999999999e-05 2024-09-16T14:49:03.336 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.337 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.337 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.337 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995567+0000", 2024-09-16T14:49:03.337 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2030000000000003e-06 2024-09-16T14:49:03.337 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.337 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.337 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.337 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995620+0000", 2024-09-16T14:49:03.337 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2516000000000003e-05 2024-09-16T14:49:03.337 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.337 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.337 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.337 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995718+0000", 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.781e-05 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.996361+0000", 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00064323600000000005 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.999417+0000", 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0030556120000000001 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.338 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003945+0000", 2024-09-16T14:49:03.339 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045287950000000004 2024-09-16T14:49:03.339 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.339 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.339 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.339 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003988+0000", 2024-09-16T14:49:03.339 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2758e-05 2024-09-16T14:49:03.339 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.339 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.339 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.339 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:467 3.5s0 3:a29e72cc:::smithi04040151-39:head [rollback 14] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.289921509, 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012554012, 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 467 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.340 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.341 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:03.341 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.341 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.341 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.341 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.341 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:03.341 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.341 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.341 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.341 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.341 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:03.341 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.341 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.341 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.342 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.342 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995581+0000", 2024-09-16T14:49:03.342 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.9009999999999998e-06 2024-09-16T14:49:03.342 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.342 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.342 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.342 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995587+0000", 2024-09-16T14:49:03.342 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8850000000000004e-06 2024-09-16T14:49:03.342 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.342 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.342 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.342 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995592+0000", 2024-09-16T14:49:03.342 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9749999999999998e-06 2024-09-16T14:49:03.343 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.343 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.343 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.343 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.996914+0000", 2024-09-16T14:49:03.343 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00132168 2024-09-16T14:49:03.343 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.343 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.343 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.343 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997029+0000", 2024-09-16T14:49:03.344 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000115203 2024-09-16T14:49:03.344 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.344 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.344 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.344 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997570+0000", 2024-09-16T14:49:03.344 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00054107999999999999 2024-09-16T14:49:03.344 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.344 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.344 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.344 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003265+0000", 2024-09-16T14:49:03.344 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0056945679999999997 2024-09-16T14:49:03.344 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.344 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.344 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.345 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008067+0000", 2024-09-16T14:49:03.345 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048018039999999998 2024-09-16T14:49:03.345 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.345 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.345 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.345 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008133+0000", 2024-09-16T14:49:03.345 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6365999999999994e-05 2024-09-16T14:49:03.345 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.345 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.345 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.345 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.345 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.345 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:468 3.5s0 3:a29e72cc:::smithi04040151-39:head [append 0~0] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:03.346 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:03.346 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.2897524489999999, 2024-09-16T14:49:03.346 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012562663999999999, 2024-09-16T14:49:03.346 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.346 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.346 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.346 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:03.346 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:03.346 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 468 2024-09-16T14:49:03.346 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.346 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.346 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.346 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.347 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:03.347 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.347 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.347 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.347 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:03.347 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.347 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.347 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.347 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995747+0000", 2024-09-16T14:49:03.347 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:03.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.347 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.348 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.348 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995752+0000", 2024-09-16T14:49:03.348 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.916e-06 2024-09-16T14:49:03.348 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.348 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.348 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.348 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995762+0000", 2024-09-16T14:49:03.348 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4609999999999997e-06 2024-09-16T14:49:03.348 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.348 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.348 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.348 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995768+0000", 2024-09-16T14:49:03.348 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8109999999999997e-06 2024-09-16T14:49:03.349 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.349 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.349 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.349 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.998577+0000", 2024-09-16T14:49:03.349 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0028088369999999998 2024-09-16T14:49:03.349 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.349 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.349 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.349 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.998680+0000", 2024-09-16T14:49:03.349 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010318399999999999 2024-09-16T14:49:03.349 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.349 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.350 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.350 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.999182+0000", 2024-09-16T14:49:03.350 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00050101099999999999 2024-09-16T14:49:03.350 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.350 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.350 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.350 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.005964+0000", 2024-09-16T14:49:03.350 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0067821490000000003 2024-09-16T14:49:03.350 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.350 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.350 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.350 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008246+0000", 2024-09-16T14:49:03.350 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0022827670000000002 2024-09-16T14:49:03.351 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.351 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.351 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.351 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008311+0000", 2024-09-16T14:49:03.351 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.4269000000000004e-05 2024-09-16T14:49:03.351 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.351 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.351 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.351 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.351 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.351 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:473 3.cs0 3:31a6e1c2:::smithi04040151-23:head [read 0~835584,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:03.351 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995811+0000", 2024-09-16T14:49:03.351 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.2896891559999997, 2024-09-16T14:49:03.351 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048106249999999998, 2024-09-16T14:49:03.352 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.352 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:03.352 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.352 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:03.352 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:03.352 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 473 2024-09-16T14:49:03.352 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.352 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.352 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.352 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.352 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995811+0000", 2024-09-16T14:49:03.352 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.352 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.352 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.353 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.353 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995811+0000", 2024-09-16T14:49:03.353 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.353 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.353 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.353 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.353 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995811+0000", 2024-09-16T14:49:03.353 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.353 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.353 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.353 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.353 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995816+0000", 2024-09-16T14:49:03.353 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.1080000000000001e-06 2024-09-16T14:49:03.353 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.354 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.354 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.354 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995823+0000", 2024-09-16T14:49:03.354 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.5579999999999997e-06 2024-09-16T14:49:03.354 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.354 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.354 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.354 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995827+0000", 2024-09-16T14:49:03.354 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.442e-06 2024-09-16T14:49:03.354 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.354 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.354 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.354 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.996896+0000", 2024-09-16T14:49:03.355 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001069375 2024-09-16T14:49:03.355 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.355 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.355 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.355 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.996997+0000", 2024-09-16T14:49:03.355 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000100998 2024-09-16T14:49:03.355 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.355 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.355 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.355 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.000622+0000", 2024-09-16T14:49:03.355 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036245909999999999 2024-09-16T14:49:03.355 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.355 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.355 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.356 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.356 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.356 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:474 3.cs0 3:31a6e1c2:::smithi04040151-23:head [read 0~835584,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:03.356 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995848+0000", 2024-09-16T14:49:03.356 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.2896524600000001, 2024-09-16T14:49:03.356 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0052021999999999997, 2024-09-16T14:49:03.356 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.356 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:03.356 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.356 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:03.356 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:03.356 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 474 2024-09-16T14:49:03.356 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.357 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.357 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.357 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.357 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995848+0000", 2024-09-16T14:49:03.357 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.357 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.357 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.357 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.357 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995848+0000", 2024-09-16T14:49:03.357 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.357 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.357 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.357 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.357 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995848+0000", 2024-09-16T14:49:03.358 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.358 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.358 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.358 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.358 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995851+0000", 2024-09-16T14:49:03.358 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2320000000000001e-06 2024-09-16T14:49:03.358 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.358 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.358 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.358 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995857+0000", 2024-09-16T14:49:03.358 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3829999999999997e-06 2024-09-16T14:49:03.358 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.358 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995862+0000", 2024-09-16T14:49:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9710000000000003e-06 2024-09-16T14:49:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997296+0000", 2024-09-16T14:49:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001433871 2024-09-16T14:49:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997374+0000", 2024-09-16T14:49:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8307999999999996e-05 2024-09-16T14:49:03.359 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.360 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.360 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.360 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.001050+0000", 2024-09-16T14:49:03.360 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036758770000000001 2024-09-16T14:49:03.360 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.360 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.360 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.360 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.360 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.360 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s5 52/27 ECSubWrite(tid=29, reqid=osd.4.0:29, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:03.360 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:58.889907+0000", 2024-09-16T14:49:03.360 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 4.3955937970000001, 2024-09-16T14:49:03.360 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0051965099999999997, 2024-09-16T14:49:03.361 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.361 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.361 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.361 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.361 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.361 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.889907+0000", 2024-09-16T14:49:03.361 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.361 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.361 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.361 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.361 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.889907+0000", 2024-09-16T14:49:03.361 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.361 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.361 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.362 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.362 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.889906+0000", 2024-09-16T14:49:03.362 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:03.362 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.362 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.362 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.362 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.889934+0000", 2024-09-16T14:49:03.362 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7586e-05 2024-09-16T14:49:03.362 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.362 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.362 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.362 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.889943+0000", 2024-09-16T14:49:03.362 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2709999999999998e-06 2024-09-16T14:49:03.362 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.363 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.363 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.363 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.889954+0000", 2024-09-16T14:49:03.363 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0643e-05 2024-09-16T14:49:03.363 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.363 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.363 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.363 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.890027+0000", 2024-09-16T14:49:03.363 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2835000000000003e-05 2024-09-16T14:49:03.363 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.363 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.363 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.363 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.890038+0000", 2024-09-16T14:49:03.364 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1564e-05 2024-09-16T14:49:03.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.364 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.364 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.364 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.895077+0000", 2024-09-16T14:49:03.364 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050386140000000003 2024-09-16T14:49:03.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.364 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.364 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.364 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.895103+0000", 2024-09-16T14:49:03.364 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6233000000000001e-05 2024-09-16T14:49:03.364 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.364 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.364 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.9s4 52/27 ECSubWrite(tid=38, reqid=osd.8.0:37, at_version=0'0, trim_to=0'0, roll_forward_to=52'23))", 2024-09-16T14:49:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:58.892213+0000", 2024-09-16T14:49:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 4.3932873929999996, 2024-09-16T14:49:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005572039, 2024-09-16T14:49:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892213+0000", 2024-09-16T14:49:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.366 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.366 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.366 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892213+0000", 2024-09-16T14:49:03.366 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.366 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.366 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.366 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.366 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892212+0000", 2024-09-16T14:49:03.366 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:03.366 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.366 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.366 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.366 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892221+0000", 2024-09-16T14:49:03.366 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9670000000000007e-06 2024-09-16T14:49:03.367 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.367 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.367 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.367 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892232+0000", 2024-09-16T14:49:03.367 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0651e-05 2024-09-16T14:49:03.367 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.367 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.367 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.367 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892240+0000", 2024-09-16T14:49:03.367 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0169999999999996e-06 2024-09-16T14:49:03.367 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.367 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.367 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.367 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892287+0000", 2024-09-16T14:49:03.368 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7119000000000001e-05 2024-09-16T14:49:03.368 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.368 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.368 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.368 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892295+0000", 2024-09-16T14:49:03.368 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.1040000000000007e-06 2024-09-16T14:49:03.368 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.368 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.368 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.368 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.897763+0000", 2024-09-16T14:49:03.368 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0054674019999999997 2024-09-16T14:49:03.368 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.368 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.368 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.369 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.897785+0000", 2024-09-16T14:49:03.369 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2504999999999999e-05 2024-09-16T14:49:03.369 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.369 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.369 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.369 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.369 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.369 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 52/27 ECSubWrite(tid=185, reqid=osd.11.0:185, at_version=52'62, trim_to=0'0, roll_forward_to=52'60))", 2024-09-16T14:49:03.369 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:58.900768+0000", 2024-09-16T14:49:03.369 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 4.3847324209999998, 2024-09-16T14:49:03.369 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083307420000000004, 2024-09-16T14:49:03.369 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.369 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.369 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.370 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.370 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.370 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900768+0000", 2024-09-16T14:49:03.370 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.370 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.370 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.370 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.370 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900768+0000", 2024-09-16T14:49:03.370 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.370 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.370 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.370 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.370 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900767+0000", 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900790+0000", 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2812000000000001e-05 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900801+0000", 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0954999999999999e-05 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.371 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900810+0000", 2024-09-16T14:49:03.372 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5019999999999992e-06 2024-09-16T14:49:03.372 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.372 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.372 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.372 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900851+0000", 2024-09-16T14:49:03.372 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1845999999999999e-05 2024-09-16T14:49:03.372 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.372 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.372 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.372 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900860+0000", 2024-09-16T14:49:03.372 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4630000000000001e-06 2024-09-16T14:49:03.372 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.372 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.372 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909074+0000", 2024-09-16T14:49:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082140349999999997 2024-09-16T14:49:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909099+0000", 2024-09-16T14:49:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4808e-05 2024-09-16T14:49:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds1 52/27 ECSubWrite(tid=215, reqid=osd.7.0:214, at_version=0'0, trim_to=0'0, roll_forward_to=52'35))", 2024-09-16T14:49:03.373 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:58.901786+0000", 2024-09-16T14:49:03.374 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 4.3837144889999999, 2024-09-16T14:49:03.374 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0073522930000000002, 2024-09-16T14:49:03.374 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.374 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.374 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.374 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.374 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.374 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901786+0000", 2024-09-16T14:49:03.374 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.374 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.374 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.374 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.374 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901786+0000", 2024-09-16T14:49:03.374 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.375 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.375 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.375 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.375 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901785+0000", 2024-09-16T14:49:03.375 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:03.375 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.375 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.375 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.375 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901796+0000", 2024-09-16T14:49:03.375 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0475000000000001e-05 2024-09-16T14:49:03.375 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.375 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.375 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.375 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901808+0000", 2024-09-16T14:49:03.376 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2755000000000001e-05 2024-09-16T14:49:03.376 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.376 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.376 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.376 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901817+0000", 2024-09-16T14:49:03.376 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7409999999999998e-06 2024-09-16T14:49:03.376 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.376 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.376 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.376 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901871+0000", 2024-09-16T14:49:03.376 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3940000000000002e-05 2024-09-16T14:49:03.376 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.376 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.376 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.377 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901880+0000", 2024-09-16T14:49:03.377 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2539999999999998e-06 2024-09-16T14:49:03.377 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.377 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.377 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.377 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909107+0000", 2024-09-16T14:49:03.377 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00722684 2024-09-16T14:49:03.377 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.377 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.377 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.377 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909138+0000", 2024-09-16T14:49:03.377 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.112e-05 2024-09-16T14:49:03.377 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.377 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.378 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.378 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.378 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.378 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s5 52/27 ECSubWrite(tid=30, reqid=osd.4.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=52'20))", 2024-09-16T14:49:03.378 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:58.903955+0000", 2024-09-16T14:49:03.378 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 4.3815454159999998, 2024-09-16T14:49:03.378 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0059277990000000001, 2024-09-16T14:49:03.378 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.378 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.378 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.378 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.378 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.378 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903955+0000", 2024-09-16T14:49:03.378 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.379 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.379 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.379 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.379 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903955+0000", 2024-09-16T14:49:03.379 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.379 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.379 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.379 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.379 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903954+0000", 2024-09-16T14:49:03.379 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:03.379 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.379 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.379 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.379 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903965+0000", 2024-09-16T14:49:03.380 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0728e-05 2024-09-16T14:49:03.380 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.380 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.380 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.380 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903981+0000", 2024-09-16T14:49:03.380 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6166000000000001e-05 2024-09-16T14:49:03.380 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.380 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.380 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.380 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903989+0000", 2024-09-16T14:49:03.380 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0809999999999994e-06 2024-09-16T14:49:03.380 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.380 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.380 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.381 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.904038+0000", 2024-09-16T14:49:03.381 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8528e-05 2024-09-16T14:49:03.381 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.381 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.381 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.381 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.904047+0000", 2024-09-16T14:49:03.381 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5049999999999996e-06 2024-09-16T14:49:03.381 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.381 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.381 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.381 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909865+0000", 2024-09-16T14:49:03.381 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0058180790000000003 2024-09-16T14:49:03.381 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.381 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.382 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.382 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909883+0000", 2024-09-16T14:49:03.382 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7495000000000001e-05 2024-09-16T14:49:03.382 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.382 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.382 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.382 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.382 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.382 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:03.384 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:49:03.384 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:03.383+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:03.485 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:03.484+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:03.527 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:03.527 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:03.527 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:03.527 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:03.535 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:49:03.585 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:03.584+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:03.637 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:03.638 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:03.638 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:03.638 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:03.638 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:03.646 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:49:03.690 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:03.689+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:03.751 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:03.751 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:03.751 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:03.751 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:03.751 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.751 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:49:03.751 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:03.751 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 41.608898727000003, 2024-09-16T14:49:03.751 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.223181782, 2024-09-16T14:49:03.751 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.751 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.751 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.752 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.752 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.752 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 1 2024-09-16T14:49:03.752 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.752 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.752 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.752 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.752 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:03.752 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.752 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.752 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.753 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.753 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:03.753 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.753 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.753 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.753 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.753 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139495+0000", 2024-09-16T14:49:03.753 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:03.753 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.753 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.753 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.753 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139504+0000", 2024-09-16T14:49:03.753 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6119999999999999e-06 2024-09-16T14:49:03.754 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.754 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.754 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.754 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139514+0000", 2024-09-16T14:49:03.754 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:49:03.754 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.754 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.754 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.754 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139533+0000", 2024-09-16T14:49:03.754 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9573e-05 2024-09-16T14:49:03.754 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.754 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.755 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.755 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139585+0000", 2024-09-16T14:49:03.755 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2170999999999999e-05 2024-09-16T14:49:03.755 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.755 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.755 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:03.755 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139594+0000", 2024-09-16T14:49:03.755 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.884e-06 2024-09-16T14:49:03.755 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.755 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.755 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.755 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359233+0000", 2024-09-16T14:49:03.755 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.21963890899999999 2024-09-16T14:49:03.756 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.756 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.756 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.756 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359437+0000", 2024-09-16T14:49:03.756 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00020374999999999999 2024-09-16T14:49:03.756 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.756 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.756 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.756 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359611+0000", 2024-09-16T14:49:03.756 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017445899999999999 2024-09-16T14:49:03.756 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.756 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.756 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.757 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.360506+0000", 2024-09-16T14:49:03.757 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00089459299999999995 2024-09-16T14:49:03.757 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.757 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.757 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.757 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362594+0000", 2024-09-16T14:49:03.757 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002087776 2024-09-16T14:49:03.757 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.757 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.757 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.757 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362636+0000", 2024-09-16T14:49:03.757 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1933e-05 2024-09-16T14:49:03.760 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.760 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.760 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.760 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362679+0000", 2024-09-16T14:49:03.761 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2877999999999998e-05 2024-09-16T14:49:03.761 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.761 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.761 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.761 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.761 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.761 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:49:03.761 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:49:03.761 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 41.360187320000001, 2024-09-16T14:49:03.761 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077691490000000004, 2024-09-16T14:49:03.761 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.761 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.761 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.762 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.762 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.762 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 21 2024-09-16T14:49:03.762 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.762 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.762 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.762 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.762 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:49:03.762 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.762 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.762 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.763 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.763 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:49:03.763 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.763 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.763 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.763 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.763 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388208+0000", 2024-09-16T14:49:03.763 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.763 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.763 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.763 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.763 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388278+0000", 2024-09-16T14:49:03.764 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0809000000000006e-05 2024-09-16T14:49:03.764 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.764 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.764 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.764 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388286+0000", 2024-09-16T14:49:03.764 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.204e-06 2024-09-16T14:49:03.764 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.764 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.764 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.764 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388291+0000", 2024-09-16T14:49:03.764 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9349999999999999e-06 2024-09-16T14:49:03.765 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.765 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.765 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.765 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388713+0000", 2024-09-16T14:49:03.765 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00042189899999999999 2024-09-16T14:49:03.765 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.765 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.765 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.765 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388766+0000", 2024-09-16T14:49:03.765 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2750999999999997e-05 2024-09-16T14:49:03.765 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.765 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.765 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.765 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388853+0000", 2024-09-16T14:49:03.766 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6635e-05 2024-09-16T14:49:03.766 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.766 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.766 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.766 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.390635+0000", 2024-09-16T14:49:03.766 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0017817950000000001 2024-09-16T14:49:03.766 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.766 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.766 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.766 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.395898+0000", 2024-09-16T14:49:03.766 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0052630510000000004 2024-09-16T14:49:03.767 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.767 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.767 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.767 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.395941+0000", 2024-09-16T14:49:03.767 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3248999999999999e-05 2024-09-16T14:49:03.767 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.767 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.767 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.767 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.395977+0000", 2024-09-16T14:49:03.767 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6501000000000002e-05 2024-09-16T14:49:03.767 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.768 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.768 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.768 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.768 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.768 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:49:03.768 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:49:03.768 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 41.360096312000003, 2024-09-16T14:49:03.768 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077998870000000001, 2024-09-16T14:49:03.768 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.768 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.768 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.768 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.769 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.769 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 22 2024-09-16T14:49:03.769 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.769 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.769 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.769 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.769 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:49:03.769 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.769 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.769 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.769 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.769 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:49:03.769 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.770 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.770 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.770 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.770 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388299+0000", 2024-09-16T14:49:03.770 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.770 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.770 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.770 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.770 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388311+0000", 2024-09-16T14:49:03.770 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2853e-05 2024-09-16T14:49:03.770 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.770 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.770 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.771 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388318+0000", 2024-09-16T14:49:03.771 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7549999999999997e-06 2024-09-16T14:49:03.771 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.771 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.771 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.771 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.388322+0000", 2024-09-16T14:49:03.771 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5889999999999999e-06 2024-09-16T14:49:03.771 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.771 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.771 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.771 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.389160+0000", 2024-09-16T14:49:03.771 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000837877 2024-09-16T14:49:03.771 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.771 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.772 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.772 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.389230+0000", 2024-09-16T14:49:03.772 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0551999999999998e-05 2024-09-16T14:49:03.772 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.772 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.772 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.772 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.389371+0000", 2024-09-16T14:49:03.772 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014102099999999999 2024-09-16T14:49:03.772 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.772 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.772 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.772 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.390881+0000", 2024-09-16T14:49:03.773 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00151004 2024-09-16T14:49:03.773 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.773 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.773 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.773 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.396022+0000", 2024-09-16T14:49:03.773 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005140716 2024-09-16T14:49:03.773 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.773 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.773 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.773 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.396074+0000", 2024-09-16T14:49:03.773 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2089999999999998e-05 2024-09-16T14:49:03.773 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.773 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.773 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.774 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.396099+0000", 2024-09-16T14:49:03.774 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5038e-05 2024-09-16T14:49:03.774 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.774 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.774 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.774 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.774 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.774 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:49:03.774 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:49:03.774 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 41.341709436000002, 2024-09-16T14:49:03.774 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0072029349999999997, 2024-09-16T14:49:03.774 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.774 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.775 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.775 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.775 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.775 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 36 2024-09-16T14:49:03.775 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.775 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.775 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.775 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.775 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:49:03.775 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.775 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.775 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.775 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.775 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:49:03.776 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.776 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.776 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.776 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.776 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406686+0000", 2024-09-16T14:49:03.776 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.776 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.776 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.776 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.776 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406695+0000", 2024-09-16T14:49:03.776 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1530000000000003e-06 2024-09-16T14:49:03.776 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.776 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.777 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.777 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406700+0000", 2024-09-16T14:49:03.777 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7079999999999996e-06 2024-09-16T14:49:03.777 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.777 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.777 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.777 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406702+0000", 2024-09-16T14:49:03.777 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7410000000000001e-06 2024-09-16T14:49:03.777 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.777 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.777 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.777 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409398+0000", 2024-09-16T14:49:03.777 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0026957420000000001 2024-09-16T14:49:03.778 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.778 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.778 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.778 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409441+0000", 2024-09-16T14:49:03.778 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3071999999999997e-05 2024-09-16T14:49:03.778 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.778 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.778 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.778 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409512+0000", 2024-09-16T14:49:03.778 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0871999999999995e-05 2024-09-16T14:49:03.778 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.778 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.778 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.779 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.410156+0000", 2024-09-16T14:49:03.779 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00064395800000000003 2024-09-16T14:49:03.779 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.779 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.779 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.779 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413850+0000", 2024-09-16T14:49:03.779 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036938959999999999 2024-09-16T14:49:03.779 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.779 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.779 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.779 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413872+0000", 2024-09-16T14:49:03.779 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2195e-05 2024-09-16T14:49:03.779 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.780 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.780 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.780 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413889+0000", 2024-09-16T14:49:03.780 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6948000000000001e-05 2024-09-16T14:49:03.780 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.780 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.780 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.780 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.780 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.780 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:49:03.780 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.406706+0000", 2024-09-16T14:49:03.780 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 41.341689858000002, 2024-09-16T14:49:03.780 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0072520049999999997, 2024-09-16T14:49:03.781 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.781 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.781 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.781 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.781 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.781 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 37 2024-09-16T14:49:03.781 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.781 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.781 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.781 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.781 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406706+0000", 2024-09-16T14:49:03.781 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.781 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.781 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.782 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.782 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406706+0000", 2024-09-16T14:49:03.782 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.782 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.782 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.782 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.782 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406705+0000", 2024-09-16T14:49:03.782 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.782 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.782 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.782 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.782 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406708+0000", 2024-09-16T14:49:03.782 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.57e-06 2024-09-16T14:49:03.782 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.783 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.783 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.783 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406712+0000", 2024-09-16T14:49:03.783 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.425e-06 2024-09-16T14:49:03.783 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.783 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.783 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.783 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.406715+0000", 2024-09-16T14:49:03.783 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7700000000000002e-06 2024-09-16T14:49:03.783 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.783 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.783 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.783 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409721+0000", 2024-09-16T14:49:03.784 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003006327 2024-09-16T14:49:03.784 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.784 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.784 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.784 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409770+0000", 2024-09-16T14:49:03.784 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8022000000000002e-05 2024-09-16T14:49:03.784 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.784 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.784 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.784 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.409853+0000", 2024-09-16T14:49:03.784 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3604999999999997e-05 2024-09-16T14:49:03.784 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.784 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.784 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.785 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.410391+0000", 2024-09-16T14:49:03.785 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00053785400000000004 2024-09-16T14:49:03.785 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.785 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.785 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.785 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413911+0000", 2024-09-16T14:49:03.785 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0035205240000000001 2024-09-16T14:49:03.785 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.785 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.785 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.785 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413939+0000", 2024-09-16T14:49:03.785 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7032000000000001e-05 2024-09-16T14:49:03.785 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.786 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.786 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.786 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.413958+0000", 2024-09-16T14:49:03.786 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9147000000000002e-05 2024-09-16T14:49:03.786 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.786 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.786 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.786 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.786 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.786 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:49:03.786 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.620638+0000", 2024-09-16T14:49:03.786 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 41.127757754999998, 2024-09-16T14:49:03.787 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0078265469999999997, 2024-09-16T14:49:03.787 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.787 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.787 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.787 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.787 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.787 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 116 2024-09-16T14:49:03.787 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.787 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.787 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.787 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.787 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620638+0000", 2024-09-16T14:49:03.787 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.787 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.788 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.788 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.788 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620638+0000", 2024-09-16T14:49:03.788 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.788 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.788 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.788 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.788 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620637+0000", 2024-09-16T14:49:03.788 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.788 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.788 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.788 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.788 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620641+0000", 2024-09-16T14:49:03.788 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9360000000000001e-06 2024-09-16T14:49:03.789 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.789 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.789 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.789 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620647+0000", 2024-09-16T14:49:03.789 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3339999999999998e-06 2024-09-16T14:49:03.789 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.789 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.789 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.789 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620652+0000", 2024-09-16T14:49:03.789 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0950000000000003e-06 2024-09-16T14:49:03.789 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.789 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.789 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.790 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625724+0000", 2024-09-16T14:49:03.790 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050719370000000003 2024-09-16T14:49:03.790 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.790 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.790 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.790 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625768+0000", 2024-09-16T14:49:03.790 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4039000000000003e-05 2024-09-16T14:49:03.790 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.790 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.790 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.790 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.625836+0000", 2024-09-16T14:49:03.790 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7181999999999995e-05 2024-09-16T14:49:03.790 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.791 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.791 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.791 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627424+0000", 2024-09-16T14:49:03.791 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015887119999999999 2024-09-16T14:49:03.791 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.791 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.791 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.791 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628421+0000", 2024-09-16T14:49:03.791 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00099648699999999989 2024-09-16T14:49:03.791 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.791 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.791 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.792 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628446+0000", 2024-09-16T14:49:03.792 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5307999999999999e-05 2024-09-16T14:49:03.792 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.792 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.792 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.792 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628464+0000", 2024-09-16T14:49:03.792 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8170999999999999e-05 2024-09-16T14:49:03.792 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.792 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.792 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.792 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.792 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.792 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:49:03.793 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:49:03.793 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 41.127738031, 2024-09-16T14:49:03.793 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0079223479999999992, 2024-09-16T14:49:03.793 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.793 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.793 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.793 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.793 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.793 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 117 2024-09-16T14:49:03.793 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.793 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.793 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.793 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.793 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:49:03.794 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.794 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.794 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.794 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.794 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:49:03.794 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.794 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.794 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.794 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.794 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620657+0000", 2024-09-16T14:49:03.794 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.794 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.794 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.795 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.795 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620662+0000", 2024-09-16T14:49:03.795 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7609999999999998e-06 2024-09-16T14:49:03.795 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.795 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.795 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.795 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620668+0000", 2024-09-16T14:49:03.795 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.84e-06 2024-09-16T14:49:03.795 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.795 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.795 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.795 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.620673+0000", 2024-09-16T14:49:03.795 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.468e-06 2024-09-16T14:49:03.795 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.796 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.796 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.796 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.626022+0000", 2024-09-16T14:49:03.796 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053484680000000003 2024-09-16T14:49:03.796 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.796 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.796 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.796 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.626071+0000", 2024-09-16T14:49:03.796 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9854000000000001e-05 2024-09-16T14:49:03.796 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.796 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.796 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.796 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.626145+0000", 2024-09-16T14:49:03.796 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.3159999999999997e-05 2024-09-16T14:49:03.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.797 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.797 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.797 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.627426+0000", 2024-09-16T14:49:03.797 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001281847 2024-09-16T14:49:03.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.797 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.797 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.797 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628501+0000", 2024-09-16T14:49:03.797 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0010748019999999999 2024-09-16T14:49:03.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.797 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.797 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.798 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628541+0000", 2024-09-16T14:49:03.798 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9598999999999998e-05 2024-09-16T14:49:03.798 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.798 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.798 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.798 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.628580+0000", 2024-09-16T14:49:03.798 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8977999999999998e-05 2024-09-16T14:49:03.798 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.798 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.798 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.798 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.798 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.798 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:03.798 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:49:03.799 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.356471138000003, 2024-09-16T14:49:03.799 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0081240189999999997, 2024-09-16T14:49:03.799 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.799 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.799 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.799 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.799 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.799 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 158 2024-09-16T14:49:03.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.799 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.799 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.799 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.799 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:49:03.800 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.800 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.800 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.800 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:49:03.800 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.800 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.800 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.800 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.391924+0000", 2024-09-16T14:49:03.800 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.800 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.800 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.801 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392007+0000", 2024-09-16T14:49:03.801 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3108000000000005e-05 2024-09-16T14:49:03.801 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.801 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.801 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.801 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392014+0000", 2024-09-16T14:49:03.801 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6170000000000003e-06 2024-09-16T14:49:03.801 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.801 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.801 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.801 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392017+0000", 2024-09-16T14:49:03.801 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3589999999999999e-06 2024-09-16T14:49:03.801 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394165+0000", 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021481270000000001 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394238+0000", 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2366999999999993e-05 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394347+0000", 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000109907 2024-09-16T14:49:03.802 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.803 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.803 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.803 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396066+0000", 2024-09-16T14:49:03.803 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001718731 2024-09-16T14:49:03.803 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.803 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.803 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.803 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.399984+0000", 2024-09-16T14:49:03.803 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0039181850000000002 2024-09-16T14:49:03.803 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.803 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.803 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.803 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400027+0000", 2024-09-16T14:49:03.804 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2184000000000001e-05 2024-09-16T14:49:03.804 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.804 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.804 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.804 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400048+0000", 2024-09-16T14:49:03.804 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1897999999999999e-05 2024-09-16T14:49:03.804 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.804 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.804 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.804 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.804 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.804 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:03.805 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:49:03.805 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.356374344000002, 2024-09-16T14:49:03.805 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0081462830000000007, 2024-09-16T14:49:03.805 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.805 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.805 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.805 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.805 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.805 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 159 2024-09-16T14:49:03.805 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.805 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.805 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.805 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.806 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:49:03.806 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.806 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.806 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.806 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.806 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:49:03.806 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.806 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.806 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.806 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.806 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:49:03.806 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.806 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.806 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392023+0000", 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392035+0000", 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2207e-05 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392040+0000", 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7450000000000003e-06 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.807 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.808 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.808 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394756+0000", 2024-09-16T14:49:03.808 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0027154010000000001 2024-09-16T14:49:03.808 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.808 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.808 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.808 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394799+0000", 2024-09-16T14:49:03.808 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3921999999999997e-05 2024-09-16T14:49:03.808 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.808 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.808 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.808 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394867+0000", 2024-09-16T14:49:03.808 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7770999999999999e-05 2024-09-16T14:49:03.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.809 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.809 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396922+0000", 2024-09-16T14:49:03.809 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0020549819999999999 2024-09-16T14:49:03.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.809 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.809 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400094+0000", 2024-09-16T14:49:03.809 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031720979999999999 2024-09-16T14:49:03.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.809 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.810 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400134+0000", 2024-09-16T14:49:03.810 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9449999999999997e-05 2024-09-16T14:49:03.810 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.810 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.810 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.810 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400167+0000", 2024-09-16T14:49:03.810 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3711000000000003e-05 2024-09-16T14:49:03.810 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.810 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.810 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.810 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.810 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.810 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:160 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:03.810 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:49:03.811 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.356352111, 2024-09-16T14:49:03.811 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082045659999999999, 2024-09-16T14:49:03.811 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.811 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.811 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.811 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.811 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.811 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 160 2024-09-16T14:49:03.811 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.811 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.811 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.811 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.811 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:49:03.812 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.812 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.812 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.812 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.812 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:49:03.812 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.812 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.812 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.812 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.812 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:49:03.812 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.812 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.812 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.813 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.813 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392045+0000", 2024-09-16T14:49:03.813 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.283e-06 2024-09-16T14:49:03.813 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.813 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.813 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.813 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392050+0000", 2024-09-16T14:49:03.813 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1099999999999996e-06 2024-09-16T14:49:03.813 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.813 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.813 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.813 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392052+0000", 2024-09-16T14:49:03.814 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4660000000000002e-06 2024-09-16T14:49:03.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.814 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.814 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.814 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395234+0000", 2024-09-16T14:49:03.814 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031823260000000001 2024-09-16T14:49:03.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.814 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.814 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.814 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395283+0000", 2024-09-16T14:49:03.814 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8347000000000003e-05 2024-09-16T14:49:03.815 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.815 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.815 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.815 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395364+0000", 2024-09-16T14:49:03.815 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0853999999999995e-05 2024-09-16T14:49:03.815 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.815 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.815 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.815 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396928+0000", 2024-09-16T14:49:03.815 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015642239999999999 2024-09-16T14:49:03.815 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.815 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.815 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.816 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400202+0000", 2024-09-16T14:49:03.816 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032747359999999999 2024-09-16T14:49:03.816 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.816 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.816 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.816 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400227+0000", 2024-09-16T14:49:03.816 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4079000000000001e-05 2024-09-16T14:49:03.816 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.816 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.816 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.816 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400248+0000", 2024-09-16T14:49:03.816 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1444e-05 2024-09-16T14:49:03.817 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.817 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.817 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.817 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.817 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.817 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:161 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:03.817 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:49:03.817 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.356340676000002, 2024-09-16T14:49:03.817 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082675720000000008, 2024-09-16T14:49:03.817 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.817 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.817 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.817 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.818 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.818 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 161 2024-09-16T14:49:03.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.818 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.818 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.818 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.818 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:49:03.818 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.818 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.818 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.818 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:49:03.818 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.819 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.819 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.819 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:49:03.819 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:03.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.819 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.819 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.819 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392122+0000", 2024-09-16T14:49:03.819 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7698000000000001e-05 2024-09-16T14:49:03.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.819 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.819 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.819 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392136+0000", 2024-09-16T14:49:03.820 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3464e-05 2024-09-16T14:49:03.820 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.820 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.820 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.820 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392140+0000", 2024-09-16T14:49:03.820 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0790000000000002e-06 2024-09-16T14:49:03.820 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.820 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.820 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.820 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395579+0000", 2024-09-16T14:49:03.820 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034393219999999999 2024-09-16T14:49:03.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.821 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.821 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.821 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395624+0000", 2024-09-16T14:49:03.821 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4946e-05 2024-09-16T14:49:03.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.821 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.821 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.821 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395692+0000", 2024-09-16T14:49:03.821 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7843999999999996e-05 2024-09-16T14:49:03.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.821 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.821 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.821 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397834+0000", 2024-09-16T14:49:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021421529999999999 2024-09-16T14:49:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400281+0000", 2024-09-16T14:49:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002447238 2024-09-16T14:49:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400305+0000", 2024-09-16T14:49:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3428999999999999e-05 2024-09-16T14:49:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.823 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400322+0000", 2024-09-16T14:49:03.823 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7632e-05 2024-09-16T14:49:03.823 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.823 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.823 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.823 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.823 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.823 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:03.823 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:49:03.823 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.356251397000001, 2024-09-16T14:49:03.823 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.008260814, 2024-09-16T14:49:03.823 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.823 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.824 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.824 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.824 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.824 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 162 2024-09-16T14:49:03.824 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.824 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.824 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.824 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.824 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:49:03.824 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.824 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.824 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.824 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.825 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:49:03.825 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.825 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.825 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.825 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.825 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:49:03.825 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.825 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.825 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.825 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.825 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392146+0000", 2024-09-16T14:49:03.825 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.526e-06 2024-09-16T14:49:03.825 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.826 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.826 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.826 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392151+0000", 2024-09-16T14:49:03.826 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2980000000000002e-06 2024-09-16T14:49:03.826 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.826 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.826 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.826 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392153+0000", 2024-09-16T14:49:03.826 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5409999999999999e-06 2024-09-16T14:49:03.826 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.826 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.826 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.826 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396103+0000", 2024-09-16T14:49:03.826 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0039497559999999996 2024-09-16T14:49:03.827 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.827 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.827 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.827 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396188+0000", 2024-09-16T14:49:03.827 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5038999999999997e-05 2024-09-16T14:49:03.827 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.827 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.827 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.827 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396315+0000", 2024-09-16T14:49:03.827 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000126895 2024-09-16T14:49:03.827 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.827 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.827 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.827 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397839+0000", 2024-09-16T14:49:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015246859999999999 2024-09-16T14:49:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400356+0000", 2024-09-16T14:49:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002516524 2024-09-16T14:49:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400379+0000", 2024-09-16T14:49:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2793e-05 2024-09-16T14:49:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.828 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.829 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400405+0000", 2024-09-16T14:49:03.829 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6140999999999999e-05 2024-09-16T14:49:03.829 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.829 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.829 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.829 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.829 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.829 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:03.829 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:49:03.829 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.356240006, 2024-09-16T14:49:03.829 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083183530000000006, 2024-09-16T14:49:03.829 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.829 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.830 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.830 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.830 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.830 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 163 2024-09-16T14:49:03.830 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.830 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.830 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.830 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.830 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:49:03.830 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.830 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.830 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.830 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.830 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:49:03.831 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.831 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.831 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.831 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392155+0000", 2024-09-16T14:49:03.831 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:03.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.831 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.831 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.831 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392159+0000", 2024-09-16T14:49:03.831 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0459999999999999e-06 2024-09-16T14:49:03.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.831 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.831 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.832 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392162+0000", 2024-09-16T14:49:03.832 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:49:03.832 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.832 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.832 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.832 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392165+0000", 2024-09-16T14:49:03.832 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.424e-06 2024-09-16T14:49:03.832 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.832 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.832 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.832 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396578+0000", 2024-09-16T14:49:03.832 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044136139999999997 2024-09-16T14:49:03.832 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.832 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.833 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.833 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396625+0000", 2024-09-16T14:49:03.833 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6486000000000002e-05 2024-09-16T14:49:03.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.833 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.833 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.833 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396694+0000", 2024-09-16T14:49:03.833 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9049999999999998e-05 2024-09-16T14:49:03.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.833 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.833 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.833 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397843+0000", 2024-09-16T14:49:03.833 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011485740000000001 2024-09-16T14:49:03.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.834 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.834 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.834 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400438+0000", 2024-09-16T14:49:03.834 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025950909999999999 2024-09-16T14:49:03.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.834 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.834 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.834 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400458+0000", 2024-09-16T14:49:03.834 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0248e-05 2024-09-16T14:49:03.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.834 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.834 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.834 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400474+0000", 2024-09-16T14:49:03.835 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5999999999999999e-05 2024-09-16T14:49:03.835 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.835 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.835 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.835 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.835 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.835 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:03.835 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:49:03.835 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.356228409000003, 2024-09-16T14:49:03.835 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083881519999999994, 2024-09-16T14:49:03.835 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.835 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.835 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.836 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.836 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.836 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 164 2024-09-16T14:49:03.836 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.836 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.836 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.836 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.836 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:49:03.836 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.836 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.836 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.836 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.836 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:49:03.836 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.837 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.837 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.837 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.837 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:49:03.837 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:03.837 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.837 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.837 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.837 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392212+0000", 2024-09-16T14:49:03.837 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5108999999999998e-05 2024-09-16T14:49:03.837 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.837 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.837 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.837 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392216+0000", 2024-09-16T14:49:03.838 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2549999999999999e-06 2024-09-16T14:49:03.838 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.838 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.838 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.838 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392220+0000", 2024-09-16T14:49:03.838 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3850000000000002e-06 2024-09-16T14:49:03.838 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.838 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.838 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.838 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396961+0000", 2024-09-16T14:49:03.838 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.004740976 2024-09-16T14:49:03.838 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.839 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.839 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.839 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397029+0000", 2024-09-16T14:49:03.839 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7889e-05 2024-09-16T14:49:03.839 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.839 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.839 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.839 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397131+0000", 2024-09-16T14:49:03.839 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000102315 2024-09-16T14:49:03.839 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.839 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.839 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.839 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400259+0000", 2024-09-16T14:49:03.839 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031285229999999998 2024-09-16T14:49:03.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.840 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.840 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.840 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400501+0000", 2024-09-16T14:49:03.840 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00024211999999999999 2024-09-16T14:49:03.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.840 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.840 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.840 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400530+0000", 2024-09-16T14:49:03.840 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8935999999999999e-05 2024-09-16T14:49:03.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.840 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.840 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.841 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400555+0000", 2024-09-16T14:49:03.841 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4861000000000001e-05 2024-09-16T14:49:03.841 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.841 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.841 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.841 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.841 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.841 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:03.841 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:49:03.841 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.356172643999997, 2024-09-16T14:49:03.841 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010593673, 2024-09-16T14:49:03.841 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.841 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.841 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.842 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:03.842 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:03.842 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 165 2024-09-16T14:49:03.842 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.842 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.842 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.842 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.842 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:49:03.842 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.842 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.842 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.842 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.842 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:49:03.842 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.843 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.843 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.843 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.843 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:49:03.843 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.843 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.843 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.843 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.843 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392226+0000", 2024-09-16T14:49:03.843 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2789999999999999e-06 2024-09-16T14:49:03.843 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.843 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.843 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.844 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392234+0000", 2024-09-16T14:49:03.844 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2369999999999992e-06 2024-09-16T14:49:03.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.844 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.844 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.844 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392237+0000", 2024-09-16T14:49:03.844 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8619999999999999e-06 2024-09-16T14:49:03.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.844 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.844 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.844 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397503+0000", 2024-09-16T14:49:03.845 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0052656220000000002 2024-09-16T14:49:03.845 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.845 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.845 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.845 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397558+0000", 2024-09-16T14:49:03.845 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5386999999999997e-05 2024-09-16T14:49:03.845 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.845 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.845 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:03.845 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397628+0000", 2024-09-16T14:49:03.845 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0119999999999999e-05 2024-09-16T14:49:03.845 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.846 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.846 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:03.846 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402533+0000", 2024-09-16T14:49:03.846 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049046200000000002 2024-09-16T14:49:03.846 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.846 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.846 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:03.846 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402778+0000", 2024-09-16T14:49:03.846 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000245348 2024-09-16T14:49:03.846 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.846 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.846 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.846 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402800+0000", 2024-09-16T14:49:03.846 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2277e-05 2024-09-16T14:49:03.847 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.847 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.847 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.847 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402817+0000", 2024-09-16T14:49:03.847 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6265000000000001e-05 2024-09-16T14:49:03.847 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.847 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.847 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.847 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.847 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.847 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:309 3.ds0 3:b9c0d236:::smithi04040151-19:head [append 0~737280] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:03.847 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:49:03.847 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.808981507999999, 2024-09-16T14:49:03.847 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01003656, 2024-09-16T14:49:03.848 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.848 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.848 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.848 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:03.848 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:03.848 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 309 2024-09-16T14:49:03.848 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.848 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.848 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.848 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.848 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:49:03.848 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.848 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.849 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.849 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.849 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:49:03.849 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.849 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.849 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.849 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.849 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939413+0000", 2024-09-16T14:49:03.849 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:03.849 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.849 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.849 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.849 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942226+0000", 2024-09-16T14:49:03.850 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0028130450000000001 2024-09-16T14:49:03.850 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.850 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.850 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.850 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942238+0000", 2024-09-16T14:49:03.850 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2639e-05 2024-09-16T14:49:03.850 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.850 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.850 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.850 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942245+0000", 2024-09-16T14:49:03.850 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3840000000000002e-06 2024-09-16T14:49:03.850 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.850 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.850 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.851 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942288+0000", 2024-09-16T14:49:03.851 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3186000000000003e-05 2024-09-16T14:49:03.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.851 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.851 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.851 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942368+0000", 2024-09-16T14:49:03.851 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0162999999999998e-05 2024-09-16T14:49:03.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.851 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.851 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.851 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943660+0000", 2024-09-16T14:49:03.851 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001291698 2024-09-16T14:49:03.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.852 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.852 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.852 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.947440+0000", 2024-09-16T14:49:03.852 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003780079 2024-09-16T14:49:03.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.852 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.852 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.852 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.949391+0000", 2024-09-16T14:49:03.852 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0019512990000000001 2024-09-16T14:49:03.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.852 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.852 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.853 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.949451+0000", 2024-09-16T14:49:03.853 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9404000000000002e-05 2024-09-16T14:49:03.853 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.853 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.853 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.853 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.853 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.853 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:312 3.ds0 3:b9c0d236:::smithi04040151-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:03.853 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:49:03.853 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.805001993000001, 2024-09-16T14:49:03.853 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084967020000000001, 2024-09-16T14:49:03.853 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.853 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:03.853 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.854 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:03.854 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:03.854 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 312 2024-09-16T14:49:03.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.854 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.854 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.854 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:49:03.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.854 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.854 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:49:03.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.855 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.855 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.855 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.855 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943393+0000", 2024-09-16T14:49:03.855 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.855 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.855 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.855 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.855 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943396+0000", 2024-09-16T14:49:03.855 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6079999999999998e-06 2024-09-16T14:49:03.855 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.855 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.855 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943402+0000", 2024-09-16T14:49:03.856 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6100000000000002e-06 2024-09-16T14:49:03.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943413+0000", 2024-09-16T14:49:03.856 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1178e-05 2024-09-16T14:49:03.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.946818+0000", 2024-09-16T14:49:03.856 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034043049999999998 2024-09-16T14:49:03.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.857 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:03.857 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.946922+0000", 2024-09-16T14:49:03.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000104076 2024-09-16T14:49:03.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.857 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.857 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.950275+0000", 2024-09-16T14:49:03.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0033530220000000002 2024-09-16T14:49:03.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.857 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.857 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.950357+0000", 2024-09-16T14:49:03.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2087000000000002e-05 2024-09-16T14:49:03.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.951890+0000", 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001533333 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:390 3.ds0 3:b855c7ac:::smithi04040151-47:head [append 0~0] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:53.864520+0000", 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 9.883875797, 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089883559999999994, 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.858 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.859 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.859 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:03.859 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:03.859 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 390 2024-09-16T14:49:03.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.859 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.859 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.859 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.859 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864520+0000", 2024-09-16T14:49:03.859 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.859 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.859 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.859 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864520+0000", 2024-09-16T14:49:03.860 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.860 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.860 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.860 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.860 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864519+0000", 2024-09-16T14:49:03.860 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.860 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.860 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.860 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.860 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:03.860 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0545e-05 2024-09-16T14:49:03.860 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.860 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864536+0000", 2024-09-16T14:49:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8069999999999996e-06 2024-09-16T14:49:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864541+0000", 2024-09-16T14:49:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0900000000000004e-06 2024-09-16T14:49:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.866852+0000", 2024-09-16T14:49:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0023115399999999999 2024-09-16T14:49:03.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.866946+0000", 2024-09-16T14:49:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3311000000000002e-05 2024-09-16T14:49:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.867413+0000", 2024-09-16T14:49:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00046786499999999998 2024-09-16T14:49:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.868091+0000", 2024-09-16T14:49:03.862 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00067759500000000002 2024-09-16T14:49:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.873468+0000", 2024-09-16T14:49:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053766930000000001 2024-09-16T14:49:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.873508+0000", 2024-09-16T14:49:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0314000000000001e-05 2024-09-16T14:49:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.864 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.864 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:459 3.ds0 3:b9c0d236:::smithi04040151-19:head [create,setxattr bk]W^gTaniZ\\TV]dqj_UT\\QgY_\\pYYUOcYdSe`m_ (1859) in=40b,setxattr XSPcap`qmQ\\ZXYl^_bQYNltYZfOtfbRjqmVk_mOa (419) in=40b,setxattr l_Qd^aXhnapg`O][bWecsVdjlWkdYc^tOnjZh]^m (1577) in=40b,setxattr facegtqpaQroppsVbsrZelUak]RbamP`srOTdms[ (350) in=40b,setxattr QllprTts[liem]moe]Pi`nNtcQko (1402) in=28b,setxattr sjm^Un[ahPriW\\TNkaYS\\NnPZfZ\\Zmq[rQ^TWYZm (1892) in=40b,setxattr kRdb__OpRdYV^sYmYmYdhes^Vbf]QaV`SVZkhRcY (209) in=40b,setxattr b[QTnoQc[_Yc^RmdY[ (1402) in=18b,setxattr ^iZkhfc`oO (1402) in=10b,setxattr d\\QUOkoScOhonabPS_YeNlheOOeOal]aQ^ZW]lc` (2805) in=40b,setxattr SioYcliigTgheX\\jnVqipcelfYhbZRjQRet]mQQU (1365) in=40b,setxattr jWcgaRWbr\\ZQ_XXr_eenPa]sXtbaPlmeshkgjOsm (1192) in=40b,setxattr WmVtOY (1402) in=6b,setxattr ]tk_t\\gnSfccZfYfnWnaqP (1402) in=22b,setxattr dS_dYomnsYPtQarmqoZQkO]Ofk_SfgjgitphTZPg (18524) in=40b,setxattr g_a^Zpt`YVahZiOW]V[jQ\\QccqSTo_ma[\\c\\[aft (454) in=40b,setxattr qmmdmgOR__hN\\UTjpR_gbY^s_pOVSdbTob`k (1402) in=36b,setxattr cW`PPOYrbfXV^hmaWhVNb`qVSXqjcdej`pY][]l^ (665) in=40b,setxattr jjo^VgcaQffrnVNPlnq_iTmcrlnTOenhaZV[_SZf (453) in=40b,setxattr n^nqj]bnVcl\\njtUkprb\\laSh^Rbmam_Z_NkeXqj (595) in=40b] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:03.864 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.993572+0000", 2024-09-16T14:49:03.864 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.7548237909999997, 2024-09-16T14:49:03.864 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010337796999999999, 2024-09-16T14:49:03.864 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.864 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:03.864 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:03.864 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:03.864 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:03.864 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 459 2024-09-16T14:49:03.864 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.864 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993572+0000", 2024-09-16T14:49:03.865 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993572+0000", 2024-09-16T14:49:03.865 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993569+0000", 2024-09-16T14:49:03.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:03.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.866 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993625+0000", 2024-09-16T14:49:03.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5105000000000002e-05 2024-09-16T14:49:03.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.866 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993647+0000", 2024-09-16T14:49:03.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1962000000000001e-05 2024-09-16T14:49:03.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.866 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.867 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993656+0000", 2024-09-16T14:49:03.867 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3179999999999996e-06 2024-09-16T14:49:03.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.867 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.867 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993717+0000", 2024-09-16T14:49:03.867 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.0738e-05 2024-09-16T14:49:03.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.867 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:03.867 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993841+0000", 2024-09-16T14:49:03.867 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000124537 2024-09-16T14:49:03.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.994891+0000", 2024-09-16T14:49:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0010499719999999999 2024-09-16T14:49:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.999326+0000", 2024-09-16T14:49:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044351240000000004 2024-09-16T14:49:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003847+0000", 2024-09-16T14:49:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045207629999999997 2024-09-16T14:49:03.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.869 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.869 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003910+0000", 2024-09-16T14:49:03.869 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2527999999999995e-05 2024-09-16T14:49:03.869 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.869 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.869 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.869 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.5s4 51/27 ECSubWrite(tid=60, reqid=client.4500.0:467, at_version=51'19, trim_to=0'0, roll_forward_to=51'16))", 2024-09-16T14:49:03.869 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.997576+0000", 2024-09-16T14:49:03.869 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.7508196659999999, 2024-09-16T14:49:03.869 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083578019999999992, 2024-09-16T14:49:03.870 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:03.870 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:03.870 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:03.870 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.870 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:03.870 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997576+0000", 2024-09-16T14:49:03.870 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.871 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.871 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:03.872 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997576+0000", 2024-09-16T14:49:03.872 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:03.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.872 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.872 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:03.872 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997575+0000", 2024-09-16T14:49:03.872 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:03.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.872 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.872 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:03.872 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997593+0000", 2024-09-16T14:49:03.872 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7645000000000001e-05 2024-09-16T14:49:03.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.872 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.873 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:03.873 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997608+0000", 2024-09-16T14:49:03.873 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5315000000000002e-05 2024-09-16T14:49:03.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.873 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.873 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:03.873 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997616+0000", 2024-09-16T14:49:03.873 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.6120000000000001e-06 2024-09-16T14:49:03.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.873 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.873 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:03.873 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997651+0000", 2024-09-16T14:49:03.873 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5256999999999998e-05 2024-09-16T14:49:03.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.874 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:03.874 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997660+0000", 2024-09-16T14:49:03.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7320000000000004e-06 2024-09-16T14:49:03.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.874 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:03.874 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.005913+0000", 2024-09-16T14:49:03.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082530840000000008 2024-09-16T14:49:03.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:03.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:03.874 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:03.874 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.005934+0000", 2024-09-16T14:49:03.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0812999999999999e-05 2024-09-16T14:49:03.875 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.875 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.875 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.875 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:03.875 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:03.875 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:03.876 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:49:03.877 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:03.875+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:03.882 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:49:03.896 INFO:tasks.rados.rados.0.smithi040.stdout:211: done (3 left) 2024-09-16T14:49:03.896 INFO:tasks.rados.rados.0.smithi040.stdout:212: done (2 left) 2024-09-16T14:49:03.896 INFO:tasks.rados.rados.0.smithi040.stdout:213: done (1 left) 2024-09-16T14:49:03.897 INFO:tasks.rados.rados.0.smithi040.stdout:214: done (0 left) 2024-09-16T14:49:03.897 INFO:tasks.rados.rados.0.smithi040.stdout:215: snap_remove snap 10 2024-09-16T14:49:03.977 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:03.976+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:03.992 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:03.992 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:03.992 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:04.001 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:49:04.078 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:04.077+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:04.105 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:04.105 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:04.105 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:04.105 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:04.105 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:04.114 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:49:04.178 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:04.178+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:04.217 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:04.217 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:04.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:04.217 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:04.218 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.218 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:295 3.9s0 3:96d2b183:::smithi04040151-29:head [copy-from ver 0] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T14:49:04.218 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:04.218 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.299772689000001, 2024-09-16T14:49:04.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010216596999999999, 2024-09-16T14:49:04.218 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.218 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:04.218 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:04.218 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:04.218 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:04.219 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 295 2024-09-16T14:49:04.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.219 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:04.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.219 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.219 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:04.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915430+0000", 2024-09-16T14:49:04.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:04.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.220 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.220 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.220 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915441+0000", 2024-09-16T14:49:04.220 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0789e-05 2024-09-16T14:49:04.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915462+0000", 2024-09-16T14:49:04.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0845000000000001e-05 2024-09-16T14:49:04.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915473+0000", 2024-09-16T14:49:04.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0805000000000001e-05 2024-09-16T14:49:04.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915528+0000", 2024-09-16T14:49:04.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4772999999999999e-05 2024-09-16T14:49:04.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:04.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915537+0000", 2024-09-16T14:49:04.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6700000000000006e-06 2024-09-16T14:49:04.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.919813+0000", 2024-09-16T14:49:04.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042757840000000004 2024-09-16T14:49:04.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:04.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.919907+0000", 2024-09-16T14:49:04.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504999999999993e-05 2024-09-16T14:49:04.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925649+0000", 2024-09-16T14:49:04.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057427870000000001 2024-09-16T14:49:04.223 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.223 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.223 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.224 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:296 3.9s0 3:96d2b183:::smithi04040151-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-16T14:49:04.224 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:04.224 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.299720772000001, 2024-09-16T14:49:04.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010423623, 2024-09-16T14:49:04.224 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.224 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:04.224 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:04.224 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:04.224 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:04.224 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 296 2024-09-16T14:49:04.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.225 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:04.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:04.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915484+0000", 2024-09-16T14:49:04.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915489+0000", 2024-09-16T14:49:04.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6079999999999998e-06 2024-09-16T14:49:04.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915496+0000", 2024-09-16T14:49:04.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7589999999999997e-06 2024-09-16T14:49:04.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915506+0000", 2024-09-16T14:49:04.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0340000000000002e-06 2024-09-16T14:49:04.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915588+0000", 2024-09-16T14:49:04.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2198000000000001e-05 2024-09-16T14:49:04.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:04.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915594+0000", 2024-09-16T14:49:04.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:49:04.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920099+0000", 2024-09-16T14:49:04.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045048700000000002 2024-09-16T14:49:04.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:04.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920147+0000", 2024-09-16T14:49:04.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8239999999999999e-05 2024-09-16T14:49:04.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925726+0000", 2024-09-16T14:49:04.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055787529999999997 2024-09-16T14:49:04.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:04.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925816+0000", 2024-09-16T14:49:04.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0185000000000005e-05 2024-09-16T14:49:04.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.229 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.229 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925908+0000", 2024-09-16T14:49:04.229 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1948999999999997e-05 2024-09-16T14:49:04.230 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.230 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.230 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.230 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.230 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s6 37/27 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=37'32, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:04.230 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921399+0000", 2024-09-16T14:49:04.230 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.293806158999999, 2024-09-16T14:49:04.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071090340000000002, 2024-09-16T14:49:04.230 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.230 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.230 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.230 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921399+0000", 2024-09-16T14:49:04.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921399+0000", 2024-09-16T14:49:04.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921399+0000", 2024-09-16T14:49:04.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921420+0000", 2024-09-16T14:49:04.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1094000000000002e-05 2024-09-16T14:49:04.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921428+0000", 2024-09-16T14:49:04.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3450000000000006e-06 2024-09-16T14:49:04.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921436+0000", 2024-09-16T14:49:04.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5789999999999998e-06 2024-09-16T14:49:04.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921525+0000", 2024-09-16T14:49:04.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8943999999999994e-05 2024-09-16T14:49:04.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921535+0000", 2024-09-16T14:49:04.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0183000000000001e-05 2024-09-16T14:49:04.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928483+0000", 2024-09-16T14:49:04.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069473800000000004 2024-09-16T14:49:04.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928508+0000", 2024-09-16T14:49:04.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5856000000000001e-05 2024-09-16T14:49:04.234 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.234 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.234 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.234 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs3 37/27 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:04.234 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921846+0000", 2024-09-16T14:49:04.234 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.293359597999999, 2024-09-16T14:49:04.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071221770000000004, 2024-09-16T14:49:04.235 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.235 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.235 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921846+0000", 2024-09-16T14:49:04.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921846+0000", 2024-09-16T14:49:04.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921845+0000", 2024-09-16T14:49:04.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921862+0000", 2024-09-16T14:49:04.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6807999999999999e-05 2024-09-16T14:49:04.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921871+0000", 2024-09-16T14:49:04.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6810000000000004e-06 2024-09-16T14:49:04.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921878+0000", 2024-09-16T14:49:04.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.109e-06 2024-09-16T14:49:04.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921935+0000", 2024-09-16T14:49:04.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7082e-05 2024-09-16T14:49:04.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921944+0000", 2024-09-16T14:49:04.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7609999999999996e-06 2024-09-16T14:49:04.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928947+0000", 2024-09-16T14:49:04.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070028870000000002 2024-09-16T14:49:04.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928968+0000", 2024-09-16T14:49:04.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1251e-05 2024-09-16T14:49:04.239 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.239 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.239 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.239 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s6 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:04.239 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921894+0000", 2024-09-16T14:49:04.239 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.293311262, 2024-09-16T14:49:04.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0072329309999999997, 2024-09-16T14:49:04.239 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.239 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.240 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921894+0000", 2024-09-16T14:49:04.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921894+0000", 2024-09-16T14:49:04.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921894+0000", 2024-09-16T14:49:04.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921908+0000", 2024-09-16T14:49:04.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3976e-05 2024-09-16T14:49:04.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921918+0000", 2024-09-16T14:49:04.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0182e-05 2024-09-16T14:49:04.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921924+0000", 2024-09-16T14:49:04.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1419999999999998e-06 2024-09-16T14:49:04.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922184+0000", 2024-09-16T14:49:04.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000260315 2024-09-16T14:49:04.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922192+0000", 2024-09-16T14:49:04.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7179999999999996e-06 2024-09-16T14:49:04.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929113+0000", 2024-09-16T14:49:04.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069209529999999997 2024-09-16T14:49:04.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929127+0000", 2024-09-16T14:49:04.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3991999999999999e-05 2024-09-16T14:49:04.246 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.246 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.246 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.246 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as4 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:04.246 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:04.246 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.291166121, 2024-09-16T14:49:04.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088260140000000001, 2024-09-16T14:49:04.246 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:04.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924062+0000", 2024-09-16T14:49:04.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2892e-05 2024-09-16T14:49:04.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924070+0000", 2024-09-16T14:49:04.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8690000000000004e-06 2024-09-16T14:49:04.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924076+0000", 2024-09-16T14:49:04.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0739999999999998e-06 2024-09-16T14:49:04.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924119+0000", 2024-09-16T14:49:04.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2549999999999997e-05 2024-09-16T14:49:04.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924126+0000", 2024-09-16T14:49:04.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8960000000000003e-06 2024-09-16T14:49:04.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932835+0000", 2024-09-16T14:49:04.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087089200000000002 2024-09-16T14:49:04.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932865+0000", 2024-09-16T14:49:04.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0054000000000001e-05 2024-09-16T14:49:04.250 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.250 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.251 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.251 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es1 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:04.251 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:04.251 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.291051533000001, 2024-09-16T14:49:04.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087127249999999993, 2024-09-16T14:49:04.251 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.251 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.251 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:04.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:04.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:04.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924168+0000", 2024-09-16T14:49:04.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4800000000000001e-05 2024-09-16T14:49:04.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924176+0000", 2024-09-16T14:49:04.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.25e-06 2024-09-16T14:49:04.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924182+0000", 2024-09-16T14:49:04.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3640000000000004e-06 2024-09-16T14:49:04.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924229+0000", 2024-09-16T14:49:04.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6477999999999998e-05 2024-09-16T14:49:04.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924238+0000", 2024-09-16T14:49:04.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4429999999999992e-06 2024-09-16T14:49:04.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932848+0000", 2024-09-16T14:49:04.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086097910000000003 2024-09-16T14:49:04.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932867+0000", 2024-09-16T14:49:04.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8927999999999999e-05 2024-09-16T14:49:04.255 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.255 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.255 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.255 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.8s7 37/27 ECSubWrite(tid=108, reqid=osd.11.0:108, at_version=37'33, trim_to=0'0, roll_forward_to=35'31))", 2024-09-16T14:49:04.255 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.940492+0000", 2024-09-16T14:49:04.255 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.274713496, 2024-09-16T14:49:04.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061198069999999997, 2024-09-16T14:49:04.255 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.255 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.256 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940492+0000", 2024-09-16T14:49:04.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940492+0000", 2024-09-16T14:49:04.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940492+0000", 2024-09-16T14:49:04.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940508+0000", 2024-09-16T14:49:04.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6767999999999999e-05 2024-09-16T14:49:04.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940520+0000", 2024-09-16T14:49:04.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2078e-05 2024-09-16T14:49:04.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940527+0000", 2024-09-16T14:49:04.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8630000000000002e-06 2024-09-16T14:49:04.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940914+0000", 2024-09-16T14:49:04.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00038634200000000002 2024-09-16T14:49:04.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.940922+0000", 2024-09-16T14:49:04.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7490000000000004e-06 2024-09-16T14:49:04.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.946588+0000", 2024-09-16T14:49:04.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0056654230000000002 2024-09-16T14:49:04.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.946612+0000", 2024-09-16T14:49:04.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4008000000000001e-05 2024-09-16T14:49:04.260 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.260 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.260 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.261 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.8s7 37/27 ECSubWrite(tid=110, reqid=osd.11.0:110, at_version=37'35, trim_to=0'0, roll_forward_to=35'31))", 2024-09-16T14:49:04.261 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.941394+0000", 2024-09-16T14:49:04.261 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.273811399, 2024-09-16T14:49:04.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0075660129999999999, 2024-09-16T14:49:04.261 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.261 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.261 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941394+0000", 2024-09-16T14:49:04.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941394+0000", 2024-09-16T14:49:04.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941393+0000", 2024-09-16T14:49:04.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:04.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941415+0000", 2024-09-16T14:49:04.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1162000000000002e-05 2024-09-16T14:49:04.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941426+0000", 2024-09-16T14:49:04.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1319e-05 2024-09-16T14:49:04.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941435+0000", 2024-09-16T14:49:04.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8769999999999999e-06 2024-09-16T14:49:04.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941762+0000", 2024-09-16T14:49:04.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00032733399999999998 2024-09-16T14:49:04.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.941772+0000", 2024-09-16T14:49:04.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9799999999999993e-06 2024-09-16T14:49:04.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.948937+0000", 2024-09-16T14:49:04.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0071645550000000004 2024-09-16T14:49:04.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.948960+0000", 2024-09-16T14:49:04.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3529000000000001e-05 2024-09-16T14:49:04.266 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.266 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.266 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.266 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:394 3.9s0 3:93498a0b:::smithi04040151-32:head [append 0~491520] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:04.266 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:04.266 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.345633052, 2024-09-16T14:49:04.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016461614999999999, 2024-09-16T14:49:04.266 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.266 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:04.267 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:04.267 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:04.267 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:04.267 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 394 2024-09-16T14:49:04.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.267 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:04.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:04.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869570+0000", 2024-09-16T14:49:04.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:04.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872427+0000", 2024-09-16T14:49:04.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028569070000000001 2024-09-16T14:49:04.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872450+0000", 2024-09-16T14:49:04.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2943e-05 2024-09-16T14:49:04.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872461+0000", 2024-09-16T14:49:04.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0882e-05 2024-09-16T14:49:04.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872521+0000", 2024-09-16T14:49:04.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9639000000000003e-05 2024-09-16T14:49:04.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:04.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872530+0000", 2024-09-16T14:49:04.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8810000000000002e-06 2024-09-16T14:49:04.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.878309+0000", 2024-09-16T14:49:04.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057788520000000001 2024-09-16T14:49:04.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:04.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.878434+0000", 2024-09-16T14:49:04.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012550899999999999 2024-09-16T14:49:04.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880126+0000", 2024-09-16T14:49:04.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016919019999999999 2024-09-16T14:49:04.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885887+0000", 2024-09-16T14:49:04.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057608959999999997 2024-09-16T14:49:04.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:04.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885957+0000", 2024-09-16T14:49:04.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0504000000000001e-05 2024-09-16T14:49:04.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886034+0000", 2024-09-16T14:49:04.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6661000000000003e-05 2024-09-16T14:49:04.274 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.274 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.274 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.274 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:395 3.9s0 3:93498a0b:::smithi04040151-32:head [append 0~770048] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:04.275 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:04.275 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.340905637000001, 2024-09-16T14:49:04.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.018911853999999999, 2024-09-16T14:49:04.275 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.275 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:04.275 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:04.275 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:04.275 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:04.275 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 395 2024-09-16T14:49:04.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.275 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:04.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:04.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874298+0000", 2024-09-16T14:49:04.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:04.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875671+0000", 2024-09-16T14:49:04.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0013727069999999999 2024-09-16T14:49:04.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875689+0000", 2024-09-16T14:49:04.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7832000000000001e-05 2024-09-16T14:49:04.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875700+0000", 2024-09-16T14:49:04.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1267000000000001e-05 2024-09-16T14:49:04.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875758+0000", 2024-09-16T14:49:04.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8060999999999999e-05 2024-09-16T14:49:04.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:04.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875766+0000", 2024-09-16T14:49:04.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9580000000000007e-06 2024-09-16T14:49:04.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881004+0000", 2024-09-16T14:49:04.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052375169999999997 2024-09-16T14:49:04.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:04.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881098+0000", 2024-09-16T14:49:04.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3875999999999994e-05 2024-09-16T14:49:04.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882500+0000", 2024-09-16T14:49:04.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014019600000000001 2024-09-16T14:49:04.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886041+0000", 2024-09-16T14:49:04.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035415210000000002 2024-09-16T14:49:04.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:04.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893136+0000", 2024-09-16T14:49:04.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070949330000000003 2024-09-16T14:49:04.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893212+0000", 2024-09-16T14:49:04.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5739000000000001e-05 2024-09-16T14:49:04.281 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.281 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.281 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.281 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:396 3.9s0 3:93498a0b:::smithi04040151-32:head [setxattr _header (57) in=7b] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:04.281 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:04.281 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.339488482, 2024-09-16T14:49:04.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017781597999999999, 2024-09-16T14:49:04.281 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.282 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:04.282 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:04.282 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:04.282 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:04.282 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 396 2024-09-16T14:49:04.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.282 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:04.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875715+0000", 2024-09-16T14:49:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875721+0000", 2024-09-16T14:49:04.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6989999999999997e-06 2024-09-16T14:49:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875728+0000", 2024-09-16T14:49:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9040000000000003e-06 2024-09-16T14:49:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875734+0000", 2024-09-16T14:49:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4519999999999996e-06 2024-09-16T14:49:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875800+0000", 2024-09-16T14:49:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6538000000000006e-05 2024-09-16T14:49:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875804+0000", 2024-09-16T14:49:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.771e-06 2024-09-16T14:49:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883232+0000", 2024-09-16T14:49:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074282009999999997 2024-09-16T14:49:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883323+0000", 2024-09-16T14:49:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0740000000000002e-05 2024-09-16T14:49:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884057+0000", 2024-09-16T14:49:04.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00073422299999999999 2024-09-16T14:49:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893020+0000", 2024-09-16T14:49:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089623210000000005 2024-09-16T14:49:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893403+0000", 2024-09-16T14:49:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00038375799999999999 2024-09-16T14:49:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893499+0000", 2024-09-16T14:49:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5284000000000005e-05 2024-09-16T14:49:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:397 3.9s0 3:93498a0b:::smithi04040151-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.339457357000001, 2024-09-16T14:49:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019838090999999999, 2024-09-16T14:49:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:04.288 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 397 2024-09-16T14:49:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875750+0000", 2024-09-16T14:49:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3939999999999999e-06 2024-09-16T14:49:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875755+0000", 2024-09-16T14:49:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.31e-06 2024-09-16T14:49:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875760+0000", 2024-09-16T14:49:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4769999999999997e-06 2024-09-16T14:49:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875824+0000", 2024-09-16T14:49:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3893000000000006e-05 2024-09-16T14:49:04.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875829+0000", 2024-09-16T14:49:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3489999999999999e-06 2024-09-16T14:49:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885652+0000", 2024-09-16T14:49:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0098232530000000005 2024-09-16T14:49:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885748+0000", 2024-09-16T14:49:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5366999999999997e-05 2024-09-16T14:49:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893876+0000", 2024-09-16T14:49:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081279479999999994 2024-09-16T14:49:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893967+0000", 2024-09-16T14:49:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0777000000000002e-05 2024-09-16T14:49:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895586+0000", 2024-09-16T14:49:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016197029999999999 2024-09-16T14:49:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(osd.3.17:4 3.9s0 3:9b721ec3:::smithi04040151-12:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.331235062999999, 2024-09-16T14:49:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011299052, 2024-09-16T14:49:04.294 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883969+0000", 2024-09-16T14:49:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883976+0000", 2024-09-16T14:49:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3730000000000004e-06 2024-09-16T14:49:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883999+0000", 2024-09-16T14:49:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3390000000000001e-05 2024-09-16T14:49:04.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884007+0000", 2024-09-16T14:49:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1489999999999997e-06 2024-09-16T14:49:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886141+0000", 2024-09-16T14:49:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0021347240000000002 2024-09-16T14:49:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:04.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886230+0000", 2024-09-16T14:49:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8234999999999998e-05 2024-09-16T14:49:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895269+0000", 2024-09-16T14:49:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090399230000000001 2024-09-16T14:49:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 47/27 ECSubWriteReply(tid=27, last_complete=47'18, committed=1, applied=1))", 2024-09-16T14:49:04.298 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.886820+0000", 2024-09-16T14:49:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.32838583, 2024-09-16T14:49:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064054309999999996, 2024-09-16T14:49:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886820+0000", 2024-09-16T14:49:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886820+0000", 2024-09-16T14:49:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886819+0000", 2024-09-16T14:49:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886825+0000", 2024-09-16T14:49:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8440000000000003e-06 2024-09-16T14:49:04.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886836+0000", 2024-09-16T14:49:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1331e-05 2024-09-16T14:49:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886844+0000", 2024-09-16T14:49:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.644e-06 2024-09-16T14:49:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893058+0000", 2024-09-16T14:49:04.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062146379999999998 2024-09-16T14:49:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893225+0000", 2024-09-16T14:49:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000166868 2024-09-16T14:49:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpReadReply(3.9s0 47/27 ECSubReadReply(tid=29, attrs_read=0))", 2024-09-16T14:49:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.886991+0000", 2024-09-16T14:49:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.328214579000001, 2024-09-16T14:49:04.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0076394690000000003, 2024-09-16T14:49:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886991+0000", 2024-09-16T14:49:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886991+0000", 2024-09-16T14:49:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886990+0000", 2024-09-16T14:49:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887086+0000", 2024-09-16T14:49:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5481000000000003e-05 2024-09-16T14:49:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887099+0000", 2024-09-16T14:49:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3713999999999999e-05 2024-09-16T14:49:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887105+0000", 2024-09-16T14:49:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0460000000000002e-06 2024-09-16T14:49:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894600+0000", 2024-09-16T14:49:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074949450000000003 2024-09-16T14:49:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894630+0000", 2024-09-16T14:49:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0026e-05 2024-09-16T14:49:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpReadReply(3.9s0 47/27 ECSubReadReply(tid=29, attrs_read=0))", 2024-09-16T14:49:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.887586+0000", 2024-09-16T14:49:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.327619894, 2024-09-16T14:49:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069926040000000004, 2024-09-16T14:49:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.306 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887586+0000", 2024-09-16T14:49:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887586+0000", 2024-09-16T14:49:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887585+0000", 2024-09-16T14:49:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887722+0000", 2024-09-16T14:49:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000137458 2024-09-16T14:49:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887736+0000", 2024-09-16T14:49:04.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3586e-05 2024-09-16T14:49:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887742+0000", 2024-09-16T14:49:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2060000000000004e-06 2024-09-16T14:49:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894548+0000", 2024-09-16T14:49:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068061019999999996 2024-09-16T14:49:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894578+0000", 2024-09-16T14:49:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9989e-05 2024-09-16T14:49:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:440 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.3391817420000001, 2024-09-16T14:49:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01253772, 2024-09-16T14:49:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:04.310 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 440 2024-09-16T14:49:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:04.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876022+0000", 2024-09-16T14:49:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876033+0000", 2024-09-16T14:49:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1474000000000001e-05 2024-09-16T14:49:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876051+0000", 2024-09-16T14:49:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7628000000000001e-05 2024-09-16T14:49:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876060+0000", 2024-09-16T14:49:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339999999999996e-06 2024-09-16T14:49:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876128+0000", 2024-09-16T14:49:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7460999999999997e-05 2024-09-16T14:49:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876138+0000", 2024-09-16T14:49:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8649999999999996e-06 2024-09-16T14:49:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882141+0000", 2024-09-16T14:49:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060036550000000001 2024-09-16T14:49:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882233+0000", 2024-09-16T14:49:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1626000000000007e-05 2024-09-16T14:49:04.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.888561+0000", 2024-09-16T14:49:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063284769999999999 2024-09-16T14:49:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:441 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.3391346850000003, 2024-09-16T14:49:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.013066019999999999, 2024-09-16T14:49:04.315 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 441 2024-09-16T14:49:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876070+0000", 2024-09-16T14:49:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876074+0000", 2024-09-16T14:49:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1330000000000001e-06 2024-09-16T14:49:04.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876083+0000", 2024-09-16T14:49:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2519999999999999e-06 2024-09-16T14:49:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876088+0000", 2024-09-16T14:49:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5509999999999999e-06 2024-09-16T14:49:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876235+0000", 2024-09-16T14:49:04.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014688900000000001 2024-09-16T14:49:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876243+0000", 2024-09-16T14:49:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5490000000000001e-06 2024-09-16T14:49:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.884127+0000", 2024-09-16T14:49:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078848489999999993 2024-09-16T14:49:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.884238+0000", 2024-09-16T14:49:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111007 2024-09-16T14:49:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889137+0000", 2024-09-16T14:49:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048983739999999996 2024-09-16T14:49:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:442 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:04.320 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.3390960270000001, 2024-09-16T14:49:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014710668, 2024-09-16T14:49:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 442 2024-09-16T14:49:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876113+0000", 2024-09-16T14:49:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0759999999999996e-06 2024-09-16T14:49:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876119+0000", 2024-09-16T14:49:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3339999999999998e-06 2024-09-16T14:49:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876126+0000", 2024-09-16T14:49:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8410000000000002e-06 2024-09-16T14:49:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876273+0000", 2024-09-16T14:49:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014714900000000001 2024-09-16T14:49:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876279+0000", 2024-09-16T14:49:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7819999999999999e-06 2024-09-16T14:49:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.885397+0000", 2024-09-16T14:49:04.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091178149999999996 2024-09-16T14:49:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.885492+0000", 2024-09-16T14:49:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5384999999999996e-05 2024-09-16T14:49:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.890820+0000", 2024-09-16T14:49:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053278559999999997 2024-09-16T14:49:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.325 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.326 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.326 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:04.327 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:49:04.328 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:04.326+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:04.391 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:49:04.428 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:04.427+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:04.430 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:04.430 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:04.430 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:04.430 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:04.439 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:49:04.516 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:04.517 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:04.529 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:04.528+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:04.533 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:04.533 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:04.541 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:04.542 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:04.542 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:04.542 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:04.542 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:04.548 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:04.549 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:04.551 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:49:04.564 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:04.565 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:04.581 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:04.582 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:04.598 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:04.598 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:04.614 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:04.615 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:04.630 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:04.629+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:04.631 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:04.631 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:04.647 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:04.647 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:04.654 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:04.654 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:04.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:04.655 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:04.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.655 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=79, reqid=osd.3.0:79, at_version=34'18, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:49:04.655 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:49:04.655 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.744881724999999, 2024-09-16T14:49:04.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069050049999999997, 2024-09-16T14:49:04.655 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.655 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.655 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:49:04.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:49:04.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907837+0000", 2024-09-16T14:49:04.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907863+0000", 2024-09-16T14:49:04.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5972000000000001e-05 2024-09-16T14:49:04.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907876+0000", 2024-09-16T14:49:04.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3230999999999999e-05 2024-09-16T14:49:04.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907900+0000", 2024-09-16T14:49:04.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3716000000000001e-05 2024-09-16T14:49:04.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907956+0000", 2024-09-16T14:49:04.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6307999999999997e-05 2024-09-16T14:49:04.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.907965+0000", 2024-09-16T14:49:04.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1530000000000003e-06 2024-09-16T14:49:04.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914715+0000", 2024-09-16T14:49:04.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067495990000000002 2024-09-16T14:49:04.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914742+0000", 2024-09-16T14:49:04.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.728e-05 2024-09-16T14:49:04.659 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.659 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.659 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.660 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=80, reqid=osd.3.0:80, at_version=34'20, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:49:04.660 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:49:04.660 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.743465055000001, 2024-09-16T14:49:04.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065560849999999997, 2024-09-16T14:49:04.660 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.660 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.660 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:49:04.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909254+0000", 2024-09-16T14:49:04.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909253+0000", 2024-09-16T14:49:04.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:04.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909278+0000", 2024-09-16T14:49:04.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4905e-05 2024-09-16T14:49:04.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909295+0000", 2024-09-16T14:49:04.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6713000000000001e-05 2024-09-16T14:49:04.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909304+0000", 2024-09-16T14:49:04.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6739999999999992e-06 2024-09-16T14:49:04.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909367+0000", 2024-09-16T14:49:04.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2700999999999995e-05 2024-09-16T14:49:04.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909391+0000", 2024-09-16T14:49:04.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3614000000000001e-05 2024-09-16T14:49:04.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915784+0000", 2024-09-16T14:49:04.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063938479999999997 2024-09-16T14:49:04.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915810+0000", 2024-09-16T14:49:04.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5457999999999999e-05 2024-09-16T14:49:04.664 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.664 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.664 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.665 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:49:04.665 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:04.665 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.742837466000001, 2024-09-16T14:49:04.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007542137, 2024-09-16T14:49:04.665 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.665 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.665 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:04.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:04.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:04.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909917+0000", 2024-09-16T14:49:04.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6513000000000001e-05 2024-09-16T14:49:04.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:49:04.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:49:04.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909941+0000", 2024-09-16T14:49:04.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1859999999999992e-06 2024-09-16T14:49:04.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910004+0000", 2024-09-16T14:49:04.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2706999999999994e-05 2024-09-16T14:49:04.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910012+0000", 2024-09-16T14:49:04.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.157e-06 2024-09-16T14:49:04.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917395+0000", 2024-09-16T14:49:04.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073827659999999998 2024-09-16T14:49:04.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917423+0000", 2024-09-16T14:49:04.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8568000000000002e-05 2024-09-16T14:49:04.669 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.669 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.669 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.669 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:49:04.669 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:49:04.670 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.738675226000002, 2024-09-16T14:49:04.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065747590000000003, 2024-09-16T14:49:04.670 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.670 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.670 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:49:04.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914044+0000", 2024-09-16T14:49:04.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914043+0000", 2024-09-16T14:49:04.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914071+0000", 2024-09-16T14:49:04.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7633000000000001e-05 2024-09-16T14:49:04.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914086+0000", 2024-09-16T14:49:04.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4973999999999999e-05 2024-09-16T14:49:04.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914095+0000", 2024-09-16T14:49:04.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2119999999999992e-06 2024-09-16T14:49:04.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915626+0000", 2024-09-16T14:49:04.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001530998 2024-09-16T14:49:04.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.915635+0000", 2024-09-16T14:49:04.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0140000000000004e-06 2024-09-16T14:49:04.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920590+0000", 2024-09-16T14:49:04.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049547740000000003 2024-09-16T14:49:04.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920618+0000", 2024-09-16T14:49:04.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8767000000000001e-05 2024-09-16T14:49:04.674 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.674 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.674 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.675 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:49:04.675 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:49:04.675 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.736352736000001, 2024-09-16T14:49:04.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006879947, 2024-09-16T14:49:04.675 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.675 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.675 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:49:04.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:49:04.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916366+0000", 2024-09-16T14:49:04.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:49:04.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:04.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8702000000000002e-05 2024-09-16T14:49:04.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916394+0000", 2024-09-16T14:49:04.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6350000000000001e-06 2024-09-16T14:49:04.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916403+0000", 2024-09-16T14:49:04.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3950000000000001e-06 2024-09-16T14:49:04.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916459+0000", 2024-09-16T14:49:04.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5889e-05 2024-09-16T14:49:04.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916467+0000", 2024-09-16T14:49:04.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2770000000000005e-06 2024-09-16T14:49:04.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923224+0000", 2024-09-16T14:49:04.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067573099999999999 2024-09-16T14:49:04.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923246+0000", 2024-09-16T14:49:04.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1882000000000002e-05 2024-09-16T14:49:04.679 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.679 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.679 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.679 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 34/27 ECSubWrite(tid=82, reqid=osd.3.0:80, at_version=0'0, trim_to=0'0, roll_forward_to=34'20))", 2024-09-16T14:49:04.679 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:49:04.679 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.734778871, 2024-09-16T14:49:04.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005378954, 2024-09-16T14:49:04.680 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.680 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.680 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:49:04.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917940+0000", 2024-09-16T14:49:04.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917939+0000", 2024-09-16T14:49:04.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917952+0000", 2024-09-16T14:49:04.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2278e-05 2024-09-16T14:49:04.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917963+0000", 2024-09-16T14:49:04.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.102e-05 2024-09-16T14:49:04.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917970+0000", 2024-09-16T14:49:04.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9589999999999998e-06 2024-09-16T14:49:04.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.918179+0000", 2024-09-16T14:49:04.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020897 2024-09-16T14:49:04.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.918188+0000", 2024-09-16T14:49:04.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9600000000000006e-06 2024-09-16T14:49:04.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923288+0000", 2024-09-16T14:49:04.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051001830000000003 2024-09-16T14:49:04.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923319+0000", 2024-09-16T14:49:04.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1131999999999999e-05 2024-09-16T14:49:04.684 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.684 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.684 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.684 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=37'32, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:04.684 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:04.684 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.731727121999999, 2024-09-16T14:49:04.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070991120000000003, 2024-09-16T14:49:04.685 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.685 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.685 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:04.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:04.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920991+0000", 2024-09-16T14:49:04.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921010+0000", 2024-09-16T14:49:04.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8529000000000002e-05 2024-09-16T14:49:04.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921018+0000", 2024-09-16T14:49:04.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4670000000000004e-06 2024-09-16T14:49:04.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921025+0000", 2024-09-16T14:49:04.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9330000000000004e-06 2024-09-16T14:49:04.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921076+0000", 2024-09-16T14:49:04.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0662999999999999e-05 2024-09-16T14:49:04.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921084+0000", 2024-09-16T14:49:04.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4780000000000008e-06 2024-09-16T14:49:04.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928073+0000", 2024-09-16T14:49:04.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069883940000000002 2024-09-16T14:49:04.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928091+0000", 2024-09-16T14:49:04.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7992999999999999e-05 2024-09-16T14:49:04.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.689 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.689 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s6 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:04.689 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:49:04.689 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.730763867, 2024-09-16T14:49:04.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059936520000000004, 2024-09-16T14:49:04.689 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.689 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.690 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:49:04.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:49:04.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921955+0000", 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921983+0000", 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8493999999999999e-05 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3110000000000006e-06 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921999+0000", 2024-09-16T14:49:04.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5499999999999997e-06 2024-09-16T14:49:04.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922042+0000", 2024-09-16T14:49:04.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3387999999999999e-05 2024-09-16T14:49:04.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922053+0000", 2024-09-16T14:49:04.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0244999999999999e-05 2024-09-16T14:49:04.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.927921+0000", 2024-09-16T14:49:04.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0058679159999999999 2024-09-16T14:49:04.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.927949+0000", 2024-09-16T14:49:04.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8038000000000002e-05 2024-09-16T14:49:04.693 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.693 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.693 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.694 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:04.694 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:49:04.694 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.730727817000002, 2024-09-16T14:49:04.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070100509999999998, 2024-09-16T14:49:04.694 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.694 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.694 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:49:04.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:49:04.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921990+0000", 2024-09-16T14:49:04.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:04.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922012+0000", 2024-09-16T14:49:04.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1681000000000001e-05 2024-09-16T14:49:04.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922025+0000", 2024-09-16T14:49:04.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3101999999999999e-05 2024-09-16T14:49:04.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922033+0000", 2024-09-16T14:49:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1899999999999995e-06 2024-09-16T14:49:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922727+0000", 2024-09-16T14:49:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00069429699999999997 2024-09-16T14:49:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922737+0000", 2024-09-16T14:49:04.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.285e-06 2024-09-16T14:49:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928986+0000", 2024-09-16T14:49:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062492679999999997 2024-09-16T14:49:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929001+0000", 2024-09-16T14:49:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5058999999999999e-05 2024-09-16T14:49:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.699 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 37/27 ECSubWrite(tid=189, reqid=osd.7.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'12))", 2024-09-16T14:49:04.699 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922523+0000", 2024-09-16T14:49:04.699 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.730195952999999, 2024-09-16T14:49:04.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055324470000000002, 2024-09-16T14:49:04.699 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.699 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.699 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922523+0000", 2024-09-16T14:49:04.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922523+0000", 2024-09-16T14:49:04.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922522+0000", 2024-09-16T14:49:04.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922531+0000", 2024-09-16T14:49:04.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6079999999999996e-06 2024-09-16T14:49:04.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922540+0000", 2024-09-16T14:49:04.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2629999999999992e-06 2024-09-16T14:49:04.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922548+0000", 2024-09-16T14:49:04.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4089999999999997e-06 2024-09-16T14:49:04.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922589+0000", 2024-09-16T14:49:04.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0853999999999999e-05 2024-09-16T14:49:04.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922597+0000", 2024-09-16T14:49:04.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3270000000000001e-06 2024-09-16T14:49:04.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928016+0000", 2024-09-16T14:49:04.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054186269999999996 2024-09-16T14:49:04.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928055+0000", 2024-09-16T14:49:04.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.9749999999999997e-05 2024-09-16T14:49:04.703 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.703 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.703 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.ds2 37/27 ECSubWrite(tid=190, reqid=osd.7.0:188, at_version=0'0, trim_to=0'0, roll_forward_to=37'20))", 2024-09-16T14:49:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:49:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.729969361999999, 2024-09-16T14:49:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062183510000000004, 2024-09-16T14:49:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:49:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:49:04.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922749+0000", 2024-09-16T14:49:04.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922758+0000", 2024-09-16T14:49:04.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6209999999999993e-06 2024-09-16T14:49:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922767+0000", 2024-09-16T14:49:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4159999999999993e-06 2024-09-16T14:49:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922780+0000", 2024-09-16T14:49:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2992e-05 2024-09-16T14:49:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922834+0000", 2024-09-16T14:49:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3835999999999998e-05 2024-09-16T14:49:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922911+0000", 2024-09-16T14:49:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7534999999999995e-05 2024-09-16T14:49:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928945+0000", 2024-09-16T14:49:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060339210000000002 2024-09-16T14:49:04.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928968+0000", 2024-09-16T14:49:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2450999999999999e-05 2024-09-16T14:49:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s6 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:49:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923698+0000", 2024-09-16T14:49:04.708 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.729021108000001, 2024-09-16T14:49:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055372479999999998, 2024-09-16T14:49:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923698+0000", 2024-09-16T14:49:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923698+0000", 2024-09-16T14:49:04.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923697+0000", 2024-09-16T14:49:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923708+0000", 2024-09-16T14:49:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1524e-05 2024-09-16T14:49:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923720+0000", 2024-09-16T14:49:04.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1620000000000001e-05 2024-09-16T14:49:04.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923732+0000", 2024-09-16T14:49:04.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1711e-05 2024-09-16T14:49:04.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.711 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.711 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923796+0000", 2024-09-16T14:49:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4673999999999998e-05 2024-09-16T14:49:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923805+0000", 2024-09-16T14:49:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3240000000000003e-06 2024-09-16T14:49:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929203+0000", 2024-09-16T14:49:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005398351 2024-09-16T14:49:04.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929235+0000", 2024-09-16T14:49:04.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1902000000000001e-05 2024-09-16T14:49:04.713 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.713 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.713 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.713 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:04.713 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:04.713 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.727518752000002, 2024-09-16T14:49:04.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079457759999999999, 2024-09-16T14:49:04.714 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.714 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.714 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:04.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925222+0000", 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1871000000000001e-05 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925238+0000", 2024-09-16T14:49:04.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6719999999999999e-05 2024-09-16T14:49:04.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925244+0000", 2024-09-16T14:49:04.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3630000000000002e-06 2024-09-16T14:49:04.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925291+0000", 2024-09-16T14:49:04.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6944000000000003e-05 2024-09-16T14:49:04.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925299+0000", 2024-09-16T14:49:04.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:49:04.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933129+0000", 2024-09-16T14:49:04.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078297330000000002 2024-09-16T14:49:04.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.717 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.717 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933146+0000", 2024-09-16T14:49:04.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.664e-05 2024-09-16T14:49:04.718 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.718 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.718 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.718 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es5 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:04.718 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:04.718 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.727467117, 2024-09-16T14:49:04.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078358790000000005, 2024-09-16T14:49:04.718 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.718 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.718 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:04.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:04.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925251+0000", 2024-09-16T14:49:04.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925265+0000", 2024-09-16T14:49:04.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4078e-05 2024-09-16T14:49:04.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925273+0000", 2024-09-16T14:49:04.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1069999999999999e-06 2024-09-16T14:49:04.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925279+0000", 2024-09-16T14:49:04.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4409999999999998e-06 2024-09-16T14:49:04.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925330+0000", 2024-09-16T14:49:04.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1208999999999997e-05 2024-09-16T14:49:04.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925339+0000", 2024-09-16T14:49:04.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8109999999999992e-06 2024-09-16T14:49:04.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933064+0000", 2024-09-16T14:49:04.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077252149999999997 2024-09-16T14:49:04.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933088+0000", 2024-09-16T14:49:04.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3317999999999999e-05 2024-09-16T14:49:04.722 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.722 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.722 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.722 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 37/27 ECSubWrite(tid=125, reqid=osd.3.0:125, at_version=37'43, trim_to=0'0, roll_forward_to=37'41))", 2024-09-16T14:49:04.722 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.943728+0000", 2024-09-16T14:49:04.723 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.708991032, 2024-09-16T14:49:04.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005567626, 2024-09-16T14:49:04.723 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.723 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.723 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943728+0000", 2024-09-16T14:49:04.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943728+0000", 2024-09-16T14:49:04.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943727+0000", 2024-09-16T14:49:04.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:04.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943749+0000", 2024-09-16T14:49:04.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1634999999999999e-05 2024-09-16T14:49:04.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943760+0000", 2024-09-16T14:49:04.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1727e-05 2024-09-16T14:49:04.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943768+0000", 2024-09-16T14:49:04.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4529999999999996e-06 2024-09-16T14:49:04.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943814+0000", 2024-09-16T14:49:04.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6346e-05 2024-09-16T14:49:04.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.943823+0000", 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3230000000000004e-06 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949278+0000", 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054548649999999997 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.949295+0000", 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7079e-05 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.726 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.727 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.727 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.ds2 47/27 ECSubWrite(tid=208, reqid=client.4500.0:390, at_version=0'0, trim_to=0'0, roll_forward_to=47'27))", 2024-09-16T14:49:04.727 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.873647+0000", 2024-09-16T14:49:04.727 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.779072127999999, 2024-09-16T14:49:04.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065883130000000002, 2024-09-16T14:49:04.727 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.727 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.727 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873647+0000", 2024-09-16T14:49:04.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873647+0000", 2024-09-16T14:49:04.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.728 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873646+0000", 2024-09-16T14:49:04.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:04.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.728 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873660+0000", 2024-09-16T14:49:04.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4056000000000001e-05 2024-09-16T14:49:04.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873675+0000", 2024-09-16T14:49:04.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.543e-05 2024-09-16T14:49:04.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873683+0000", 2024-09-16T14:49:04.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0879999999999995e-06 2024-09-16T14:49:04.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873732+0000", 2024-09-16T14:49:04.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8738e-05 2024-09-16T14:49:04.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873742+0000", 2024-09-16T14:49:04.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8949999999999993e-06 2024-09-16T14:49:04.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880210+0000", 2024-09-16T14:49:04.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006468321 2024-09-16T14:49:04.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880235+0000", 2024-09-16T14:49:04.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.495e-05 2024-09-16T14:49:04.731 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.731 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.731 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.731 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 47/27 ECSubWrite(tid=45, reqid=client.4500.0:393, at_version=47'34, trim_to=0'0, roll_forward_to=43'30))", 2024-09-16T14:49:04.731 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.873840+0000", 2024-09-16T14:49:04.732 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.778878805, 2024-09-16T14:49:04.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064368940000000003, 2024-09-16T14:49:04.732 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.732 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.732 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873840+0000", 2024-09-16T14:49:04.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873840+0000", 2024-09-16T14:49:04.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873839+0000", 2024-09-16T14:49:04.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:04.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873858+0000", 2024-09-16T14:49:04.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9228e-05 2024-09-16T14:49:04.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873870+0000", 2024-09-16T14:49:04.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1450999999999999e-05 2024-09-16T14:49:04.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873878+0000", 2024-09-16T14:49:04.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3999999999999992e-06 2024-09-16T14:49:04.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873936+0000", 2024-09-16T14:49:04.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7374999999999999e-05 2024-09-16T14:49:04.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.873945+0000", 2024-09-16T14:49:04.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.662e-06 2024-09-16T14:49:04.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880251+0000", 2024-09-16T14:49:04.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063056099999999997 2024-09-16T14:49:04.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880277+0000", 2024-09-16T14:49:04.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6013999999999999e-05 2024-09-16T14:49:04.736 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.736 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.736 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.736 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.9s3 47/27 ECSubWrite(tid=28, reqid=client.4500.0:396, at_version=47'19, trim_to=0'0, roll_forward_to=40'15))", 2024-09-16T14:49:04.736 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.885200+0000", 2024-09-16T14:49:04.736 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.767519182999999, 2024-09-16T14:49:04.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053724640000000004, 2024-09-16T14:49:04.736 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.736 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.736 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885200+0000", 2024-09-16T14:49:04.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885200+0000", 2024-09-16T14:49:04.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885199+0000", 2024-09-16T14:49:04.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:04.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885217+0000", 2024-09-16T14:49:04.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8190999999999999e-05 2024-09-16T14:49:04.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885232+0000", 2024-09-16T14:49:04.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4676e-05 2024-09-16T14:49:04.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885240+0000", 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6240000000000008e-06 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885296+0000", 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5705000000000003e-05 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885307+0000", 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1601e-05 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890555+0000", 2024-09-16T14:49:04.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052472229999999996 2024-09-16T14:49:04.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890572+0000", 2024-09-16T14:49:04.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7394999999999999e-05 2024-09-16T14:49:04.740 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.740 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.740 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.740 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 51/27 ECSubWrite(tid=165, reqid=osd.3.0:165, at_version=51'48, trim_to=0'0, roll_forward_to=50'46))", 2024-09-16T14:49:04.741 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.877962+0000", 2024-09-16T14:49:04.741 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.7747565539999997, 2024-09-16T14:49:04.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0050691460000000001, 2024-09-16T14:49:04.741 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.741 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.741 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.877962+0000", 2024-09-16T14:49:04.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.877962+0000", 2024-09-16T14:49:04.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.877962+0000", 2024-09-16T14:49:04.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.877999+0000", 2024-09-16T14:49:04.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7058999999999999e-05 2024-09-16T14:49:04.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.878015+0000", 2024-09-16T14:49:04.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6370000000000001e-05 2024-09-16T14:49:04.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.878024+0000", 2024-09-16T14:49:04.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1880000000000008e-06 2024-09-16T14:49:04.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.878113+0000", 2024-09-16T14:49:04.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8469999999999998e-05 2024-09-16T14:49:04.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.878122+0000", 2024-09-16T14:49:04.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6830000000000003e-06 2024-09-16T14:49:04.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.883000+0000", 2024-09-16T14:49:04.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048776189999999997 2024-09-16T14:49:04.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.883031+0000", 2024-09-16T14:49:04.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1316999999999998e-05 2024-09-16T14:49:04.745 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.745 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.745 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.745 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 51/27 ECSubWrite(tid=171, reqid=osd.11.0:171, at_version=51'22, trim_to=0'0, roll_forward_to=49'20))", 2024-09-16T14:49:04.745 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.881321+0000", 2024-09-16T14:49:04.745 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.7713977129999998, 2024-09-16T14:49:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064808820000000003, 2024-09-16T14:49:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881321+0000", 2024-09-16T14:49:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881321+0000", 2024-09-16T14:49:04.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881321+0000", 2024-09-16T14:49:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881482+0000", 2024-09-16T14:49:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000161318 2024-09-16T14:49:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:04.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881495+0000", 2024-09-16T14:49:04.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3210999999999999e-05 2024-09-16T14:49:04.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:04.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881504+0000", 2024-09-16T14:49:04.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.772e-06 2024-09-16T14:49:04.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.748 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:04.748 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881563+0000", 2024-09-16T14:49:04.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9608999999999998e-05 2024-09-16T14:49:04.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:04.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881575+0000", 2024-09-16T14:49:04.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1904e-05 2024-09-16T14:49:04.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:04.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.887760+0000", 2024-09-16T14:49:04.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061849319999999998 2024-09-16T14:49:04.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:04.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:04.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:04.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.887802+0000", 2024-09-16T14:49:04.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1706999999999999e-05 2024-09-16T14:49:04.750 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.750 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.750 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.750 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:04.750 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:04.750 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:04.751 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:04.751 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:04.752 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:04.752 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:04.753 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:49:04.753 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:04.750+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:04.753 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:04.754 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:04.852 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:04.851+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:04.896 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:04.897 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:04.897 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is unset 2024-09-16T14:49:04.897 INFO:tasks.rados.rados.0.smithi040.stdout:215: done (0 left) 2024-09-16T14:49:04.897 INFO:tasks.rados.rados.0.smithi040.stdout:216: snap_create 2024-09-16T14:49:04.911 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:49:04.953 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:04.952+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:05.054 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:05.052+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:05.054 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:05.054 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:05.063 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:49:05.155 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:05.154+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:05.171 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 6] 2024-09-16T14:49:05.171 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:49:05.172 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-09-16T14:49:05.172 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:49:05.205 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:05.205 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:05.213 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:49:05.256 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:05.255+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:05.316 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:05.316 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:05.324 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:49:05.357 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:05.357+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:05.426 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:05.426 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:05.435 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:49:05.458 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:05.457+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:05.463 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:49:05.463 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:49:05.475 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-16T14:49:04.895798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39141939,"num_objects":68,"num_object_clones":18,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":5,"num_read":394,"num_read_kb":45764,"num_write":666,"num_write_kb":52476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":5390336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":622,"ondisk_log_size":622,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":11,"num_per_pool_osds":11,"num_per_pool_omap_osds":11,"kb":1038090240,"kb_used":399700,"kb_used_data":81984,"kb_used_omap":17,"kb_used_meta":317550,"kb_avail":1037690540,"statfs":{"total":1063004405760,"available":1062595112960,"internally_reserved":0,"allocated":83951616,"data_stored":96496618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17537,"internal_metadata":325172095},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":917504,"num_objects":4,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":24,"num_read_kb":12014,"num_write":87,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":5390336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.046313"},"pg_stats":[{"pgid":"3.f","version":"38'15","reported_seq":102,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T14:49:00.854391+0000","last_change":"2024-09-16T14:49:00.854684+0000","last_active":"2024-09-16T14:49:00.854391+0000","last_peered":"2024-09-16T14:49:00.854391+0000","last_clean":"2024-09-16T14:49:00.854391+0000","last_became_active":"2024-09-16T14:48:35.075111+0000","last_became_peered":"2024-09-16T14:48:35.075111+0000","last_unstale":"2024-09-16T14:49:00.854391+0000","last_undegraded":"2024-09-16T14:49:00.854391+0000","last_fullsized":"2024-09-16T14:49:00.854391+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:58.979883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047847800000000001,"stat_sum":{"num_bytes":1835008,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":8,"num_write_kb":2192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"18","length":"1"}]},{"pgid":"3.e","version":"55'22","reported_seq":118,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T14:49:01.869004+0000","last_change":"2024-09-16T14:49:01.869737+0000","last_active":"2024-09-16T14:49:01.869004+0000","last_peered":"2024-09-16T14:49:01.869004+0000","last_clean":"2024-09-16T14:49:01.869004+0000","last_became_active":"2024-09-16T14:48:35.075626+0000","last_became_peered":"2024-09-16T14:48:35.075626+0000","last_unstale":"2024-09-16T14:49:01.869004+0000","last_undegraded":"2024-09-16T14:49:01.869004+0000","last_fullsized":"2024-09-16T14:49:01.869004+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.908348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012205996,"stat_sum":{"num_bytes":1490944,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":4435,"num_write":48,"num_write_kb":2016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"}]},{"pgid":"3.d","version":"52'36","reported_seq":133,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T14:49:00.855069+0000","last_change":"2024-09-16T14:49:00.855400+0000","last_active":"2024-09-16T14:49:00.855069+0000","last_peered":"2024-09-16T14:49:00.855069+0000","last_clean":"2024-09-16T14:49:00.855069+0000","last_became_active":"2024-09-16T14:48:35.074954+0000","last_became_peered":"2024-09-16T14:48:35.074954+0000","last_unstale":"2024-09-16T14:49:00.855069+0000","last_undegraded":"2024-09-16T14:49:00.855069+0000","last_fullsized":"2024-09-16T14:49:00.855069+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:34.761306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047027400000000002,"stat_sum":{"num_bytes":2981888,"num_objects":6,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":25,"num_read_kb":1209,"num_write":44,"num_write_kb":2912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,11,1,3,4,0],"acting":[7,5,9,11,1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"18","length":"1"}]},{"pgid":"3.c","version":"54'17","reported_seq":116,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T14:49:00.873137+0000","last_change":"2024-09-16T14:49:00.851002+0000","last_active":"2024-09-16T14:49:00.873137+0000","last_peered":"2024-09-16T14:49:00.873137+0000","last_clean":"2024-09-16T14:49:00.873137+0000","last_became_active":"2024-09-16T14:48:35.074891+0000","last_became_peered":"2024-09-16T14:48:35.074891+0000","last_unstale":"2024-09-16T14:49:00.873137+0000","last_undegraded":"2024-09-16T14:49:00.873137+0000","last_fullsized":"2024-09-16T14:49:00.873137+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:03.774203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000468042,"stat_sum":{"num_bytes":2441216,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":21,"num_read_kb":4614,"num_write":13,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4,10,9,2,0,11],"acting":[5,3,4,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"18","length":"1"}]},{"pgid":"3.b","version":"52'28","reported_seq":114,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T14:49:00.854083+0000","last_change":"2024-09-16T14:49:00.854375+0000","last_active":"2024-09-16T14:49:00.854083+0000","last_peered":"2024-09-16T14:49:00.854083+0000","last_clean":"2024-09-16T14:49:00.854083+0000","last_became_active":"2024-09-16T14:48:35.076810+0000","last_became_peered":"2024-09-16T14:48:35.076810+0000","last_unstale":"2024-09-16T14:49:00.854083+0000","last_undegraded":"2024-09-16T14:49:00.854083+0000","last_fullsized":"2024-09-16T14:49:00.854083+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:59.031166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048368800000000002,"stat_sum":{"num_bytes":1785856,"num_objects":5,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":2452,"num_write":14,"num_write_kb":1744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,1,9],"acting":[3,0,4,8,11,7,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"18","length":"1"}]},{"pgid":"3.a","version":"52'24","reported_seq":142,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T14:49:01.857565+0000","last_change":"2024-09-16T14:49:01.858877+0000","last_active":"2024-09-16T14:49:01.857565+0000","last_peered":"2024-09-16T14:49:01.857565+0000","last_clean":"2024-09-16T14:49:01.857565+0000","last_became_active":"2024-09-16T14:49:00.861296+0000","last_became_peered":"2024-09-16T14:49:00.861296+0000","last_unstale":"2024-09-16T14:49:01.857565+0000","last_undegraded":"2024-09-16T14:49:01.857565+0000","last_fullsized":"2024-09-16T14:49:01.857565+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:48.641677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001632386,"stat_sum":{"num_bytes":2965504,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":1061,"num_write":33,"num_write_kb":3248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":3293184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,5,10,8,9,3,7],"acting":[11,4,5,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"}]},{"pgid":"3.9","version":"52'23","reported_seq":120,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T14:49:00.858050+0000","last_change":"2024-09-16T14:49:00.858441+0000","last_active":"2024-09-16T14:49:00.858050+0000","last_peered":"2024-09-16T14:49:00.858050+0000","last_clean":"2024-09-16T14:49:00.858050+0000","last_became_active":"2024-09-16T14:48:35.075076+0000","last_became_peered":"2024-09-16T14:48:35.075076+0000","last_unstale":"2024-09-16T14:49:00.858050+0000","last_undegraded":"2024-09-16T14:49:00.858050+0000","last_fullsized":"2024-09-16T14:49:00.858050+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:46.349013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052462899999999998,"stat_sum":{"num_bytes":2572288,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":582,"num_write":14,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"18","length":"1"}]},{"pgid":"3.8","version":"55'69","reported_seq":170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T14:49:01.884535+0000","last_change":"2024-09-16T14:49:01.885289+0000","last_active":"2024-09-16T14:49:01.884535+0000","last_peered":"2024-09-16T14:49:01.884535+0000","last_clean":"2024-09-16T14:49:01.884535+0000","last_became_active":"2024-09-16T14:48:34.881078+0000","last_became_peered":"2024-09-16T14:48:34.881078+0000","last_unstale":"2024-09-16T14:49:01.884535+0000","last_undegraded":"2024-09-16T14:49:01.884535+0000","last_fullsized":"2024-09-16T14:49:01.884535+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:03.772938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027757232999999999,"stat_sum":{"num_bytes":6750208,"num_objects":8,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":37,"num_read_kb":15126,"num_write":70,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2,4,3,5,10,8],"acting":[11,1,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"}]},{"pgid":"1.0","version":"20'137","reported_seq":217,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T14:48:59.857549+0000","last_change":"2024-09-16T14:48:22.359960+0000","last_active":"2024-09-16T14:48:59.857549+0000","last_peered":"2024-09-16T14:48:59.857549+0000","last_clean":"2024-09-16T14:48:59.857549+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:48:59.857549+0000","last_undegraded":"2024-09-16T14:48:59.857549+0000","last_fullsized":"2024-09-16T14:48:59.857549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:48:21.118926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:37.878583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":48,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T14:48:59.851465+0000","last_change":"2024-09-16T14:48:26.159458+0000","last_active":"2024-09-16T14:48:59.851465+0000","last_peered":"2024-09-16T14:48:59.851465+0000","last_clean":"2024-09-16T14:48:59.851465+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:48:59.851465+0000","last_undegraded":"2024-09-16T14:48:59.851465+0000","last_fullsized":"2024-09-16T14:48:59.851465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:33.086957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"54'59","reported_seq":152,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T14:49:00.855844+0000","last_change":"2024-09-16T14:49:00.855844+0000","last_active":"2024-09-16T14:49:00.855844+0000","last_peered":"2024-09-16T14:49:00.855844+0000","last_clean":"2024-09-16T14:49:00.855844+0000","last_became_active":"2024-09-16T14:48:35.077801+0000","last_became_peered":"2024-09-16T14:48:35.077801+0000","last_unstale":"2024-09-16T14:49:00.855844+0000","last_undegraded":"2024-09-16T14:49:00.855844+0000","last_fullsized":"2024-09-16T14:49:00.855844+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:35.886560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018401069999999999,"stat_sum":{"num_bytes":3227648,"num_objects":6,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":29,"num_read_kb":6823,"num_write":30,"num_write_kb":5872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,7,4,5,1,8,10],"acting":[3,9,7,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"18","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T14:48:59.857451+0000","last_change":"2024-09-16T14:48:26.568303+0000","last_active":"2024-09-16T14:48:59.857451+0000","last_peered":"2024-09-16T14:48:59.857451+0000","last_clean":"2024-09-16T14:48:59.857451+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:48:59.857451+0000","last_undegraded":"2024-09-16T14:48:59.857451+0000","last_fullsized":"2024-09-16T14:48:59.857451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:35.481460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T14:49:01.851507+0000","last_change":"2024-09-16T14:49:01.851900+0000","last_active":"2024-09-16T14:49:01.851507+0000","last_peered":"2024-09-16T14:49:01.851507+0000","last_clean":"2024-09-16T14:49:01.851507+0000","last_became_active":"2024-09-16T14:48:35.074694+0000","last_became_peered":"2024-09-16T14:48:35.074694+0000","last_unstale":"2024-09-16T14:49:01.851507+0000","last_undegraded":"2024-09-16T14:49:01.851507+0000","last_fullsized":"2024-09-16T14:49:01.851507+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:49.712487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052877499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,2,1],"acting":[0,4,10,11,5,9,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T14:48:59.859439+0000","last_change":"2024-09-16T14:48:26.156468+0000","last_active":"2024-09-16T14:48:59.859439+0000","last_peered":"2024-09-16T14:48:59.859439+0000","last_clean":"2024-09-16T14:48:59.859439+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:48:59.859439+0000","last_undegraded":"2024-09-16T14:48:59.859439+0000","last_fullsized":"2024-09-16T14:48:59.859439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:25.252442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"49'20","reported_seq":115,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T14:49:01.869817+0000","last_change":"2024-09-16T14:49:01.872235+0000","last_active":"2024-09-16T14:49:01.869817+0000","last_peered":"2024-09-16T14:49:01.869817+0000","last_clean":"2024-09-16T14:49:01.869817+0000","last_became_active":"2024-09-16T14:48:35.074494+0000","last_became_peered":"2024-09-16T14:48:35.074494+0000","last_unstale":"2024-09-16T14:49:01.869817+0000","last_undegraded":"2024-09-16T14:49:01.869817+0000","last_fullsized":"2024-09-16T14:49:01.869817+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:46.564680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014474152000000001,"stat_sum":{"num_bytes":1294336,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":1109,"num_write":12,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,2,1,7,8,0,5],"acting":[11,9,2,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":49,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T14:48:59.851377+0000","last_change":"2024-09-16T14:48:26.159919+0000","last_active":"2024-09-16T14:48:59.851377+0000","last_peered":"2024-09-16T14:48:59.851377+0000","last_clean":"2024-09-16T14:48:59.851377+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:48:59.851377+0000","last_undegraded":"2024-09-16T14:48:59.851377+0000","last_fullsized":"2024-09-16T14:48:59.851377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:25.005776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"52'20","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T14:49:00.851405+0000","last_change":"2024-09-16T14:49:00.851703+0000","last_active":"2024-09-16T14:49:00.851405+0000","last_peered":"2024-09-16T14:49:00.851405+0000","last_clean":"2024-09-16T14:49:00.851405+0000","last_became_active":"2024-09-16T14:48:35.075027+0000","last_became_peered":"2024-09-16T14:48:35.075027+0000","last_unstale":"2024-09-16T14:49:00.851405+0000","last_undegraded":"2024-09-16T14:49:00.851405+0000","last_fullsized":"2024-09-16T14:49:00.851405+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:43.560786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040054700000000001,"stat_sum":{"num_bytes":622592,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":16,"num_read_kb":857,"num_write":27,"num_write_kb":608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0,3,11,5,9,10],"acting":[4,7,0,3,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"18","length":"1"}]},{"pgid":"3.5","version":"54'24","reported_seq":142,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T14:49:00.900593+0000","last_change":"2024-09-16T14:49:00.900845+0000","last_active":"2024-09-16T14:49:00.900593+0000","last_peered":"2024-09-16T14:49:00.900593+0000","last_clean":"2024-09-16T14:49:00.900593+0000","last_became_active":"2024-09-16T14:49:00.870775+0000","last_became_peered":"2024-09-16T14:49:00.870775+0000","last_unstale":"2024-09-16T14:49:00.900593+0000","last_undegraded":"2024-09-16T14:49:00.900593+0000","last_fullsized":"2024-09-16T14:49:00.900593+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:57.502051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064147049999999997,"stat_sum":{"num_bytes":1425408,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":11,"num_read_kb":1971,"num_write":9,"num_write_kb":2208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":2097152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,0,7,8,4,10],"acting":[5,3,2,0,7,8,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"18","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T14:49:00.858373+0000","last_change":"2024-09-16T14:48:26.161822+0000","last_active":"2024-09-16T14:49:00.858373+0000","last_peered":"2024-09-16T14:49:00.858373+0000","last_clean":"2024-09-16T14:49:00.858373+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:49:00.858373+0000","last_undegraded":"2024-09-16T14:49:00.858373+0000","last_fullsized":"2024-09-16T14:49:00.858373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:48.998147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"52'40","reported_seq":132,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T14:49:01.863368+0000","last_change":"2024-09-16T14:49:01.863708+0000","last_active":"2024-09-16T14:49:01.863368+0000","last_peered":"2024-09-16T14:49:01.863368+0000","last_clean":"2024-09-16T14:49:01.863368+0000","last_became_active":"2024-09-16T14:48:35.075289+0000","last_became_peered":"2024-09-16T14:48:35.075289+0000","last_unstale":"2024-09-16T14:49:01.863368+0000","last_undegraded":"2024-09-16T14:49:01.863368+0000","last_fullsized":"2024-09-16T14:49:01.863368+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:34.771117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047893099999999999,"stat_sum":{"num_bytes":3391488,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":16,"num_read_kb":1350,"num_write":56,"num_write_kb":5024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,9,5,10,2,3,0],"acting":[1,11,9,5,10,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-16T14:48:59.855660+0000","last_change":"2024-09-16T14:48:26.156486+0000","last_active":"2024-09-16T14:48:59.855660+0000","last_peered":"2024-09-16T14:48:59.855660+0000","last_clean":"2024-09-16T14:48:59.855660+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:48:59.855660+0000","last_undegraded":"2024-09-16T14:48:59.855660+0000","last_fullsized":"2024-09-16T14:48:59.855660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.015966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"52'62","reported_seq":154,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T14:49:00.853963+0000","last_change":"2024-09-16T14:49:00.854410+0000","last_active":"2024-09-16T14:49:00.853963+0000","last_peered":"2024-09-16T14:49:00.853963+0000","last_clean":"2024-09-16T14:49:00.853963+0000","last_became_active":"2024-09-16T14:48:35.075213+0000","last_became_peered":"2024-09-16T14:48:35.075213+0000","last_unstale":"2024-09-16T14:49:00.853963+0000","last_undegraded":"2024-09-16T14:49:00.853963+0000","last_fullsized":"2024-09-16T14:49:00.853963+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:38.895447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058602999999999997,"stat_sum":{"num_bytes":4112384,"num_objects":6,"num_object_clones":1,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":30,"num_read_kb":3866,"num_write":89,"num_write_kb":5456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"18","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T14:49:00.858473+0000","last_change":"2024-09-16T14:48:26.161840+0000","last_active":"2024-09-16T14:49:00.858473+0000","last_peered":"2024-09-16T14:49:00.858473+0000","last_clean":"2024-09-16T14:49:00.858473+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:49:00.858473+0000","last_undegraded":"2024-09-16T14:49:00.858473+0000","last_fullsized":"2024-09-16T14:49:00.858473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.436189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"51'23","reported_seq":117,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T14:49:01.851958+0000","last_change":"2024-09-16T14:49:01.852265+0000","last_active":"2024-09-16T14:49:01.851958+0000","last_peered":"2024-09-16T14:49:01.851958+0000","last_clean":"2024-09-16T14:49:01.851958+0000","last_became_active":"2024-09-16T14:48:35.076301+0000","last_became_peered":"2024-09-16T14:48:35.076301+0000","last_unstale":"2024-09-16T14:49:01.851958+0000","last_undegraded":"2024-09-16T14:49:01.851958+0000","last_fullsized":"2024-09-16T14:49:01.851958+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:50.342709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061528699999999995,"stat_sum":{"num_bytes":1654784,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":197,"num_write":16,"num_write_kb":3296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,5],"acting":[0,1,4,7,8,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"c","length":"1"},{"start":"12","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":40,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-16T14:49:00.856267+0000","last_change":"2024-09-16T14:48:33.867086+0000","last_active":"2024-09-16T14:49:00.856267+0000","last_peered":"2024-09-16T14:49:00.856267+0000","last_clean":"2024-09-16T14:49:00.856267+0000","last_became_active":"2024-09-16T14:48:33.866805+0000","last_became_peered":"2024-09-16T14:48:33.866805+0000","last_unstale":"2024-09-16T14:49:00.856267+0000","last_undegraded":"2024-09-16T14:49:00.856267+0000","last_fullsized":"2024-09-16T14:49:00.856267+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:52.480623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2],"acting":[10,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":38551552,"num_objects":64,"num_object_clones":18,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":5,"num_read":268,"num_read_kb":45655,"num_write":483,"num_write_kb":49536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":5390336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81244160,"data_stored":98181120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":482,"ondisk_log_size":482,"up":128,"acting":128,"num_store_stats":11},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411338,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40100,"kb_used_data":8984,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94331740,"statfs":{"total":96636764160,"available":96595701760,"internally_reserved":0,"allocated":9199616,"data_stored":10724959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":10,"up_from":18,"seq":77309411340,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39640,"kb_used_data":8520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94332200,"statfs":{"total":96636764160,"available":96596172800,"internally_reserved":0,"allocated":8724480,"data_stored":9654403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":6,"up_from":0,"seq":176093659141,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":18,"seq":77309411340,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38860,"kb_used_data":7740,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94332980,"statfs":{"total":96636764160,"available":96596971520,"internally_reserved":0,"allocated":7925760,"data_stored":9389138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]}]},{"osd":0,"up_from":18,"seq":77309411338,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37336,"kb_used_data":6216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94334504,"statfs":{"total":96636764160,"available":96598532096,"internally_reserved":0,"allocated":6365184,"data_stored":7488575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":5,"up_from":18,"seq":77309411339,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38956,"kb_used_data":7836,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94332884,"statfs":{"total":96636764160,"available":96596873216,"internally_reserved":0,"allocated":8024064,"data_stored":9122898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]}]},{"osd":9,"up_from":17,"seq":73014444043,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34968,"kb_used_data":7948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336872,"statfs":{"total":96636764160,"available":96600956928,"internally_reserved":0,"allocated":8138752,"data_stored":9249678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":8,"up_from":17,"seq":73014444043,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33356,"kb_used_data":6336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338484,"statfs":{"total":96636764160,"available":96602607616,"internally_reserved":0,"allocated":6488064,"data_stored":7070606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]},{"osd":7,"up_from":17,"seq":73014444043,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34108,"kb_used_data":7088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337732,"statfs":{"total":96636764160,"available":96601837568,"internally_reserved":0,"allocated":7258112,"data_stored":8390239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]},{"osd":4,"up_from":17,"seq":73014444042,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35472,"kb_used_data":8452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336368,"statfs":{"total":96636764160,"available":96600440832,"internally_reserved":0,"allocated":8654848,"data_stored":10077070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.248}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]}]},{"osd":3,"up_from":17,"seq":73014444044,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35168,"kb_used_data":8148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336672,"statfs":{"total":96636764160,"available":96600752128,"internally_reserved":0,"allocated":8343552,"data_stored":10015630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":2,"up_from":17,"seq":73014444043,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":4716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":4829184,"data_stored":5313422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6504448,"data_stored":8187904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8433664,"data_stored":10571776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4284416,"data_stored":5025792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7798784,"data_stored":9728000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8634368,"data_stored":10313728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7413760,"data_stored":8777728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6119424,"data_stored":7516160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":6418432,"data_stored":7258112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":7593984,"data_stored":8962048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":8863744,"data_stored":10461184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":9179136,"data_stored":11378688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:49:05.478 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pgp_num 16 2024-09-16T14:49:05.536 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:05.537 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:05.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:49:05.590 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:05.589+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:05.660 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:05.659+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:05.669 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:49:05.761 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:05.760+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:05.812 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:05.813 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:05.821 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:49:05.865 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:05.864+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:05.900 INFO:teuthology.orchestra.run.smithi040.stderr:set pool 3 pgp_num to 16 2024-09-16T14:49:05.901 INFO:tasks.rados.rados.0.smithi040.stdout:216: done (0 left) 2024-09-16T14:49:05.901 INFO:tasks.rados.rados.0.smithi040.stdout:217: setattr oid 41 current snap is 17 2024-09-16T14:49:05.902 INFO:tasks.rados.rados.0.smithi040.stdout:218: delete oid 21 current snap is 17 2024-09-16T14:49:05.908 INFO:tasks.rados.rados.0.smithi040.stdout:218: done (1 left) 2024-09-16T14:49:05.908 INFO:tasks.rados.rados.0.smithi040.stdout:219: rollback oid 23 current snap is 17 2024-09-16T14:49:05.908 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 23 to 14 2024-09-16T14:49:05.908 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 35 current snap is 17 2024-09-16T14:49:05.908 INFO:tasks.rados.rados.0.smithi040.stdout:220: seq_num 91 ranges {1458176=770048,2228224=475136,2703360=294912} 2024-09-16T14:49:05.920 INFO:tasks.rados.rados.0.smithi040.stdout:220: writing smithi04040151-35 from 1458176 to 2228224 tid 1 2024-09-16T14:49:05.923 INFO:tasks.rados.rados.0.smithi040.stdout:220: writing smithi04040151-35 from 2228224 to 2703360 tid 2 2024-09-16T14:49:05.925 INFO:tasks.rados.rados.0.smithi040.stdout:220: writing smithi04040151-35 from 2703360 to 2998272 tid 3 2024-09-16T14:49:05.925 INFO:tasks.rados.rados.0.smithi040.stdout:221: read oid 42 snap -1 2024-09-16T14:49:05.925 INFO:tasks.rados.rados.0.smithi040.stdout:221: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-16T14:49:05.925 INFO:tasks.rados.rados.0.smithi040.stdout:222: copy_from oid 27 from oid 29 current snap is 17 2024-09-16T14:49:05.925 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 41 v 17 (ObjNum 68 snap 4 seq_num 68) dirty exists 2024-09-16T14:49:05.925 INFO:tasks.rados.rados.0.smithi040.stdout:219: finishing rollback tid 0 to smithi04040151-23 2024-09-16T14:49:05.926 INFO:tasks.rados.rados.0.smithi040.stdout:219: finishing rollback tid 1 to smithi04040151-23 2024-09-16T14:49:05.926 INFO:tasks.rados.rados.0.smithi040.stdout:219: finishing rollback tid 2 to smithi04040151-23 2024-09-16T14:49:05.926 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 23 v 25 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-16T14:49:05.926 INFO:tasks.rados.rados.0.smithi040.stdout:217: done (4 left) 2024-09-16T14:49:05.926 INFO:tasks.rados.rados.0.smithi040.stdout:219: done (3 left) 2024-09-16T14:49:05.926 INFO:tasks.rados.rados.0.smithi040.stdout:223: snap_remove snap 12 2024-09-16T14:49:05.931 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:05.931 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:05.941 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:49:05.985 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:05.984+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:06.049 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:06.049 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:06.057 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:49:06.064 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:06.063+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:06.165 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:06.164+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:06.205 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:06.205 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:06.214 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:49:06.299 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:06.298+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:06.360 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:06.361 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:06.367 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:06.366+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:06.370 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:49:06.468 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:06.467+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:06.516 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:06.517 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:06.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:49:06.569 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:06.568+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:06.632 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:06.632 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:06.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:49:06.669 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:06.668+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:06.746 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:06.747 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:06.754 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:49:06.756 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:49:06.770 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:06.770+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:06.872 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:06.870+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:06.905 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:06.905 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:06.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:49:06.924 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:49:06.925 INFO:tasks.rados.rados.0.smithi040.stdout:223: done (3 left) 2024-09-16T14:49:06.925 INFO:tasks.rados.rados.0.smithi040.stdout:224: read oid 34 snap -1 2024-09-16T14:49:06.925 INFO:tasks.rados.rados.0.smithi040.stdout:224: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-16T14:49:06.925 INFO:tasks.rados.rados.0.smithi040.stdout:225: snap_remove snap 8 2024-09-16T14:49:06.931 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:06.931 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:06.966 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:06.967 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:06.972 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:06.971+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:06.998 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:07.016 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:07.016 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:07.033 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:07.033 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:07.040 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:07.041 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:07.041 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:07.041 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:07.041 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:07.050 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:49:07.051 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:07.069 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:07.069 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:07.073 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:07.072+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:07.088 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:07.088 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:07.105 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:07.107 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:07.122 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:07.122 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:07.139 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:07.139 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:07.156 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:07.156 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:07.168 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:07.168 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:07.168 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:07.168 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:07.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.168 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 32/27 ECSubWrite(tid=21, reqid=client.4500.0:214, at_version=32'12, trim_to=0'0, roll_forward_to=31'10))", 2024-09-16T14:49:07.168 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:38.899885+0000", 2024-09-16T14:49:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.265636938, 2024-09-16T14:49:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0038085100000000002, 2024-09-16T14:49:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.899885+0000", 2024-09-16T14:49:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.899885+0000", 2024-09-16T14:49:07.169 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.899885+0000", 2024-09-16T14:49:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.900007+0000", 2024-09-16T14:49:07.170 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000121595 2024-09-16T14:49:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.900029+0000", 2024-09-16T14:49:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1818999999999999e-05 2024-09-16T14:49:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.900040+0000", 2024-09-16T14:49:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.117e-05 2024-09-16T14:49:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.171 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.900199+0000", 2024-09-16T14:49:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00015960700000000001 2024-09-16T14:49:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.900209+0000", 2024-09-16T14:49:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.013e-05 2024-09-16T14:49:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.903672+0000", 2024-09-16T14:49:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0034625089999999999 2024-09-16T14:49:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:38.903694+0000", 2024-09-16T14:49:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2035000000000002e-05 2024-09-16T14:49:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 37/27 ECSubWrite(tid=33, reqid=osd.1.0:33, at_version=37'24, trim_to=0'0, roll_forward_to=35'22))", 2024-09-16T14:49:07.173 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.920208+0000", 2024-09-16T14:49:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.245314552, 2024-09-16T14:49:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044137789999999996, 2024-09-16T14:49:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920208+0000", 2024-09-16T14:49:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920208+0000", 2024-09-16T14:49:07.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920208+0000", 2024-09-16T14:49:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920231+0000", 2024-09-16T14:49:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3153999999999999e-05 2024-09-16T14:49:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.175 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920262+0000", 2024-09-16T14:49:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1189999999999998e-05 2024-09-16T14:49:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920271+0000", 2024-09-16T14:49:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7590000000000003e-06 2024-09-16T14:49:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920450+0000", 2024-09-16T14:49:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000179018 2024-09-16T14:49:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.176 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.920459+0000", 2024-09-16T14:49:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7860000000000002e-06 2024-09-16T14:49:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924581+0000", 2024-09-16T14:49:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.004122466 2024-09-16T14:49:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.924622+0000", 2024-09-16T14:49:07.177 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.0632e-05 2024-09-16T14:49:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.0s2 37/27 ECSubWrite(tid=109, reqid=osd.11.0:109, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.940727+0000", 2024-09-16T14:49:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.224795171, 2024-09-16T14:49:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0045177710000000003, 2024-09-16T14:49:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.178 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940727+0000", 2024-09-16T14:49:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940727+0000", 2024-09-16T14:49:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940727+0000", 2024-09-16T14:49:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940772+0000", 2024-09-16T14:49:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.5407000000000001e-05 2024-09-16T14:49:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940783+0000", 2024-09-16T14:49:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1239e-05 2024-09-16T14:49:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940792+0000", 2024-09-16T14:49:07.180 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7809999999999994e-06 2024-09-16T14:49:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940922+0000", 2024-09-16T14:49:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000129796 2024-09-16T14:49:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940929+0000", 2024-09-16T14:49:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3270000000000003e-06 2024-09-16T14:49:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.181 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945214+0000", 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0042845549999999998 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945245+0000", 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1151e-05 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 37/27 ECSubWrite(tid=110, reqid=osd.11.0:110, at_version=37'35, trim_to=0'0, roll_forward_to=35'31))", 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.941339+0000", 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.224183108999998, 2024-09-16T14:49:07.182 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0052892429999999999, 2024-09-16T14:49:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.941339+0000", 2024-09-16T14:49:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.941339+0000", 2024-09-16T14:49:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.941338+0000", 2024-09-16T14:49:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.941365+0000", 2024-09-16T14:49:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6135999999999998e-05 2024-09-16T14:49:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.941529+0000", 2024-09-16T14:49:07.184 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00016411799999999999 2024-09-16T14:49:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.941541+0000", 2024-09-16T14:49:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2381000000000001e-05 2024-09-16T14:49:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.941593+0000", 2024-09-16T14:49:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1974000000000001e-05 2024-09-16T14:49:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.185 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.941603+0000", 2024-09-16T14:49:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5480000000000007e-06 2024-09-16T14:49:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.946603+0000", 2024-09-16T14:49:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0050008530000000004 2024-09-16T14:49:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.946629+0000", 2024-09-16T14:49:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5089e-05 2024-09-16T14:49:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.186 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 42/27 ECSubWrite(tid=39, reqid=client.4500.0:365, at_version=42'28, trim_to=0'0, roll_forward_to=38'26))", 2024-09-16T14:49:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.953352+0000", 2024-09-16T14:49:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 18.212170831000002, 2024-09-16T14:49:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.004103102, 2024-09-16T14:49:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.953352+0000", 2024-09-16T14:49:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.953352+0000", 2024-09-16T14:49:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.953351+0000", 2024-09-16T14:49:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.188 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.953802+0000", 2024-09-16T14:49:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00045159099999999999 2024-09-16T14:49:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.953825+0000", 2024-09-16T14:49:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.228e-05 2024-09-16T14:49:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.953839+0000", 2024-09-16T14:49:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4129e-05 2024-09-16T14:49:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.189 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.953915+0000", 2024-09-16T14:49:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5704000000000006e-05 2024-09-16T14:49:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.953928+0000", 2024-09-16T14:49:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3899e-05 2024-09-16T14:49:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.957414+0000", 2024-09-16T14:49:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0034851280000000001 2024-09-16T14:49:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.957455+0000", 2024-09-16T14:49:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.1097999999999999e-05 2024-09-16T14:49:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=128, reqid=client.4500.0:369, at_version=42'38, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 18.205545316999999, 2024-09-16T14:49:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060794769999999998, 2024-09-16T14:49:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.191 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.192 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959976+0000", 2024-09-16T14:49:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960436+0000", 2024-09-16T14:49:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00046022400000000002 2024-09-16T14:49:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960455+0000", 2024-09-16T14:49:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8162999999999999e-05 2024-09-16T14:49:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960468+0000", 2024-09-16T14:49:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3135999999999999e-05 2024-09-16T14:49:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960527+0000", 2024-09-16T14:49:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8859e-05 2024-09-16T14:49:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960533+0000", 2024-09-16T14:49:07.194 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:49:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.966018+0000", 2024-09-16T14:49:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0054845069999999996 2024-09-16T14:49:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.966057+0000", 2024-09-16T14:49:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.8927999999999997e-05 2024-09-16T14:49:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=129, reqid=client.4500.0:370, at_version=42'39, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 18.204289558999999, 2024-09-16T14:49:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074279639999999996, 2024-09-16T14:49:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961232+0000", 2024-09-16T14:49:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962118+0000", 2024-09-16T14:49:07.197 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00088592399999999996 2024-09-16T14:49:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962132+0000", 2024-09-16T14:49:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4127e-05 2024-09-16T14:49:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962142+0000", 2024-09-16T14:49:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0181e-05 2024-09-16T14:49:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.198 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962192+0000", 2024-09-16T14:49:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9826e-05 2024-09-16T14:49:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962202+0000", 2024-09-16T14:49:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8400000000000007e-06 2024-09-16T14:49:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968622+0000", 2024-09-16T14:49:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00642021 2024-09-16T14:49:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.199 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968661+0000", 2024-09-16T14:49:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.8670000000000001e-05 2024-09-16T14:49:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=130, reqid=client.4500.0:371, at_version=42'40, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 18.202954303999999, 2024-09-16T14:49:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0061144900000000002, 2024-09-16T14:49:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.200 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962567+0000", 2024-09-16T14:49:07.201 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962595+0000", 2024-09-16T14:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7824999999999999e-05 2024-09-16T14:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962611+0000", 2024-09-16T14:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962620+0000", 2024-09-16T14:49:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8259999999999999e-06 2024-09-16T14:49:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.963285+0000", 2024-09-16T14:49:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00066560100000000004 2024-09-16T14:49:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.963295+0000", 2024-09-16T14:49:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0036e-05 2024-09-16T14:49:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.203 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968667+0000", 2024-09-16T14:49:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0053715960000000002 2024-09-16T14:49:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968683+0000", 2024-09-16T14:49:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5522000000000001e-05 2024-09-16T14:49:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.204 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 43/27 ECSubWrite(tid=42, reqid=osd.1.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=43'30))", 2024-09-16T14:49:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:49.851949+0000", 2024-09-16T14:49:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 17.313572922999999, 2024-09-16T14:49:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0057984890000000004, 2024-09-16T14:49:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851949+0000", 2024-09-16T14:49:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.205 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851949+0000", 2024-09-16T14:49:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851949+0000", 2024-09-16T14:49:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851962+0000", 2024-09-16T14:49:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3189e-05 2024-09-16T14:49:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.206 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851979+0000", 2024-09-16T14:49:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6724000000000001e-05 2024-09-16T14:49:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851987+0000", 2024-09-16T14:49:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.568e-06 2024-09-16T14:49:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.852055+0000", 2024-09-16T14:49:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.7519000000000002e-05 2024-09-16T14:49:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.207 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.852065+0000", 2024-09-16T14:49:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0111e-05 2024-09-16T14:49:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.857719+0000", 2024-09-16T14:49:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056545429999999997 2024-09-16T14:49:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.857748+0000", 2024-09-16T14:49:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8609e-05 2024-09-16T14:49:07.208 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 43/27 ECSubWrite(tid=133, reqid=osd.11.0:133, at_version=43'42, trim_to=0'0, roll_forward_to=42'40))", 2024-09-16T14:49:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 17.311746509999999, 2024-09-16T14:49:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073093740000000004, 2024-09-16T14:49:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.209 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.210 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854515+0000", 2024-09-16T14:49:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00073950999999999997 2024-09-16T14:49:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854527+0000", 2024-09-16T14:49:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2012e-05 2024-09-16T14:49:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854535+0000", 2024-09-16T14:49:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:49:07.211 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854952+0000", 2024-09-16T14:49:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00041779399999999998 2024-09-16T14:49:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854961+0000", 2024-09-16T14:49:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7749999999999998e-06 2024-09-16T14:49:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.861052+0000", 2024-09-16T14:49:07.212 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006090608 2024-09-16T14:49:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.861085+0000", 2024-09-16T14:49:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3442999999999998e-05 2024-09-16T14:49:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 43/27 ECSubWrite(tid=134, reqid=osd.11.0:133, at_version=0'0, trim_to=0'0, roll_forward_to=43'42))", 2024-09-16T14:49:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:49.862027+0000", 2024-09-16T14:49:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 17.303495118000001, 2024-09-16T14:49:07.213 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0041326069999999999, 2024-09-16T14:49:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.862027+0000", 2024-09-16T14:49:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.862027+0000", 2024-09-16T14:49:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.214 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.862026+0000", 2024-09-16T14:49:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.862055+0000", 2024-09-16T14:49:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8343e-05 2024-09-16T14:49:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.862070+0000", 2024-09-16T14:49:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5474000000000001e-05 2024-09-16T14:49:07.215 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.862079+0000", 2024-09-16T14:49:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8510000000000005e-06 2024-09-16T14:49:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.862128+0000", 2024-09-16T14:49:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9276000000000001e-05 2024-09-16T14:49:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.862139+0000", 2024-09-16T14:49:07.216 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0208e-05 2024-09-16T14:49:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.866128+0000", 2024-09-16T14:49:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0039897659999999996 2024-09-16T14:49:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.866160+0000", 2024-09-16T14:49:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1557999999999998e-05 2024-09-16T14:49:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.217 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s2 51/27 ECSubWrite(tid=55, reqid=osd.5.0:55, at_version=51'16, trim_to=0'0, roll_forward_to=49'13))", 2024-09-16T14:49:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.880577+0000", 2024-09-16T14:49:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.2849456929999992, 2024-09-16T14:49:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0039843559999999997, 2024-09-16T14:49:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880577+0000", 2024-09-16T14:49:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.218 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880577+0000", 2024-09-16T14:49:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880576+0000", 2024-09-16T14:49:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880605+0000", 2024-09-16T14:49:07.219 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8963999999999999e-05 2024-09-16T14:49:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880616+0000", 2024-09-16T14:49:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1206000000000001e-05 2024-09-16T14:49:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880626+0000", 2024-09-16T14:49:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4029999999999996e-06 2024-09-16T14:49:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.220 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880707+0000", 2024-09-16T14:49:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1212999999999996e-05 2024-09-16T14:49:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880716+0000", 2024-09-16T14:49:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2609999999999999e-06 2024-09-16T14:49:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.884537+0000", 2024-09-16T14:49:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0038205729999999999 2024-09-16T14:49:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.221 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.884561+0000", 2024-09-16T14:49:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4124000000000001e-05 2024-09-16T14:49:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 51/27 ECSubWrite(tid=169, reqid=osd.11.0:169, at_version=51'52, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.2843365890000005, 2024-09-16T14:49:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006781848, 2024-09-16T14:49:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.222 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881185+0000", 2024-09-16T14:49:07.223 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881208+0000", 2024-09-16T14:49:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2823000000000002e-05 2024-09-16T14:49:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881218+0000", 2024-09-16T14:49:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3929999999999997e-06 2024-09-16T14:49:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.224 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881225+0000", 2024-09-16T14:49:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1629999999999999e-06 2024-09-16T14:49:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881279+0000", 2024-09-16T14:49:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4147000000000002e-05 2024-09-16T14:49:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881288+0000", 2024-09-16T14:49:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4630000000000007e-06 2024-09-16T14:49:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.225 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.887941+0000", 2024-09-16T14:49:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0066530210000000003 2024-09-16T14:49:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.887968+0000", 2024-09-16T14:49:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6228999999999999e-05 2024-09-16T14:49:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 51/27 ECSubWrite(tid=173, reqid=osd.11.0:173, at_version=51'54, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:07.226 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.2841853620000006, 2024-09-16T14:49:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0080949799999999999, 2024-09-16T14:49:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:07.227 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881336+0000", 2024-09-16T14:49:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881356+0000", 2024-09-16T14:49:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9334000000000002e-05 2024-09-16T14:49:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.228 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881366+0000", 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0742e-05 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881374+0000", 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5020000000000003e-06 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.882087+0000", 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00071271300000000002 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.229 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.882096+0000", 2024-09-16T14:49:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4550000000000001e-06 2024-09-16T14:49:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889403+0000", 2024-09-16T14:49:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073067749999999997 2024-09-16T14:49:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889432+0000", 2024-09-16T14:49:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9065999999999999e-05 2024-09-16T14:49:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.230 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s2 51/27 ECSubWrite(tid=56, reqid=osd.5.0:55, at_version=0'0, trim_to=0'0, roll_forward_to=51'16))", 2024-09-16T14:49:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.890744+0000", 2024-09-16T14:49:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.2747788169999996, 2024-09-16T14:49:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044012560000000001, 2024-09-16T14:49:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.890744+0000", 2024-09-16T14:49:07.231 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.890744+0000", 2024-09-16T14:49:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.890743+0000", 2024-09-16T14:49:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.232 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.890753+0000", 2024-09-16T14:49:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0793e-05 2024-09-16T14:49:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.890769+0000", 2024-09-16T14:49:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.543e-05 2024-09-16T14:49:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.890779+0000", 2024-09-16T14:49:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6870000000000006e-06 2024-09-16T14:49:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.233 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.890866+0000", 2024-09-16T14:49:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7698e-05 2024-09-16T14:49:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.890876+0000", 2024-09-16T14:49:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9210000000000004e-06 2024-09-16T14:49:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.895126+0000", 2024-09-16T14:49:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0042494789999999996 2024-09-16T14:49:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.234 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.895145+0000", 2024-09-16T14:49:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9133000000000001e-05 2024-09-16T14:49:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 51/27 ECSubWrite(tid=179, reqid=osd.11.0:176, at_version=0'0, trim_to=0'0, roll_forward_to=51'56))", 2024-09-16T14:49:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.901099+0000", 2024-09-16T14:49:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.2644235879999997, 2024-09-16T14:49:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0045652560000000002, 2024-09-16T14:49:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.235 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.901099+0000", 2024-09-16T14:49:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.901099+0000", 2024-09-16T14:49:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.236 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.901098+0000", 2024-09-16T14:49:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.901109+0000", 2024-09-16T14:49:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1181e-05 2024-09-16T14:49:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.901127+0000", 2024-09-16T14:49:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7377999999999999e-05 2024-09-16T14:49:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.237 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.901135+0000", 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5720000000000003e-06 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.901189+0000", 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4271999999999998e-05 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.901199+0000", 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.166e-06 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.238 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.905639+0000", 2024-09-16T14:49:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044401409999999999 2024-09-16T14:49:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.905664+0000", 2024-09-16T14:49:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5299e-05 2024-09-16T14:49:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.239 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 52/27 ECSubWrite(tid=49, reqid=osd.1.0:49, at_version=52'38, trim_to=0'0, roll_forward_to=51'36))", 2024-09-16T14:49:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.881298+0000", 2024-09-16T14:49:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.2842248359999999, 2024-09-16T14:49:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0048856109999999998, 2024-09-16T14:49:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.881298+0000", 2024-09-16T14:49:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.240 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.881298+0000", 2024-09-16T14:49:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.881297+0000", 2024-09-16T14:49:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.881326+0000", 2024-09-16T14:49:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9184e-05 2024-09-16T14:49:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.241 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.881350+0000", 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3924e-05 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.881366+0000", 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.521e-05 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.881615+0000", 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00024962200000000002 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.242 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.881628+0000", 2024-09-16T14:49:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.29e-05 2024-09-16T14:49:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.886154+0000", 2024-09-16T14:49:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0045264210000000001 2024-09-16T14:49:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.886183+0000", 2024-09-16T14:49:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8699000000000001e-05 2024-09-16T14:49:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.243 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 52/27 ECSubWrite(tid=50, reqid=osd.1.0:50, at_version=52'40, trim_to=0'0, roll_forward_to=51'36))", 2024-09-16T14:49:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.2829824930000004, 2024-09-16T14:49:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060730380000000002, 2024-09-16T14:49:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.244 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882539+0000", 2024-09-16T14:49:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.245 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882566+0000", 2024-09-16T14:49:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7101999999999999e-05 2024-09-16T14:49:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882580+0000", 2024-09-16T14:49:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4605e-05 2024-09-16T14:49:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882589+0000", 2024-09-16T14:49:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1020000000000003e-06 2024-09-16T14:49:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.246 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.884101+0000", 2024-09-16T14:49:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001511601 2024-09-16T14:49:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.884112+0000", 2024-09-16T14:49:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1055e-05 2024-09-16T14:49:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888581+0000", 2024-09-16T14:49:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044684240000000004 2024-09-16T14:49:07.247 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888613+0000", 2024-09-16T14:49:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2354000000000002e-05 2024-09-16T14:49:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 52/27 ECSubWrite(tid=181, reqid=osd.11.0:181, at_version=52'58, trim_to=0'0, roll_forward_to=51'56))", 2024-09-16T14:49:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.885957+0000", 2024-09-16T14:49:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.2795656199999996, 2024-09-16T14:49:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0036324360000000002, 2024-09-16T14:49:07.248 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.885957+0000", 2024-09-16T14:49:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.885957+0000", 2024-09-16T14:49:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.249 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.885956+0000", 2024-09-16T14:49:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.885983+0000", 2024-09-16T14:49:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7047999999999999e-05 2024-09-16T14:49:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.886047+0000", 2024-09-16T14:49:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.3330999999999994e-05 2024-09-16T14:49:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.250 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.886057+0000", 2024-09-16T14:49:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0519e-05 2024-09-16T14:49:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.886113+0000", 2024-09-16T14:49:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6174999999999997e-05 2024-09-16T14:49:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.251 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.886124+0000", 2024-09-16T14:49:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1049999999999999e-05 2024-09-16T14:49:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889560+0000", 2024-09-16T14:49:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0034353019999999999 2024-09-16T14:49:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889589+0000", 2024-09-16T14:49:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9450000000000001e-05 2024-09-16T14:49:07.252 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 52/27 ECSubWrite(tid=185, reqid=osd.11.0:185, at_version=52'62, trim_to=0'0, roll_forward_to=52'60))", 2024-09-16T14:49:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.900612+0000", 2024-09-16T14:49:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.2649100959999995, 2024-09-16T14:49:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0039283410000000001, 2024-09-16T14:49:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.253 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.900612+0000", 2024-09-16T14:49:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.900612+0000", 2024-09-16T14:49:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.900611+0000", 2024-09-16T14:49:07.254 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.900639+0000", 2024-09-16T14:49:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7574000000000001e-05 2024-09-16T14:49:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.900656+0000", 2024-09-16T14:49:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6575e-05 2024-09-16T14:49:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.255 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.900665+0000", 2024-09-16T14:49:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0839999999999998e-06 2024-09-16T14:49:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.900736+0000", 2024-09-16T14:49:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1280999999999997e-05 2024-09-16T14:49:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.900748+0000", 2024-09-16T14:49:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1685000000000001e-05 2024-09-16T14:49:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.256 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.904502+0000", 2024-09-16T14:49:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.003754669 2024-09-16T14:49:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.904541+0000", 2024-09-16T14:49:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.8287999999999997e-05 2024-09-16T14:49:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.257 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.258 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.258 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.258 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.258 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:07.259 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:49:07.260 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:07.257+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:07.360 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:07.359+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:07.407 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:07.407 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:07.407 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:07.407 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:07.416 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:49:07.461 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:07.460+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:07.522 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:07.523 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:07.523 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:07.531 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:49:07.562 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:07.560+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:07.639 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:303 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:07.639 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 22.704456439000001, 2024-09-16T14:49:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014763235, 2024-09-16T14:49:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 303 2024-09-16T14:49:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.640 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932467+0000", 2024-09-16T14:49:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932474+0000", 2024-09-16T14:49:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3189999999999997e-06 2024-09-16T14:49:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932490+0000", 2024-09-16T14:49:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:49:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.642 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932498+0000", 2024-09-16T14:49:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7789999999999996e-06 2024-09-16T14:49:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932544+0000", 2024-09-16T14:49:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6486000000000002e-05 2024-09-16T14:49:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932550+0000", 2024-09-16T14:49:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.3149999999999999e-06 2024-09-16T14:49:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.939655+0000", 2024-09-16T14:49:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071050039999999998 2024-09-16T14:49:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.939762+0000", 2024-09-16T14:49:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000106885 2024-09-16T14:49:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.644 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.947232+0000", 2024-09-16T14:49:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074701619999999998 2024-09-16T14:49:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:304 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 22.704418698000001, 2024-09-16T14:49:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015391726, 2024-09-16T14:49:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.645 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 304 2024-09-16T14:49:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.647 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932510+0000", 2024-09-16T14:49:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.1069999999999998e-06 2024-09-16T14:49:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932516+0000", 2024-09-16T14:49:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4699999999999999e-06 2024-09-16T14:49:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932522+0000", 2024-09-16T14:49:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4750000000000001e-06 2024-09-16T14:49:07.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932610+0000", 2024-09-16T14:49:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8623999999999997e-05 2024-09-16T14:49:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932616+0000", 2024-09-16T14:49:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.9649999999999996e-06 2024-09-16T14:49:07.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940205+0000", 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0075882570000000002 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940285+0000", 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0783000000000002e-05 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.947898+0000", 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0076124820000000003 2024-09-16T14:49:07.650 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:305 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 22.704337907999999, 2024-09-16T14:49:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015660833999999998, 2024-09-16T14:49:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.651 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 305 2024-09-16T14:49:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.652 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932586+0000", 2024-09-16T14:49:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932591+0000", 2024-09-16T14:49:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2470000000000002e-06 2024-09-16T14:49:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932602+0000", 2024-09-16T14:49:07.653 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1396000000000001e-05 2024-09-16T14:49:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932608+0000", 2024-09-16T14:49:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.7169999999999996e-06 2024-09-16T14:49:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932646+0000", 2024-09-16T14:49:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.7978999999999997e-05 2024-09-16T14:49:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932652+0000", 2024-09-16T14:49:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0310000000000004e-06 2024-09-16T14:49:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941247+0000", 2024-09-16T14:49:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085954180000000005 2024-09-16T14:49:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941327+0000", 2024-09-16T14:49:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9354e-05 2024-09-16T14:49:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.948248+0000", 2024-09-16T14:49:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0069212400000000004 2024-09-16T14:49:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:362 3.2s0 3:4ee78709:::smithi04040151-40:head [copy-from ver 6] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-16T14:49:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:47.938030+0000", 2024-09-16T14:49:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.698894948, 2024-09-16T14:49:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0092047480000000004, 2024-09-16T14:49:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 362 2024-09-16T14:49:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938030+0000", 2024-09-16T14:49:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938030+0000", 2024-09-16T14:49:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938025+0000", 2024-09-16T14:49:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999952 2024-09-16T14:49:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938040+0000", 2024-09-16T14:49:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4786e-05 2024-09-16T14:49:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938062+0000", 2024-09-16T14:49:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1725e-05 2024-09-16T14:49:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938072+0000", 2024-09-16T14:49:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0516000000000001e-05 2024-09-16T14:49:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938126+0000", 2024-09-16T14:49:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.3168000000000003e-05 2024-09-16T14:49:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.938131+0000", 2024-09-16T14:49:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5350000000000004e-06 2024-09-16T14:49:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.939892+0000", 2024-09-16T14:49:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001761295 2024-09-16T14:49:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.939971+0000", 2024-09-16T14:49:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8417999999999994e-05 2024-09-16T14:49:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.944590+0000", 2024-09-16T14:49:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0046191219999999998 2024-09-16T14:49:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.945941+0000", 2024-09-16T14:49:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0013506029999999999 2024-09-16T14:49:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.947193+0000", 2024-09-16T14:49:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001252555 2024-09-16T14:49:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:47.947235+0000", 2024-09-16T14:49:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.1569999999999997e-05 2024-09-16T14:49:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:375 3.7s0 3:e91f0d6c:::smithi04040151-31:head [append 0~0] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.960177+0000", 2024-09-16T14:49:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 18.676748052000001, 2024-09-16T14:49:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.009691926, 2024-09-16T14:49:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 375 2024-09-16T14:49:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960177+0000", 2024-09-16T14:49:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960177+0000", 2024-09-16T14:49:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960176+0000", 2024-09-16T14:49:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960180+0000", 2024-09-16T14:49:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.4640000000000002e-06 2024-09-16T14:49:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960524+0000", 2024-09-16T14:49:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034462499999999999 2024-09-16T14:49:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960535+0000", 2024-09-16T14:49:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0971e-05 2024-09-16T14:49:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.963989+0000", 2024-09-16T14:49:07.667 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0034539449999999999 2024-09-16T14:49:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.964073+0000", 2024-09-16T14:49:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.3725000000000002e-05 2024-09-16T14:49:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.964387+0000", 2024-09-16T14:49:07.668 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00031368200000000002 2024-09-16T14:49:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.969143+0000", 2024-09-16T14:49:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0047564260000000002 2024-09-16T14:49:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.969786+0000", 2024-09-16T14:49:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00064274600000000003 2024-09-16T14:49:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.669 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.969869+0000", 2024-09-16T14:49:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2804000000000002e-05 2024-09-16T14:49:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:403 3.2s0 3:421e4b92:::smithi04040151-11:head [copy-from ver 2] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 13.753360011, 2024-09-16T14:49:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015904270000000002, 2024-09-16T14:49:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:07.670 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 403 2024-09-16T14:49:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:07.671 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883563+0000", 2024-09-16T14:49:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.672 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883573+0000", 2024-09-16T14:49:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0583e-05 2024-09-16T14:49:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883593+0000", 2024-09-16T14:49:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0052e-05 2024-09-16T14:49:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883603+0000", 2024-09-16T14:49:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4409999999999999e-06 2024-09-16T14:49:07.673 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883667+0000", 2024-09-16T14:49:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4671000000000005e-05 2024-09-16T14:49:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883775+0000", 2024-09-16T14:49:07.674 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000108068 2024-09-16T14:49:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.896809+0000", 2024-09-16T14:49:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01303355 2024-09-16T14:49:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897717+0000", 2024-09-16T14:49:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00090801599999999999 2024-09-16T14:49:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.675 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.899408+0000", 2024-09-16T14:49:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0016911459999999999 2024-09-16T14:49:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.899469+0000", 2024-09-16T14:49:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0968999999999999e-05 2024-09-16T14:49:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:404 3.2s0 3:421e4b92:::smithi04040151-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 13.753306714000001, 2024-09-16T14:49:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0139831, 2024-09-16T14:49:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 404 2024-09-16T14:49:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.677 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883617+0000", 2024-09-16T14:49:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.678 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883622+0000", 2024-09-16T14:49:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7489999999999998e-06 2024-09-16T14:49:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883631+0000", 2024-09-16T14:49:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6489999999999997e-06 2024-09-16T14:49:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883639+0000", 2024-09-16T14:49:07.679 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2120000000000003e-06 2024-09-16T14:49:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883930+0000", 2024-09-16T14:49:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029138200000000002 2024-09-16T14:49:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883961+0000", 2024-09-16T14:49:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0190999999999999e-05 2024-09-16T14:49:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.680 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897411+0000", 2024-09-16T14:49:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013450037999999999 2024-09-16T14:49:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897506+0000", 2024-09-16T14:49:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5785000000000005e-05 2024-09-16T14:49:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897601+0000", 2024-09-16T14:49:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4868e-05 2024-09-16T14:49:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.681 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:460 3.2s0 3:421e4b92:::smithi04040151-11:head [append 0~0] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.993559+0000", 2024-09-16T14:49:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.6433654030000007, 2024-09-16T14:49:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0095830099999999994, 2024-09-16T14:49:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 460 2024-09-16T14:49:07.682 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993559+0000", 2024-09-16T14:49:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993559+0000", 2024-09-16T14:49:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.683 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993557+0000", 2024-09-16T14:49:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993569+0000", 2024-09-16T14:49:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1644999999999999e-05 2024-09-16T14:49:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993588+0000", 2024-09-16T14:49:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8793000000000001e-05 2024-09-16T14:49:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.684 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993597+0000", 2024-09-16T14:49:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3710000000000006e-06 2024-09-16T14:49:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993650+0000", 2024-09-16T14:49:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.3001999999999998e-05 2024-09-16T14:49:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993768+0000", 2024-09-16T14:49:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000117947 2024-09-16T14:49:07.685 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994233+0000", 2024-09-16T14:49:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00046465800000000001 2024-09-16T14:49:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997829+0000", 2024-09-16T14:49:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0035960779999999999 2024-09-16T14:49:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003062+0000", 2024-09-16T14:49:07.686 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0052332100000000003 2024-09-16T14:49:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003142+0000", 2024-09-16T14:49:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0585000000000002e-05 2024-09-16T14:49:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:461 3.2s0 3:421e4b92:::smithi04040151-11:head [rollback 10] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:07.687 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.6433166710000009, 2024-09-16T14:49:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011419016000000001, 2024-09-16T14:49:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 461 2024-09-16T14:49:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.688 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993611+0000", 2024-09-16T14:49:07.689 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6440000000000002e-06 2024-09-16T14:49:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993619+0000", 2024-09-16T14:49:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.199e-06 2024-09-16T14:49:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993624+0000", 2024-09-16T14:49:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.8790000000000002e-06 2024-09-16T14:49:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.690 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994734+0000", 2024-09-16T14:49:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0011103339999999999 2024-09-16T14:49:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994830+0000", 2024-09-16T14:49:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6125000000000005e-05 2024-09-16T14:49:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995335+0000", 2024-09-16T14:49:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00050461799999999997 2024-09-16T14:49:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.691 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.001584+0000", 2024-09-16T14:49:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0062489609999999999 2024-09-16T14:49:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004966+0000", 2024-09-16T14:49:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.003382674 2024-09-16T14:49:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.005027+0000", 2024-09-16T14:49:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0974000000000003e-05 2024-09-16T14:49:07.692 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:462 3.bs0 3:d9d8585e:::smithi04040151-8:head [append 0~0] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.993639+0000", 2024-09-16T14:49:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.6432858350000004, 2024-09-16T14:49:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010145506, 2024-09-16T14:49:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.693 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 462 2024-09-16T14:49:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993639+0000", 2024-09-16T14:49:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.694 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993639+0000", 2024-09-16T14:49:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993639+0000", 2024-09-16T14:49:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993641+0000", 2024-09-16T14:49:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4789999999999999e-06 2024-09-16T14:49:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.695 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993651+0000", 2024-09-16T14:49:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4059999999999994e-06 2024-09-16T14:49:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993655+0000", 2024-09-16T14:49:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.5369999999999999e-06 2024-09-16T14:49:07.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993720+0000", 2024-09-16T14:49:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5048000000000005e-05 2024-09-16T14:49:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993816+0000", 2024-09-16T14:49:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5568000000000004e-05 2024-09-16T14:49:07.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994512+0000", 2024-09-16T14:49:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00069592099999999995 2024-09-16T14:49:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.000585+0000", 2024-09-16T14:49:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060736160000000004 2024-09-16T14:49:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003712+0000", 2024-09-16T14:49:07.698 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0031268239999999998 2024-09-16T14:49:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003785+0000", 2024-09-16T14:49:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2495000000000003e-05 2024-09-16T14:49:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:463 3.bs0 3:d9d8585e:::smithi04040151-8:head [rollback 10] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.993666+0000", 2024-09-16T14:49:07.699 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.6432586810000007, 2024-09-16T14:49:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010553527, 2024-09-16T14:49:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 463 2024-09-16T14:49:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993666+0000", 2024-09-16T14:49:07.700 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993666+0000", 2024-09-16T14:49:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993666+0000", 2024-09-16T14:49:07.701 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993668+0000", 2024-09-16T14:49:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5629999999999999e-06 2024-09-16T14:49:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993675+0000", 2024-09-16T14:49:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.139e-06 2024-09-16T14:49:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.702 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993688+0000", 2024-09-16T14:49:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3237e-05 2024-09-16T14:49:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995227+0000", 2024-09-16T14:49:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0015395949999999999 2024-09-16T14:49:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995317+0000", 2024-09-16T14:49:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0026999999999997e-05 2024-09-16T14:49:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995832+0000", 2024-09-16T14:49:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00051462399999999998 2024-09-16T14:49:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002431+0000", 2024-09-16T14:49:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0065994540000000003 2024-09-16T14:49:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004155+0000", 2024-09-16T14:49:07.704 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001723961 2024-09-16T14:49:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004220+0000", 2024-09-16T14:49:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4319999999999994e-05 2024-09-16T14:49:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:464 3.bs0 3:d9d8585e:::smithi04040151-8:head [append 0~0] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.993720+0000", 2024-09-16T14:49:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.6432048530000003, 2024-09-16T14:49:07.705 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010696298, 2024-09-16T14:49:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 464 2024-09-16T14:49:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993720+0000", 2024-09-16T14:49:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.706 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993720+0000", 2024-09-16T14:49:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993720+0000", 2024-09-16T14:49:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993722+0000", 2024-09-16T14:49:07.707 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9359999999999999e-06 2024-09-16T14:49:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993730+0000", 2024-09-16T14:49:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3669999999999999e-06 2024-09-16T14:49:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993737+0000", 2024-09-16T14:49:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.985e-06 2024-09-16T14:49:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.708 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997488+0000", 2024-09-16T14:49:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0037506879999999999 2024-09-16T14:49:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997578+0000", 2024-09-16T14:49:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0210000000000005e-05 2024-09-16T14:49:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.998012+0000", 2024-09-16T14:49:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000434201 2024-09-16T14:49:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.709 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002434+0000", 2024-09-16T14:49:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044225080000000003 2024-09-16T14:49:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004356+0000", 2024-09-16T14:49:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0019215720000000001 2024-09-16T14:49:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004416+0000", 2024-09-16T14:49:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0316999999999999e-05 2024-09-16T14:49:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.710 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:465 3.7s0 3:e91f0d6c:::smithi04040151-31:head [create,setxattr bk]W^gTaniZ\\TV]dqj_UT\\QgY_\\pYYUOcYdSe`m_ (1859) in=40b,setxattr XSPcap`qmQ\\ZXYl^_bQYNltYZfOtfbRjqmVk_mOa (419) in=40b,setxattr l_Qd^aXhnapg`O][bWecsVdjlWkdYc^tOnjZh]^m (1577) in=40b,setxattr facegtqpaQroppsVbsrZelUak]RbamP`srOTdms[ (350) in=40b,setxattr QllprTts[liem]moe]Pi`nNtcQko (1402) in=28b,setxattr sjm^Un[ahPriW\\TNkaYS\\NnPZfZ\\Zmq[rQ^TWYZm (1892) in=40b,setxattr kRdb__OpRdYV^sYmYmYdhes^Vbf]QaV`SVZkhRcY (209) in=40b,setxattr b[QTnoQc[_Yc^RmdY[ (1402) in=18b,setxattr ^iZkhfc`oO (1402) in=10b,setxattr d\\QUOkoScOhonabPS_YeNlheOOeOal]aQ^ZW]lc` (2805) in=40b,setxattr SioYcliigTgheX\\jnVqipcelfYhbZRjQRet]mQQU (1365) in=40b,setxattr jWcgaRWbr\\ZQ_XXr_eenPa]sXtbaPlmeshkgjOsm (1192) in=40b,setxattr WmVtOY (1402) in=6b,setxattr ]tk_t\\gnSfccZfYfnWnaqP (1402) in=22b,setxattr dS_dYomnsYPtQarmqoZQkO]Ofk_SfgjgitphTZPg (18524) in=40b,setxattr g_a^Zpt`YVahZiOW]V[jQ\\QccqSTo_ma[\\c\\[aft (454) in=40b,setxattr qmmdmgOR__hN\\UTjpR_gbY^s_pOVSdbTob`k (1402) in=36b,setxattr cW`PPOYrbfXV^hmaWhVNb`qVSXqjcdej`pY][]l^ (665) in=40b,setxattr jjo^VgcaQffrnVNPlnq_iTmcrlnTOenhaZV[_SZf (453) in=40b,setxattr n^nqj]bnVcl\\njtUkprb\\laSh^Rbmam_Z_NkeXqj (595) in=40b] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.995572+0000", 2024-09-16T14:49:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.6413525470000003, 2024-09-16T14:49:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0092652180000000004, 2024-09-16T14:49:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.711 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 465 2024-09-16T14:49:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995572+0000", 2024-09-16T14:49:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995572+0000", 2024-09-16T14:49:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.712 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995571+0000", 2024-09-16T14:49:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995588+0000", 2024-09-16T14:49:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7146000000000002e-05 2024-09-16T14:49:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995599+0000", 2024-09-16T14:49:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0489000000000001e-05 2024-09-16T14:49:07.713 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995605+0000", 2024-09-16T14:49:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6189999999999996e-06 2024-09-16T14:49:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995654+0000", 2024-09-16T14:49:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.8857999999999999e-05 2024-09-16T14:49:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995779+0000", 2024-09-16T14:49:07.714 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012446000000000001 2024-09-16T14:49:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.996775+0000", 2024-09-16T14:49:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00099642999999999997 2024-09-16T14:49:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002430+0000", 2024-09-16T14:49:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056549249999999999 2024-09-16T14:49:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.715 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004762+0000", 2024-09-16T14:49:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0023316589999999998 2024-09-16T14:49:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004838+0000", 2024-09-16T14:49:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5910999999999999e-05 2024-09-16T14:49:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:469 3.2s0 3:4ee78709:::smithi04040151-40:head [read 0~1474560,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:07.716 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.995831+0000", 2024-09-16T14:49:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.6410933980000006, 2024-09-16T14:49:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090063799999999996, 2024-09-16T14:49:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 469 2024-09-16T14:49:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995831+0000", 2024-09-16T14:49:07.717 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995831+0000", 2024-09-16T14:49:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995831+0000", 2024-09-16T14:49:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.718 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995836+0000", 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9439999999999996e-06 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995845+0000", 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5189999999999998e-06 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995852+0000", 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6370000000000002e-06 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.719 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995970+0000", 2024-09-16T14:49:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011843100000000001 2024-09-16T14:49:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.996056+0000", 2024-09-16T14:49:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5445999999999995e-05 2024-09-16T14:49:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004838+0000", 2024-09-16T14:49:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0087821040000000006 2024-09-16T14:49:07.720 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:470 3.2s0 3:4ee78709:::smithi04040151-40:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.6410622070000009, 2024-09-16T14:49:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011056359999999999, 2024-09-16T14:49:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.721 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 470 2024-09-16T14:49:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.722 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995862+0000", 2024-09-16T14:49:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995865+0000", 2024-09-16T14:49:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2440000000000002e-06 2024-09-16T14:49:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.723 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995872+0000", 2024-09-16T14:49:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5640000000000002e-06 2024-09-16T14:49:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995877+0000", 2024-09-16T14:49:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1680000000000003e-06 2024-09-16T14:49:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.996768+0000", 2024-09-16T14:49:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00089129699999999999 2024-09-16T14:49:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.724 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.996867+0000", 2024-09-16T14:49:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8259999999999998e-05 2024-09-16T14:49:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.006919+0000", 2024-09-16T14:49:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010052284 2024-09-16T14:49:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.725 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:471 3.2s0 3:4ee78709:::smithi04040151-40:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.6410434810000005, 2024-09-16T14:49:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012013398999999999, 2024-09-16T14:49:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 471 2024-09-16T14:49:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.726 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.727 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995884+0000", 2024-09-16T14:49:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:49:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995891+0000", 2024-09-16T14:49:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.72e-06 2024-09-16T14:49:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995896+0000", 2024-09-16T14:49:07.728 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1889999999999998e-06 2024-09-16T14:49:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997239+0000", 2024-09-16T14:49:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0013430460000000001 2024-09-16T14:49:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997352+0000", 2024-09-16T14:49:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011287500000000001 2024-09-16T14:49:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.729 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007895+0000", 2024-09-16T14:49:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010543113999999999 2024-09-16T14:49:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s4 52/27 ECSubWrite(tid=183, reqid=osd.11.0:183, at_version=52'60, trim_to=0'0, roll_forward_to=51'56))", 2024-09-16T14:49:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.7496546100000003, 2024-09-16T14:49:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011901715, 2024-09-16T14:49:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.730 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:07.731 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887292+0000", 2024-09-16T14:49:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1920999999999999e-05 2024-09-16T14:49:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887303+0000", 2024-09-16T14:49:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1472e-05 2024-09-16T14:49:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.732 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887311+0000", 2024-09-16T14:49:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887360+0000", 2024-09-16T14:49:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9002999999999998e-05 2024-09-16T14:49:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887369+0000", 2024-09-16T14:49:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9299999999999992e-06 2024-09-16T14:49:07.733 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.899136+0000", 2024-09-16T14:49:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011767123000000001 2024-09-16T14:49:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.899172+0000", 2024-09-16T14:49:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6161000000000002e-05 2024-09-16T14:49:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.734 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es6 52/27 ECSubWrite(tid=182, reqid=osd.11.0:182, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887409+0000", 2024-09-16T14:49:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.7495159230000006, 2024-09-16T14:49:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010651061999999999, 2024-09-16T14:49:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887409+0000", 2024-09-16T14:49:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.735 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887409+0000", 2024-09-16T14:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887408+0000", 2024-09-16T14:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887448+0000", 2024-09-16T14:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.998e-05 2024-09-16T14:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887460+0000", 2024-09-16T14:49:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1931e-05 2024-09-16T14:49:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887468+0000", 2024-09-16T14:49:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1240000000000005e-06 2024-09-16T14:49:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.737 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887710+0000", 2024-09-16T14:49:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000241272 2024-09-16T14:49:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887722+0000", 2024-09-16T14:49:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2515e-05 2024-09-16T14:49:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.738 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.739 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.898028+0000", 2024-09-16T14:49:07.739 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010305365 2024-09-16T14:49:07.740 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.740 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.740 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.740 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.898060+0000", 2024-09-16T14:49:07.740 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2391999999999998e-05 2024-09-16T14:49:07.740 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds5 52/27 ECSubWrite(tid=214, reqid=osd.7.0:214, at_version=52'35, trim_to=0'0, roll_forward_to=51'31))", 2024-09-16T14:49:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.7488754459999996, 2024-09-16T14:49:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010846751, 2024-09-16T14:49:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.741 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.742 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888072+0000", 2024-09-16T14:49:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3008000000000001e-05 2024-09-16T14:49:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888082+0000", 2024-09-16T14:49:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0129000000000001e-05 2024-09-16T14:49:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888090+0000", 2024-09-16T14:49:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7510000000000008e-06 2024-09-16T14:49:07.743 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889906+0000", 2024-09-16T14:49:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001816603 2024-09-16T14:49:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889931+0000", 2024-09-16T14:49:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4165e-05 2024-09-16T14:49:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.744 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.898870+0000", 2024-09-16T14:49:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089394299999999999 2024-09-16T14:49:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.898896+0000", 2024-09-16T14:49:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6188999999999999e-05 2024-09-16T14:49:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s3 52/27 ECSubWrite(tid=29, reqid=osd.4.0:29, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:07.745 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.889955+0000", 2024-09-16T14:49:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.7469694810000007, 2024-09-16T14:49:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0095517620000000001, 2024-09-16T14:49:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889955+0000", 2024-09-16T14:49:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889955+0000", 2024-09-16T14:49:07.746 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889955+0000", 2024-09-16T14:49:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889980+0000", 2024-09-16T14:49:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4765000000000001e-05 2024-09-16T14:49:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:07.747 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.891022+0000", 2024-09-16T14:49:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0010423800000000001 2024-09-16T14:49:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.891037+0000", 2024-09-16T14:49:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4501000000000001e-05 2024-09-16T14:49:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.891261+0000", 2024-09-16T14:49:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00022435599999999999 2024-09-16T14:49:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.891270+0000", 2024-09-16T14:49:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8780000000000004e-06 2024-09-16T14:49:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.899486+0000", 2024-09-16T14:49:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0082159580000000006 2024-09-16T14:49:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.899507+0000", 2024-09-16T14:49:07.749 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1311e-05 2024-09-16T14:49:07.750 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.750 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.750 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.750 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:07.750 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:07.750 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:07.751 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:49:07.752 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:07.750+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:07.852 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:07.850+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:07.866 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:07.866 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:07.867 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:07.867 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:07.876 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:49:07.924 INFO:tasks.rados.rados.0.smithi040.stdout:225: done (4 left) 2024-09-16T14:49:07.925 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 25 current snap is 17 2024-09-16T14:49:07.925 INFO:tasks.rados.rados.0.smithi040.stdout:226: seq_num 92 ranges {0=786432,786432=491520,1277952=278528} 2024-09-16T14:49:07.925 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is set 2024-09-16T14:49:07.930 INFO:tasks.rados.rados.0.smithi040.stdout:226: writing smithi04040151-25 from 0 to 786432 tid 1 2024-09-16T14:49:07.933 INFO:tasks.rados.rados.0.smithi040.stdout:226: writing smithi04040151-25 from 786432 to 1277952 tid 2 2024-09-16T14:49:07.935 INFO:tasks.rados.rados.0.smithi040.stdout:226: writing smithi04040151-25 from 1277952 to 1556480 tid 3 2024-09-16T14:49:07.936 INFO:tasks.rados.rados.0.smithi040.stdout:227: snap_remove snap 15 2024-09-16T14:49:07.959 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:07.958+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:08.032 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:08.032 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:08.032 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:08.032 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:08.040 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:49:08.082 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:08.080+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:08.156 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:08.155+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:08.185 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:08.186 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:08.186 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:08.186 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:08.186 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.186 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s3 37/27 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=37'32, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:08.186 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.920644+0000", 2024-09-16T14:49:08.186 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.26223792, 2024-09-16T14:49:08.187 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0056995509999999997, 2024-09-16T14:49:08.187 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.187 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.187 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.187 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.187 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.188 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920644+0000", 2024-09-16T14:49:08.188 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.188 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.188 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.188 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.188 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920644+0000", 2024-09-16T14:49:08.188 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.189 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.189 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.189 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.189 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920644+0000", 2024-09-16T14:49:08.189 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:08.189 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.190 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.190 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.190 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920668+0000", 2024-09-16T14:49:08.190 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3832000000000002e-05 2024-09-16T14:49:08.190 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.190 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.191 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.191 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920680+0000", 2024-09-16T14:49:08.191 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2021999999999999e-05 2024-09-16T14:49:08.192 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.192 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.192 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.192 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920689+0000", 2024-09-16T14:49:08.192 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2760000000000006e-06 2024-09-16T14:49:08.192 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.192 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.193 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.193 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920734+0000", 2024-09-16T14:49:08.193 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4943e-05 2024-09-16T14:49:08.193 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.193 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.194 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.194 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920744+0000", 2024-09-16T14:49:08.194 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9480000000000003e-06 2024-09-16T14:49:08.194 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.194 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.194 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.195 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.926318+0000", 2024-09-16T14:49:08.195 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0055746700000000003 2024-09-16T14:49:08.195 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.195 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.195 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.195 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.926344+0000", 2024-09-16T14:49:08.196 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5072e-05 2024-09-16T14:49:08.196 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.196 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.196 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.196 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.196 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.196 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs2 37/27 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:08.197 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:08.197 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.262178443, 2024-09-16T14:49:08.197 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010231874, 2024-09-16T14:49:08.197 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.197 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.197 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.197 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.197 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.198 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:08.198 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.198 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.198 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.198 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.198 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:08.198 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.198 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.198 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.199 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.199 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:08.199 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.199 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.199 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.199 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.199 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920725+0000", 2024-09-16T14:49:08.199 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2198999999999999e-05 2024-09-16T14:49:08.200 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.200 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.200 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.200 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920735+0000", 2024-09-16T14:49:08.200 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0122000000000001e-05 2024-09-16T14:49:08.200 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.201 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.201 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.201 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920742+0000", 2024-09-16T14:49:08.201 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7179999999999999e-06 2024-09-16T14:49:08.201 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.201 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.202 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.202 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921079+0000", 2024-09-16T14:49:08.202 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00033676000000000002 2024-09-16T14:49:08.202 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.202 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.202 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.202 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921088+0000", 2024-09-16T14:49:08.202 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9290000000000004e-06 2024-09-16T14:49:08.202 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.202 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.202 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930916+0000", 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0098279749999999992 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930935+0000", 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9510000000000001e-05 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.fs7 37/27 ECSubWrite(tid=189, reqid=osd.7.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'12))", 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:08.203 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.261990743999998, 2024-09-16T14:49:08.204 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010007947999999999, 2024-09-16T14:49:08.204 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.204 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.204 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.204 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.204 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.204 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:08.204 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.204 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.204 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.204 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.204 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:08.204 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.205 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.205 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.205 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.205 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920890+0000", 2024-09-16T14:49:08.205 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:08.205 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.205 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.205 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.205 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920900+0000", 2024-09-16T14:49:08.205 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9839999999999996e-06 2024-09-16T14:49:08.205 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.205 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.205 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920910+0000", 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4220000000000006e-06 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920917+0000", 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5349999999999999e-06 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920967+0000", 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9348000000000003e-05 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.206 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.207 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920976+0000", 2024-09-16T14:49:08.207 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8810000000000002e-06 2024-09-16T14:49:08.207 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.207 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.207 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.207 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930871+0000", 2024-09-16T14:49:08.207 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0098950500000000007 2024-09-16T14:49:08.207 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.207 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.207 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.207 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930899+0000", 2024-09-16T14:49:08.207 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.847e-05 2024-09-16T14:49:08.207 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.208 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.208 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.208 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.208 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.208 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s3 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:08.208 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:08.208 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.260993702, 2024-09-16T14:49:08.208 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011669621999999999, 2024-09-16T14:49:08.208 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.208 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.208 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.208 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.208 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.209 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:08.209 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.209 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.209 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.209 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.209 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:08.209 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.209 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.209 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.209 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.209 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921887+0000", 2024-09-16T14:49:08.209 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:08.209 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.210 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.210 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.210 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921911+0000", 2024-09-16T14:49:08.210 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4012999999999999e-05 2024-09-16T14:49:08.210 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.210 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.210 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.210 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921923+0000", 2024-09-16T14:49:08.211 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1642e-05 2024-09-16T14:49:08.211 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.211 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.211 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.211 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921932+0000", 2024-09-16T14:49:08.211 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0210000000000005e-06 2024-09-16T14:49:08.211 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.211 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.211 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.211 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921988+0000", 2024-09-16T14:49:08.211 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5822999999999997e-05 2024-09-16T14:49:08.211 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.212 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.212 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.212 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921997+0000", 2024-09-16T14:49:08.212 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7900000000000005e-06 2024-09-16T14:49:08.212 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.212 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.212 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.213 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933539+0000", 2024-09-16T14:49:08.213 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011542248 2024-09-16T14:49:08.213 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.213 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.213 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.213 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933558+0000", 2024-09-16T14:49:08.213 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8964999999999999e-05 2024-09-16T14:49:08.213 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.213 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.214 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.214 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.214 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.214 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds6 37/27 ECSubWrite(tid=190, reqid=osd.7.0:188, at_version=0'0, trim_to=0'0, roll_forward_to=37'20))", 2024-09-16T14:49:08.214 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:08.214 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.260912101999999, 2024-09-16T14:49:08.214 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0090799120000000007, 2024-09-16T14:49:08.214 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.214 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.214 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.214 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.214 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.214 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:08.214 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.215 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.215 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.215 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.215 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:08.215 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.215 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.215 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.215 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.215 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921969+0000", 2024-09-16T14:49:08.215 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:08.215 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.215 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.215 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.215 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921983+0000", 2024-09-16T14:49:08.216 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4403000000000001e-05 2024-09-16T14:49:08.216 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.216 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.216 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.216 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921996+0000", 2024-09-16T14:49:08.216 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.259e-05 2024-09-16T14:49:08.216 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.216 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.216 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.216 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922005+0000", 2024-09-16T14:49:08.216 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9080000000000001e-06 2024-09-16T14:49:08.216 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.216 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.217 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.217 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922073+0000", 2024-09-16T14:49:08.217 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8127999999999996e-05 2024-09-16T14:49:08.217 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.217 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.217 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.217 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922085+0000", 2024-09-16T14:49:08.217 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1712999999999999e-05 2024-09-16T14:49:08.217 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.217 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.217 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.217 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931031+0000", 2024-09-16T14:49:08.217 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089464669999999996 2024-09-16T14:49:08.218 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.218 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.218 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.218 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931050+0000", 2024-09-16T14:49:08.218 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8471999999999999e-05 2024-09-16T14:49:08.218 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.218 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.218 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.218 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.218 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.218 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.6s2 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:49:08.218 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.923513+0000", 2024-09-16T14:49:08.218 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.259369147000001, 2024-09-16T14:49:08.218 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0075220499999999997, 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923513+0000", 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923513+0000", 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.219 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.220 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923512+0000", 2024-09-16T14:49:08.220 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:08.220 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.220 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.220 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.220 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923534+0000", 2024-09-16T14:49:08.220 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1892999999999999e-05 2024-09-16T14:49:08.220 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.220 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.220 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.221 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923551+0000", 2024-09-16T14:49:08.221 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7127999999999999e-05 2024-09-16T14:49:08.221 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.221 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.221 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.221 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923559+0000", 2024-09-16T14:49:08.221 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6400000000000003e-06 2024-09-16T14:49:08.221 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.221 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.221 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.221 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923615+0000", 2024-09-16T14:49:08.221 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5560000000000003e-05 2024-09-16T14:49:08.221 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.221 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.222 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.222 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923627+0000", 2024-09-16T14:49:08.222 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2296999999999999e-05 2024-09-16T14:49:08.222 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.222 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.222 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.222 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931010+0000", 2024-09-16T14:49:08.222 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.007382258 2024-09-16T14:49:08.222 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.222 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.222 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.222 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931035+0000", 2024-09-16T14:49:08.222 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5194e-05 2024-09-16T14:49:08.223 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.223 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.223 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.223 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.223 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.223 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as1 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:08.223 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:08.223 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.258816852999999, 2024-09-16T14:49:08.223 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0095052339999999996, 2024-09-16T14:49:08.223 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.223 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.224 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.224 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.224 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.224 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:08.224 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.225 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.225 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.225 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.225 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:08.225 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.225 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.225 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.226 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.226 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:08.226 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:08.226 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.226 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.227 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.227 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924089+0000", 2024-09-16T14:49:08.227 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3768999999999999e-05 2024-09-16T14:49:08.227 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.227 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.227 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.227 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924101+0000", 2024-09-16T14:49:08.227 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2294e-05 2024-09-16T14:49:08.227 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.227 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.227 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.227 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924109+0000", 2024-09-16T14:49:08.228 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7389999999999999e-06 2024-09-16T14:49:08.228 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.228 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.228 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.228 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924163+0000", 2024-09-16T14:49:08.228 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.4052999999999999e-05 2024-09-16T14:49:08.228 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.228 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.228 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.228 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924172+0000", 2024-09-16T14:49:08.228 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7189999999999996e-06 2024-09-16T14:49:08.229 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.229 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.229 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.229 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933561+0000", 2024-09-16T14:49:08.229 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0093884659999999998 2024-09-16T14:49:08.229 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.229 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.229 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.229 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933570+0000", 2024-09-16T14:49:08.229 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4120000000000007e-06 2024-09-16T14:49:08.229 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.230 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.230 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.230 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.230 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.230 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.es3 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:08.230 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:08.230 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.258759418, 2024-09-16T14:49:08.230 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0094028089999999998, 2024-09-16T14:49:08.230 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.230 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.231 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.231 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.231 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.231 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:08.231 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.231 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.231 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.231 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.231 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:08.231 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.231 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.231 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924142+0000", 2024-09-16T14:49:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0162000000000001e-05 2024-09-16T14:49:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924150+0000", 2024-09-16T14:49:08.232 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0919999999999998e-06 2024-09-16T14:49:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924157+0000", 2024-09-16T14:49:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6069999999999996e-06 2024-09-16T14:49:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924215+0000", 2024-09-16T14:49:08.233 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8192000000000001e-05 2024-09-16T14:49:08.234 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.234 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.234 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.234 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924225+0000", 2024-09-16T14:49:08.234 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9760000000000007e-06 2024-09-16T14:49:08.234 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.234 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.234 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.234 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933494+0000", 2024-09-16T14:49:08.234 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0092692720000000003 2024-09-16T14:49:08.234 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.234 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.234 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.235 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933525+0000", 2024-09-16T14:49:08.235 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0976999999999998e-05 2024-09-16T14:49:08.235 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.235 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.235 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.235 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:384 3.3s0 3:c9d06351:::smithi04040151-33:head [delete] snapc 12=[12,10,c] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:53.856763+0000", 2024-09-16T14:49:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 14.326118523, 2024-09-16T14:49:08.236 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0073332120000000004, 2024-09-16T14:49:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 384 2024-09-16T14:49:08.237 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.238 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.238 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.238 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.238 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856763+0000", 2024-09-16T14:49:08.238 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.238 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.239 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.239 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.239 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856763+0000", 2024-09-16T14:49:08.239 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.239 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.239 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.239 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.240 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856762+0000", 2024-09-16T14:49:08.240 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:08.240 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.240 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.240 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.240 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856770+0000", 2024-09-16T14:49:08.241 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4400000000000005e-06 2024-09-16T14:49:08.241 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.241 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.241 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.241 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856786+0000", 2024-09-16T14:49:08.241 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5608e-05 2024-09-16T14:49:08.241 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.242 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.242 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.242 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856794+0000", 2024-09-16T14:49:08.242 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9910000000000002e-06 2024-09-16T14:49:08.242 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.242 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.242 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.242 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856845+0000", 2024-09-16T14:49:08.243 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0862999999999997e-05 2024-09-16T14:49:08.243 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.243 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.243 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:08.243 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.856855+0000", 2024-09-16T14:49:08.243 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9539999999999999e-06 2024-09-16T14:49:08.243 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.244 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.244 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.244 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.858580+0000", 2024-09-16T14:49:08.244 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001725096 2024-09-16T14:49:08.244 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.244 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.245 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:08.245 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.858688+0000", 2024-09-16T14:49:08.245 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000108879 2024-09-16T14:49:08.245 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.245 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.245 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.246 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.859596+0000", 2024-09-16T14:49:08.246 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00090781499999999997 2024-09-16T14:49:08.246 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.246 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.246 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.246 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.860493+0000", 2024-09-16T14:49:08.247 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00089657300000000001 2024-09-16T14:49:08.247 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.247 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.247 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:08.247 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864053+0000", 2024-09-16T14:49:08.247 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0035596339999999999 2024-09-16T14:49:08.247 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.248 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.248 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.248 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864097+0000", 2024-09-16T14:49:08.248 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4058999999999999e-05 2024-09-16T14:49:08.248 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.248 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.248 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.248 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.249 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.249 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s3 51/27 ECSubWrite(tid=169, reqid=osd.11.0:169, at_version=51'52, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:08.249 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.880653+0000", 2024-09-16T14:49:08.249 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.302229219999999, 2024-09-16T14:49:08.249 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005579428, 2024-09-16T14:49:08.249 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.249 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.249 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.250 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.250 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.250 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880653+0000", 2024-09-16T14:49:08.250 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.250 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.250 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.250 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.251 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880653+0000", 2024-09-16T14:49:08.251 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.251 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.251 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.251 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.251 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880652+0000", 2024-09-16T14:49:08.252 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:08.252 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.252 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.252 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.252 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880672+0000", 2024-09-16T14:49:08.252 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9930999999999999e-05 2024-09-16T14:49:08.252 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.252 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.252 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.253 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880684+0000", 2024-09-16T14:49:08.253 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1871999999999999e-05 2024-09-16T14:49:08.253 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.253 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.253 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.253 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880691+0000", 2024-09-16T14:49:08.253 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4780000000000002e-06 2024-09-16T14:49:08.253 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.253 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.253 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.254 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880739+0000", 2024-09-16T14:49:08.254 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8136999999999997e-05 2024-09-16T14:49:08.254 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.254 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.254 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.254 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.880749+0000", 2024-09-16T14:49:08.254 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5529999999999998e-06 2024-09-16T14:49:08.254 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.254 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.254 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.254 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.886204+0000", 2024-09-16T14:49:08.255 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005455321 2024-09-16T14:49:08.255 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.255 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.255 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.255 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.886232+0000", 2024-09-16T14:49:08.255 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7895e-05 2024-09-16T14:49:08.255 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.255 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.255 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.256 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.256 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.256 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as1 52/27 ECSubWrite(tid=187, reqid=osd.11.0:187, at_version=52'24, trim_to=0'0, roll_forward_to=51'22))", 2024-09-16T14:49:08.256 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:58.903387+0000", 2024-09-16T14:49:08.256 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 9.2794953410000005, 2024-09-16T14:49:08.256 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0059598029999999996, 2024-09-16T14:49:08.256 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.256 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.256 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.256 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.256 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.256 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903387+0000", 2024-09-16T14:49:08.256 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.257 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.257 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.257 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.257 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903387+0000", 2024-09-16T14:49:08.257 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.257 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.257 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.257 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.257 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903386+0000", 2024-09-16T14:49:08.257 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.257 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.257 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.257 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.258 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903399+0000", 2024-09-16T14:49:08.258 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3086e-05 2024-09-16T14:49:08.258 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.258 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.258 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.258 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903405+0000", 2024-09-16T14:49:08.258 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8159999999999999e-06 2024-09-16T14:49:08.258 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.258 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.258 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.258 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903411+0000", 2024-09-16T14:49:08.258 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.84e-06 2024-09-16T14:49:08.258 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.258 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.259 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.259 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903467+0000", 2024-09-16T14:49:08.259 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5878000000000003e-05 2024-09-16T14:49:08.259 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.259 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.259 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.259 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903476+0000", 2024-09-16T14:49:08.259 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8119999999999997e-06 2024-09-16T14:49:08.259 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.259 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.259 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.259 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909323+0000", 2024-09-16T14:49:08.259 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005847316 2024-09-16T14:49:08.260 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.260 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.260 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.260 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909346+0000", 2024-09-16T14:49:08.260 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.338e-05 2024-09-16T14:49:08.260 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.260 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.260 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.260 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.260 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.260 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s3 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:08.260 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932044+0000", 2024-09-16T14:49:08.260 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.250837898, 2024-09-16T14:49:08.261 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0086059629999999995, 2024-09-16T14:49:08.261 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.261 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.261 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.261 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.261 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.261 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932044+0000", 2024-09-16T14:49:08.261 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.262 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.262 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.262 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.262 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932044+0000", 2024-09-16T14:49:08.262 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.262 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.262 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.263 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.263 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932043+0000", 2024-09-16T14:49:08.263 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.263 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.263 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.263 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.263 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932067+0000", 2024-09-16T14:49:08.263 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3501000000000001e-05 2024-09-16T14:49:08.263 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.263 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.263 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.263 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932078+0000", 2024-09-16T14:49:08.264 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1151999999999999e-05 2024-09-16T14:49:08.264 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.264 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.264 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.264 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932085+0000", 2024-09-16T14:49:08.264 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4019999999999996e-06 2024-09-16T14:49:08.264 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.264 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.264 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.264 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932134+0000", 2024-09-16T14:49:08.264 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8698e-05 2024-09-16T14:49:08.264 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.264 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.264 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.265 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932144+0000", 2024-09-16T14:49:08.265 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7440000000000002e-06 2024-09-16T14:49:08.265 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.265 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.265 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.265 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.940624+0000", 2024-09-16T14:49:08.265 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084803870000000007 2024-09-16T14:49:08.265 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.265 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.265 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.265 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.940650+0000", 2024-09-16T14:49:08.265 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5769999999999999e-05 2024-09-16T14:49:08.265 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.266 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.266 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.266 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.266 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.266 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds6 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:08.266 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932207+0000", 2024-09-16T14:49:08.266 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.25067483699999998, 2024-09-16T14:49:08.266 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0085983610000000005, 2024-09-16T14:49:08.267 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.267 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.267 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.267 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.267 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.267 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932207+0000", 2024-09-16T14:49:08.267 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.267 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.267 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.267 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.267 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932207+0000", 2024-09-16T14:49:08.267 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.267 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.268 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.268 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.268 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932206+0000", 2024-09-16T14:49:08.268 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.268 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.268 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.268 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.268 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932229+0000", 2024-09-16T14:49:08.268 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3074e-05 2024-09-16T14:49:08.268 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.268 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.268 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.268 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932241+0000", 2024-09-16T14:49:08.269 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1505000000000001e-05 2024-09-16T14:49:08.269 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.269 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.269 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.269 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932249+0000", 2024-09-16T14:49:08.269 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7400000000000004e-06 2024-09-16T14:49:08.269 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.269 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.269 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.269 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932295+0000", 2024-09-16T14:49:08.269 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5908999999999997e-05 2024-09-16T14:49:08.269 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.269 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.269 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.270 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932304+0000", 2024-09-16T14:49:08.270 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1249999999999999e-06 2024-09-16T14:49:08.270 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.270 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.270 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.270 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.940790+0000", 2024-09-16T14:49:08.270 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084867380000000006 2024-09-16T14:49:08.270 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.270 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.270 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.270 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.940805+0000", 2024-09-16T14:49:08.270 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4934e-05 2024-09-16T14:49:08.270 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.270 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.271 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.271 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.271 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.271 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as1 61/53 ECSubWrite(tid=225, reqid=osd.11.0:225, at_version=61'26, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:08.271 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932623+0000", 2024-09-16T14:49:08.271 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.25025903799999999, 2024-09-16T14:49:08.271 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089969119999999993, 2024-09-16T14:49:08.271 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.271 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.271 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.271 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.271 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.271 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932623+0000", 2024-09-16T14:49:08.272 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.272 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.272 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.272 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.272 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932623+0000", 2024-09-16T14:49:08.272 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.272 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.272 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.272 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.272 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932622+0000", 2024-09-16T14:49:08.272 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.272 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.273 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.273 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.273 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932643+0000", 2024-09-16T14:49:08.273 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0611e-05 2024-09-16T14:49:08.273 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.273 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.273 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.273 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932656+0000", 2024-09-16T14:49:08.274 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3613e-05 2024-09-16T14:49:08.274 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.274 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.274 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.274 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932663+0000", 2024-09-16T14:49:08.274 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0149999999999998e-06 2024-09-16T14:49:08.274 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.274 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.274 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.274 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932790+0000", 2024-09-16T14:49:08.275 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00012615099999999999 2024-09-16T14:49:08.275 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.275 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.275 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.275 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932803+0000", 2024-09-16T14:49:08.275 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3355e-05 2024-09-16T14:49:08.275 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.275 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.275 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.275 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941602+0000", 2024-09-16T14:49:08.275 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0087990859999999994 2024-09-16T14:49:08.275 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.275 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.276 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.276 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941620+0000", 2024-09-16T14:49:08.276 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7710000000000002e-05 2024-09-16T14:49:08.276 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.276 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.276 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.276 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.276 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.276 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.cs2 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:08.276 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932695+0000", 2024-09-16T14:49:08.277 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.250186409, 2024-09-16T14:49:08.277 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089530259999999993, 2024-09-16T14:49:08.277 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.277 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.277 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.277 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.277 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.277 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932695+0000", 2024-09-16T14:49:08.277 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.278 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.278 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.278 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.278 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932695+0000", 2024-09-16T14:49:08.278 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.278 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.279 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.279 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.279 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932695+0000", 2024-09-16T14:49:08.279 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.279 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.279 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.279 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.279 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932711+0000", 2024-09-16T14:49:08.279 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6248000000000001e-05 2024-09-16T14:49:08.279 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.279 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.280 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.280 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932721+0000", 2024-09-16T14:49:08.280 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7489999999999993e-06 2024-09-16T14:49:08.280 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.280 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.280 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.280 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932728+0000", 2024-09-16T14:49:08.280 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:49:08.280 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.280 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.280 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.281 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932815+0000", 2024-09-16T14:49:08.281 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6984999999999995e-05 2024-09-16T14:49:08.281 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.281 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.281 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.281 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932825+0000", 2024-09-16T14:49:08.281 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0459999999999994e-06 2024-09-16T14:49:08.281 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.281 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.281 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.281 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941636+0000", 2024-09-16T14:49:08.281 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0088109880000000005 2024-09-16T14:49:08.281 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.281 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.282 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.282 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941649+0000", 2024-09-16T14:49:08.282 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2989e-05 2024-09-16T14:49:08.282 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.282 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.282 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.282 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.282 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.282 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.5s6 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:08.282 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932903+0000", 2024-09-16T14:49:08.282 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.249978538, 2024-09-16T14:49:08.283 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0087662160000000003, 2024-09-16T14:49:08.283 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.283 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.283 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.283 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.283 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.283 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932903+0000", 2024-09-16T14:49:08.283 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.283 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.283 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.283 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.283 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932903+0000", 2024-09-16T14:49:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932903+0000", 2024-09-16T14:49:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932917+0000", 2024-09-16T14:49:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4443e-05 2024-09-16T14:49:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.284 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.285 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932927+0000", 2024-09-16T14:49:08.285 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8910000000000007e-06 2024-09-16T14:49:08.285 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.285 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.285 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.285 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932934+0000", 2024-09-16T14:49:08.285 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.985e-06 2024-09-16T14:49:08.285 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.285 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.285 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.285 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932978+0000", 2024-09-16T14:49:08.285 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3730000000000003e-05 2024-09-16T14:49:08.285 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.286 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.286 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.286 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932987+0000", 2024-09-16T14:49:08.286 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3940000000000002e-06 2024-09-16T14:49:08.286 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.286 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.286 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.286 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941659+0000", 2024-09-16T14:49:08.286 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0086717840000000001 2024-09-16T14:49:08.286 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.286 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.286 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.286 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941670+0000", 2024-09-16T14:49:08.286 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0654e-05 2024-09-16T14:49:08.287 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.287 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.287 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.287 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.287 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.287 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.cs2 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:08.287 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.933500+0000", 2024-09-16T14:49:08.287 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.24938168699999999, 2024-09-16T14:49:08.287 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0087068370000000003, 2024-09-16T14:49:08.287 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.288 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.288 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.288 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.288 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.288 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933500+0000", 2024-09-16T14:49:08.288 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.288 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.288 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.288 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.288 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933500+0000", 2024-09-16T14:49:08.288 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.288 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.288 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.289 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.289 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933499+0000", 2024-09-16T14:49:08.289 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:08.289 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.289 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.289 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.289 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933519+0000", 2024-09-16T14:49:08.289 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9932000000000002e-05 2024-09-16T14:49:08.290 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.290 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.290 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.290 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933541+0000", 2024-09-16T14:49:08.290 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1698000000000001e-05 2024-09-16T14:49:08.290 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.290 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.290 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.290 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933554+0000", 2024-09-16T14:49:08.290 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2995e-05 2024-09-16T14:49:08.291 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.291 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.291 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.291 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935629+0000", 2024-09-16T14:49:08.291 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002075061 2024-09-16T14:49:08.291 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.291 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.291 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.291 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935637+0000", 2024-09-16T14:49:08.291 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4079999999999998e-06 2024-09-16T14:49:08.291 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.291 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.292 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.292 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942191+0000", 2024-09-16T14:49:08.292 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0065539630000000003 2024-09-16T14:49:08.292 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.292 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.292 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.292 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942207+0000", 2024-09-16T14:49:08.292 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5622999999999999e-05 2024-09-16T14:49:08.292 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.292 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.292 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.292 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.292 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.293 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.fs7 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:08.293 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.934963+0000", 2024-09-16T14:49:08.293 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.24791929800000001, 2024-09-16T14:49:08.293 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0068608339999999997, 2024-09-16T14:49:08.293 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.293 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.293 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.293 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.293 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.293 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.934963+0000", 2024-09-16T14:49:08.293 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.293 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.293 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.934963+0000", 2024-09-16T14:49:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.934953+0000", 2024-09-16T14:49:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999905 2024-09-16T14:49:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935094+0000", 2024-09-16T14:49:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014116399999999999 2024-09-16T14:49:08.294 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.295 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.295 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.295 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935105+0000", 2024-09-16T14:49:08.295 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1279e-05 2024-09-16T14:49:08.295 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.295 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.295 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.295 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935113+0000", 2024-09-16T14:49:08.295 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7300000000000005e-06 2024-09-16T14:49:08.295 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.295 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.295 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.295 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935179+0000", 2024-09-16T14:49:08.295 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.5569999999999997e-05 2024-09-16T14:49:08.296 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.296 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.296 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.296 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935189+0000", 2024-09-16T14:49:08.296 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0074e-05 2024-09-16T14:49:08.296 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.296 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.296 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.296 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941808+0000", 2024-09-16T14:49:08.296 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0066187620000000003 2024-09-16T14:49:08.297 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.297 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.297 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.297 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941823+0000", 2024-09-16T14:49:08.297 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5905000000000001e-05 2024-09-16T14:49:08.297 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.297 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.297 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.298 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.298 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.298 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds6 61/27 ECSubWrite(tid=237, reqid=osd.7.0:237, at_version=61'50, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:08.298 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.936341+0000", 2024-09-16T14:49:08.298 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.246541339, 2024-09-16T14:49:08.298 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0059225149999999997, 2024-09-16T14:49:08.298 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.298 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.298 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.298 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.298 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.299 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.936341+0000", 2024-09-16T14:49:08.299 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.299 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.299 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.299 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.299 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.936341+0000", 2024-09-16T14:49:08.299 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.299 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.299 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.299 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.299 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.936340+0000", 2024-09-16T14:49:08.299 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.299 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.299 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.300 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.300 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.936362+0000", 2024-09-16T14:49:08.300 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1861e-05 2024-09-16T14:49:08.300 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.300 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.300 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.300 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.936369+0000", 2024-09-16T14:49:08.300 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5989999999999996e-06 2024-09-16T14:49:08.300 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.300 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.300 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.300 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.936374+0000", 2024-09-16T14:49:08.300 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9010000000000002e-06 2024-09-16T14:49:08.300 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.301 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.301 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.301 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.936407+0000", 2024-09-16T14:49:08.301 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2530000000000002e-05 2024-09-16T14:49:08.301 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.301 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.301 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.301 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.936412+0000", 2024-09-16T14:49:08.301 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.7139999999999998e-06 2024-09-16T14:49:08.301 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.301 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.301 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.301 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942253+0000", 2024-09-16T14:49:08.302 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0058404909999999997 2024-09-16T14:49:08.302 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.302 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.302 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.302 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942263+0000", 2024-09-16T14:49:08.302 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0105999999999999e-05 2024-09-16T14:49:08.302 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.302 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.302 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.302 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.302 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.302 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.es3 61/27 ECSubWrite(tid=231, reqid=osd.11.0:231, at_version=61'37, trim_to=0'0, roll_forward_to=60'33))", 2024-09-16T14:49:08.303 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.951596+0000", 2024-09-16T14:49:08.303 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.23128563299999999, 2024-09-16T14:49:08.303 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.006470394, 2024-09-16T14:49:08.303 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.303 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.303 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.303 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.303 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.303 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.951596+0000", 2024-09-16T14:49:08.303 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.303 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.303 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.303 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.304 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.951596+0000", 2024-09-16T14:49:08.304 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.304 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.304 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.304 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.304 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.951596+0000", 2024-09-16T14:49:08.304 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.304 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.304 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.304 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.304 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.951615+0000", 2024-09-16T14:49:08.304 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9102000000000001e-05 2024-09-16T14:49:08.304 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.304 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.305 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.305 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.951624+0000", 2024-09-16T14:49:08.305 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9600000000000006e-06 2024-09-16T14:49:08.305 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.305 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.305 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.305 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.951628+0000", 2024-09-16T14:49:08.305 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6840000000000004e-06 2024-09-16T14:49:08.305 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.305 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.305 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.305 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.951665+0000", 2024-09-16T14:49:08.305 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6297999999999997e-05 2024-09-16T14:49:08.306 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.306 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.306 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.306 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.951673+0000", 2024-09-16T14:49:08.306 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8470000000000002e-06 2024-09-16T14:49:08.306 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.306 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.306 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.306 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.958043+0000", 2024-09-16T14:49:08.306 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0063693279999999996 2024-09-16T14:49:08.306 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.307 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.307 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.307 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.958067+0000", 2024-09-16T14:49:08.307 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.385e-05 2024-09-16T14:49:08.307 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.307 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.307 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.307 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.307 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.307 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:08.309 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:49:08.309 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:08.307+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:08.410 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:08.408+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:08.452 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:08.452 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:08.452 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:08.452 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:08.461 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:49:08.511 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:08.510+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:08.603 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:08.603 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:08.603 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:08.603 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:08.603 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:08.612 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:49:08.612 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:08.610+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:08.713 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:08.712+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:08.716 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:08.716 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:08.716 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:08.716 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:08.716 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.716 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s4 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:08.717 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921852+0000", 2024-09-16T14:49:08.717 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.792492477, 2024-09-16T14:49:08.717 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0065836089999999998, 2024-09-16T14:49:08.717 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.717 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.717 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.717 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.717 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.717 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921852+0000", 2024-09-16T14:49:08.717 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.717 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.717 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.717 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.717 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921852+0000", 2024-09-16T14:49:08.718 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.718 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.718 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.718 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.718 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921851+0000", 2024-09-16T14:49:08.718 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:08.718 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.718 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.718 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.718 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921890+0000", 2024-09-16T14:49:08.718 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8668000000000003e-05 2024-09-16T14:49:08.718 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.718 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.719 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.719 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921906+0000", 2024-09-16T14:49:08.719 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6772000000000001e-05 2024-09-16T14:49:08.719 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.719 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.719 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.719 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921920+0000", 2024-09-16T14:49:08.719 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3525e-05 2024-09-16T14:49:08.719 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.719 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.719 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.719 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923071+0000", 2024-09-16T14:49:08.719 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001150812 2024-09-16T14:49:08.719 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.720 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.720 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.720 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923079+0000", 2024-09-16T14:49:08.720 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6119999999999999e-06 2024-09-16T14:49:08.720 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.720 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.720 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.720 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.928412+0000", 2024-09-16T14:49:08.720 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053327419999999997 2024-09-16T14:49:08.720 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.720 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.721 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.721 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.928435+0000", 2024-09-16T14:49:08.721 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3419999999999999e-05 2024-09-16T14:49:08.721 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.721 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.721 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.721 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.721 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.721 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:449 3.5s0 3:a7111aa9:::smithi04040151-45:head [stat,delete] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:08.721 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:08.721 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.837957869, 2024-09-16T14:49:08.721 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013571421, 2024-09-16T14:49:08.721 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.722 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:08.722 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:08.722 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:08.722 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:08.722 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 449 2024-09-16T14:49:08.722 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.722 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.722 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.722 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.722 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:08.722 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.722 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.722 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.723 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.723 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:08.723 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.723 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.723 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.723 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.723 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876385+0000", 2024-09-16T14:49:08.723 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:08.723 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.723 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.723 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.723 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876396+0000", 2024-09-16T14:49:08.723 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1094000000000001e-05 2024-09-16T14:49:08.723 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.724 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.724 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.724 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876412+0000", 2024-09-16T14:49:08.724 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6563e-05 2024-09-16T14:49:08.724 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.724 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.724 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.724 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876421+0000", 2024-09-16T14:49:08.724 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5189999999999992e-06 2024-09-16T14:49:08.724 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.724 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.724 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.724 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876477+0000", 2024-09-16T14:49:08.725 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.6679999999999999e-05 2024-09-16T14:49:08.725 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.725 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.725 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:08.725 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876487+0000", 2024-09-16T14:49:08.725 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4509999999999998e-06 2024-09-16T14:49:08.725 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.725 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.725 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.725 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.878486+0000", 2024-09-16T14:49:08.725 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001999635 2024-09-16T14:49:08.725 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.725 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.725 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:08.726 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.878592+0000", 2024-09-16T14:49:08.726 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000105398 2024-09-16T14:49:08.726 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.726 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.726 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.726 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.889958+0000", 2024-09-16T14:49:08.726 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01136595 2024-09-16T14:49:08.726 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.726 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.726 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.726 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.726 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.726 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:466 3.5s0 3:a29e72cc:::smithi04040151-39:head [append 0~0] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:08.727 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995540+0000", 2024-09-16T14:49:08.727 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.718804287999999, 2024-09-16T14:49:08.727 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084481160000000003, 2024-09-16T14:49:08.727 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.727 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:08.727 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:08.727 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:08.727 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:08.727 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 466 2024-09-16T14:49:08.727 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.727 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.727 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.727 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.727 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995540+0000", 2024-09-16T14:49:08.728 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.728 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.728 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.728 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.728 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995540+0000", 2024-09-16T14:49:08.728 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.728 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.728 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.728 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.728 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995538+0000", 2024-09-16T14:49:08.728 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:08.728 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.728 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.729 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.729 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995548+0000", 2024-09-16T14:49:08.729 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1570000000000006e-06 2024-09-16T14:49:08.729 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.729 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.729 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.729 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995560+0000", 2024-09-16T14:49:08.729 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2527999999999999e-05 2024-09-16T14:49:08.729 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.729 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.729 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.729 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995567+0000", 2024-09-16T14:49:08.729 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2030000000000003e-06 2024-09-16T14:49:08.730 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.730 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.730 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.730 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995620+0000", 2024-09-16T14:49:08.730 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2516000000000003e-05 2024-09-16T14:49:08.730 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.730 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.730 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:08.730 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995718+0000", 2024-09-16T14:49:08.730 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.781e-05 2024-09-16T14:49:08.730 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.730 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.730 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.730 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.996361+0000", 2024-09-16T14:49:08.731 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00064323600000000005 2024-09-16T14:49:08.731 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.731 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.731 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.731 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.999417+0000", 2024-09-16T14:49:08.731 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0030556120000000001 2024-09-16T14:49:08.731 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.731 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.731 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:08.731 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003945+0000", 2024-09-16T14:49:08.732 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045287950000000004 2024-09-16T14:49:08.732 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.732 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.732 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.732 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003988+0000", 2024-09-16T14:49:08.732 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2758e-05 2024-09-16T14:49:08.732 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.732 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.733 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.733 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.733 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.733 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:467 3.5s0 3:a29e72cc:::smithi04040151-39:head [rollback 14] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:08.733 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:08.733 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.718765311, 2024-09-16T14:49:08.733 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012554012, 2024-09-16T14:49:08.733 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.733 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:08.733 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:08.733 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:08.733 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:08.733 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 467 2024-09-16T14:49:08.734 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.734 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.734 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.734 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.734 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:08.734 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.734 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.734 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.734 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.734 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:08.734 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.734 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995581+0000", 2024-09-16T14:49:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.9009999999999998e-06 2024-09-16T14:49:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995587+0000", 2024-09-16T14:49:08.736 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8850000000000004e-06 2024-09-16T14:49:08.736 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.736 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.736 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.736 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995592+0000", 2024-09-16T14:49:08.736 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9749999999999998e-06 2024-09-16T14:49:08.736 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.736 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.736 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.736 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.996914+0000", 2024-09-16T14:49:08.736 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00132168 2024-09-16T14:49:08.736 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.736 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.737 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:08.737 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997029+0000", 2024-09-16T14:49:08.737 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000115203 2024-09-16T14:49:08.737 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.737 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.737 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.737 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997570+0000", 2024-09-16T14:49:08.737 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00054107999999999999 2024-09-16T14:49:08.737 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.737 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.737 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.737 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003265+0000", 2024-09-16T14:49:08.737 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0056945679999999997 2024-09-16T14:49:08.737 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.738 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.738 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:08.738 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008067+0000", 2024-09-16T14:49:08.738 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048018039999999998 2024-09-16T14:49:08.738 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.738 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.738 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.738 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008133+0000", 2024-09-16T14:49:08.738 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6365999999999994e-05 2024-09-16T14:49:08.738 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.738 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.738 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.738 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.739 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.739 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:468 3.5s0 3:a29e72cc:::smithi04040151-39:head [append 0~0] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:08.739 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:08.739 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.718596250999999, 2024-09-16T14:49:08.739 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012562663999999999, 2024-09-16T14:49:08.739 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.739 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:08.739 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:08.739 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:08.739 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:08.739 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 468 2024-09-16T14:49:08.739 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.739 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.740 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.740 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.740 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:08.740 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.740 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.740 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.740 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.740 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:08.740 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.740 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.740 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.740 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.740 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995747+0000", 2024-09-16T14:49:08.741 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:08.741 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.741 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.741 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.741 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995752+0000", 2024-09-16T14:49:08.741 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.916e-06 2024-09-16T14:49:08.741 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.741 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.741 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.741 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995762+0000", 2024-09-16T14:49:08.741 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4609999999999997e-06 2024-09-16T14:49:08.741 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.741 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.741 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.742 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995768+0000", 2024-09-16T14:49:08.742 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8109999999999997e-06 2024-09-16T14:49:08.742 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.742 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.742 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.742 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.998577+0000", 2024-09-16T14:49:08.742 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0028088369999999998 2024-09-16T14:49:08.742 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.742 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.742 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:08.742 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.998680+0000", 2024-09-16T14:49:08.742 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010318399999999999 2024-09-16T14:49:08.742 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.742 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.743 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.743 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.999182+0000", 2024-09-16T14:49:08.743 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00050101099999999999 2024-09-16T14:49:08.743 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.743 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.743 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.743 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.005964+0000", 2024-09-16T14:49:08.743 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0067821490000000003 2024-09-16T14:49:08.743 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.743 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.743 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:08.743 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008246+0000", 2024-09-16T14:49:08.743 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0022827670000000002 2024-09-16T14:49:08.744 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.744 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.744 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.744 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008311+0000", 2024-09-16T14:49:08.744 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.4269000000000004e-05 2024-09-16T14:49:08.744 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.744 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.744 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.744 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.744 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.744 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.9s4 52/27 ECSubWrite(tid=38, reqid=osd.8.0:37, at_version=0'0, trim_to=0'0, roll_forward_to=52'23))", 2024-09-16T14:49:08.744 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:58.892213+0000", 2024-09-16T14:49:08.744 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 9.8221311950000008, 2024-09-16T14:49:08.744 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005572039, 2024-09-16T14:49:08.745 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.745 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.745 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.745 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.745 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.745 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892213+0000", 2024-09-16T14:49:08.745 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.745 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.745 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.745 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.745 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892213+0000", 2024-09-16T14:49:08.745 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.745 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.745 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.746 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.746 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892212+0000", 2024-09-16T14:49:08.746 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:08.746 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.746 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.746 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.746 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892221+0000", 2024-09-16T14:49:08.746 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9670000000000007e-06 2024-09-16T14:49:08.746 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.746 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.746 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.746 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892232+0000", 2024-09-16T14:49:08.746 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0651e-05 2024-09-16T14:49:08.747 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.747 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.747 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.747 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892240+0000", 2024-09-16T14:49:08.747 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0169999999999996e-06 2024-09-16T14:49:08.747 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.747 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.747 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.747 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892287+0000", 2024-09-16T14:49:08.747 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7119000000000001e-05 2024-09-16T14:49:08.747 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.747 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.747 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.748 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.892295+0000", 2024-09-16T14:49:08.748 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.1040000000000007e-06 2024-09-16T14:49:08.748 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.748 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.748 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.748 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.897763+0000", 2024-09-16T14:49:08.748 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0054674019999999997 2024-09-16T14:49:08.748 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.748 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.748 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.748 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.897785+0000", 2024-09-16T14:49:08.748 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2504999999999999e-05 2024-09-16T14:49:08.748 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.749 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.749 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.749 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.749 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.749 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 52/27 ECSubWrite(tid=185, reqid=osd.11.0:185, at_version=52'62, trim_to=0'0, roll_forward_to=52'60))", 2024-09-16T14:49:08.749 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:58.900768+0000", 2024-09-16T14:49:08.749 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 9.8135762230000001, 2024-09-16T14:49:08.749 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083307420000000004, 2024-09-16T14:49:08.749 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.749 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.749 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.749 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.749 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.749 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900768+0000", 2024-09-16T14:49:08.750 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.750 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.750 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.750 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.750 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900768+0000", 2024-09-16T14:49:08.750 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.750 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.750 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.750 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.750 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900767+0000", 2024-09-16T14:49:08.750 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.750 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.750 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.751 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.751 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900790+0000", 2024-09-16T14:49:08.751 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2812000000000001e-05 2024-09-16T14:49:08.751 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.751 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.751 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.751 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900801+0000", 2024-09-16T14:49:08.751 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0954999999999999e-05 2024-09-16T14:49:08.751 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.751 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.751 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.751 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900810+0000", 2024-09-16T14:49:08.751 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5019999999999992e-06 2024-09-16T14:49:08.751 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.752 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.752 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.752 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900851+0000", 2024-09-16T14:49:08.752 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1845999999999999e-05 2024-09-16T14:49:08.752 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.752 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.752 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.752 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900860+0000", 2024-09-16T14:49:08.752 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4630000000000001e-06 2024-09-16T14:49:08.752 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.752 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.752 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.752 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909074+0000", 2024-09-16T14:49:08.753 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082140349999999997 2024-09-16T14:49:08.753 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.753 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.753 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.753 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909099+0000", 2024-09-16T14:49:08.753 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4808e-05 2024-09-16T14:49:08.753 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.753 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.753 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.753 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.753 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.753 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds1 52/27 ECSubWrite(tid=215, reqid=osd.7.0:214, at_version=0'0, trim_to=0'0, roll_forward_to=52'35))", 2024-09-16T14:49:08.753 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:58.901786+0000", 2024-09-16T14:49:08.753 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 9.8125582910000002, 2024-09-16T14:49:08.754 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0073522930000000002, 2024-09-16T14:49:08.754 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.754 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.754 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.754 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.754 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.754 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901786+0000", 2024-09-16T14:49:08.754 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.754 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.754 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.754 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.754 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901786+0000", 2024-09-16T14:49:08.754 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.755 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.755 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.755 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.755 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901785+0000", 2024-09-16T14:49:08.755 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:08.755 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.755 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.755 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.755 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901796+0000", 2024-09-16T14:49:08.755 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0475000000000001e-05 2024-09-16T14:49:08.755 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.755 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.755 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.755 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901808+0000", 2024-09-16T14:49:08.756 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2755000000000001e-05 2024-09-16T14:49:08.756 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.756 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.756 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.756 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901817+0000", 2024-09-16T14:49:08.756 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7409999999999998e-06 2024-09-16T14:49:08.756 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.756 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.756 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.756 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901871+0000", 2024-09-16T14:49:08.756 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3940000000000002e-05 2024-09-16T14:49:08.756 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.756 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.757 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.757 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901880+0000", 2024-09-16T14:49:08.757 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2539999999999998e-06 2024-09-16T14:49:08.757 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.757 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.757 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.757 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909107+0000", 2024-09-16T14:49:08.757 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00722684 2024-09-16T14:49:08.757 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.757 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.757 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.757 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909138+0000", 2024-09-16T14:49:08.757 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.112e-05 2024-09-16T14:49:08.758 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.758 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.758 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.758 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.758 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.758 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s5 52/27 ECSubWrite(tid=30, reqid=osd.4.0:29, at_version=0'0, trim_to=0'0, roll_forward_to=52'20))", 2024-09-16T14:49:08.758 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:58.903955+0000", 2024-09-16T14:49:08.758 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 9.8103892179999992, 2024-09-16T14:49:08.758 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0059277990000000001, 2024-09-16T14:49:08.758 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.758 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.758 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.758 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.758 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.759 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903955+0000", 2024-09-16T14:49:08.759 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.759 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.759 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.759 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.759 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903955+0000", 2024-09-16T14:49:08.759 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.759 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.759 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.759 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.759 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903954+0000", 2024-09-16T14:49:08.759 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:08.759 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.759 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.760 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.760 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903965+0000", 2024-09-16T14:49:08.760 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0728e-05 2024-09-16T14:49:08.760 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.760 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.760 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.760 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903981+0000", 2024-09-16T14:49:08.760 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6166000000000001e-05 2024-09-16T14:49:08.760 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.760 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.760 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.760 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.903989+0000", 2024-09-16T14:49:08.760 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0809999999999994e-06 2024-09-16T14:49:08.761 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.761 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.761 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.761 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.904038+0000", 2024-09-16T14:49:08.761 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8528e-05 2024-09-16T14:49:08.761 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.761 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.761 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.761 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.904047+0000", 2024-09-16T14:49:08.761 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5049999999999996e-06 2024-09-16T14:49:08.761 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.761 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.761 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.762 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909865+0000", 2024-09-16T14:49:08.762 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0058180790000000003 2024-09-16T14:49:08.762 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.762 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.762 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.762 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909883+0000", 2024-09-16T14:49:08.762 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7495000000000001e-05 2024-09-16T14:49:08.762 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.762 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.762 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.762 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.762 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.762 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:08.762 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932515+0000", 2024-09-16T14:49:08.763 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.781829518, 2024-09-16T14:49:08.763 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010770217, 2024-09-16T14:49:08.763 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.763 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.763 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.763 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.763 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.763 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932515+0000", 2024-09-16T14:49:08.763 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.763 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.763 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.763 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.763 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932515+0000", 2024-09-16T14:49:08.763 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.764 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.764 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.764 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.764 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932514+0000", 2024-09-16T14:49:08.764 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.764 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.764 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.764 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.764 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932539+0000", 2024-09-16T14:49:08.764 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4494e-05 2024-09-16T14:49:08.764 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.764 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.764 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.765 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932549+0000", 2024-09-16T14:49:08.765 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0028e-05 2024-09-16T14:49:08.765 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.765 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.765 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.765 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932556+0000", 2024-09-16T14:49:08.765 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.6920000000000002e-06 2024-09-16T14:49:08.765 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.765 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.765 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.765 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932618+0000", 2024-09-16T14:49:08.765 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.1600000000000007e-05 2024-09-16T14:49:08.765 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.765 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.766 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.766 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932626+0000", 2024-09-16T14:49:08.766 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4139999999999994e-06 2024-09-16T14:49:08.766 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.766 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.766 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.766 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943268+0000", 2024-09-16T14:49:08.766 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010641695 2024-09-16T14:49:08.766 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.766 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.766 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.766 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943285+0000", 2024-09-16T14:49:08.766 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6929e-05 2024-09-16T14:49:08.767 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.767 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.767 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.767 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.767 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.767 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds1 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:08.767 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:08.767 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.781576838, 2024-09-16T14:49:08.767 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010600091000000001, 2024-09-16T14:49:08.767 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.767 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.767 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.767 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.767 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.768 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:08.768 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.768 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.768 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.768 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.768 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:08.768 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.768 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.768 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.768 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.768 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:08.768 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.768 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.769 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.769 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.769 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932785+0000", 2024-09-16T14:49:08.769 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7829000000000002e-05 2024-09-16T14:49:08.769 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.769 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.769 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.769 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932794+0000", 2024-09-16T14:49:08.769 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5179999999999993e-06 2024-09-16T14:49:08.769 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.769 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.769 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.769 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932801+0000", 2024-09-16T14:49:08.770 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.793e-06 2024-09-16T14:49:08.770 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.770 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.770 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.770 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933189+0000", 2024-09-16T14:49:08.770 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00038759100000000001 2024-09-16T14:49:08.770 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.770 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.770 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.770 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933197+0000", 2024-09-16T14:49:08.770 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5290000000000008e-06 2024-09-16T14:49:08.770 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.770 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.770 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.771 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943351+0000", 2024-09-16T14:49:08.771 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010153673 2024-09-16T14:49:08.771 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.771 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.771 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.771 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943368+0000", 2024-09-16T14:49:08.771 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6538999999999999e-05 2024-09-16T14:49:08.771 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.771 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.771 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.771 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.771 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.771 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/27 ECSubWrite(tid=63, reqid=osd.1.0:63, at_version=61'50, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:08.771 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:08.772 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.78152410299999997, 2024-09-16T14:49:08.772 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010214164, 2024-09-16T14:49:08.772 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.772 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.772 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.772 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.772 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.772 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:08.772 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.772 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.772 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.772 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.772 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:08.772 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.773 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.773 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.773 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.773 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:08.773 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.773 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.773 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.773 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.773 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932838+0000", 2024-09-16T14:49:08.773 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8576e-05 2024-09-16T14:49:08.773 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.773 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.773 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.774 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932847+0000", 2024-09-16T14:49:08.774 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6519999999999995e-06 2024-09-16T14:49:08.774 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.774 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.774 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.774 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932855+0000", 2024-09-16T14:49:08.774 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7600000000000002e-06 2024-09-16T14:49:08.774 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.774 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.774 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.774 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932929+0000", 2024-09-16T14:49:08.774 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4134999999999994e-05 2024-09-16T14:49:08.774 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.774 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.775 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.775 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932937+0000", 2024-09-16T14:49:08.775 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.1519999999999992e-06 2024-09-16T14:49:08.775 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.775 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.775 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.775 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941947+0000", 2024-09-16T14:49:08.775 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0090100430000000006 2024-09-16T14:49:08.775 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.775 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.775 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.775 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943034+0000", 2024-09-16T14:49:08.775 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001087111 2024-09-16T14:49:08.776 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.776 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.776 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.776 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.776 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.776 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as2 61/53 ECSubWrite(tid=225, reqid=osd.11.0:225, at_version=61'26, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:08.776 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:08.776 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.78143011600000001, 2024-09-16T14:49:08.776 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010782797, 2024-09-16T14:49:08.776 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.776 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.776 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.776 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.778 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.778 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.778 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932954+0000", 2024-09-16T14:49:08.778 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0166000000000001e-05 2024-09-16T14:49:08.778 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.778 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.778 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.778 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932980+0000", 2024-09-16T14:49:08.778 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5843e-05 2024-09-16T14:49:08.778 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.778 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.778 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.778 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932987+0000", 2024-09-16T14:49:08.778 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0079999999999996e-06 2024-09-16T14:49:08.779 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.779 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.779 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.779 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933034+0000", 2024-09-16T14:49:08.779 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6934999999999997e-05 2024-09-16T14:49:08.779 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.779 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.779 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.779 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933042+0000", 2024-09-16T14:49:08.779 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8429999999999999e-06 2024-09-16T14:49:08.779 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.779 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.779 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.779 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943652+0000", 2024-09-16T14:49:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010609288999999999 2024-09-16T14:49:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943697+0000", 2024-09-16T14:49:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5296999999999997e-05 2024-09-16T14:49:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:08.780 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.78138003099999997, 2024-09-16T14:49:08.781 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010665579999999999, 2024-09-16T14:49:08.781 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.781 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.781 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.781 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.781 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.781 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:08.781 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.781 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.781 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.781 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.781 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:08.781 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.781 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932995+0000", 2024-09-16T14:49:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1084000000000003e-05 2024-09-16T14:49:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933003+0000", 2024-09-16T14:49:08.782 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2199999999999992e-06 2024-09-16T14:49:08.783 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.783 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.783 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.783 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933010+0000", 2024-09-16T14:49:08.783 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.4359999999999999e-06 2024-09-16T14:49:08.783 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.783 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.783 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.783 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935460+0000", 2024-09-16T14:49:08.783 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0024503350000000001 2024-09-16T14:49:08.783 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.783 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.783 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.783 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935469+0000", 2024-09-16T14:49:08.784 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.602e-06 2024-09-16T14:49:08.784 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.784 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.784 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.784 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943610+0000", 2024-09-16T14:49:08.784 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0081417390000000003 2024-09-16T14:49:08.784 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.784 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.784 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.784 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943630+0000", 2024-09-16T14:49:08.784 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9584000000000001e-05 2024-09-16T14:49:08.784 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.784 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.785 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.785 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.785 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.785 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds1 61/27 ECSubWrite(tid=237, reqid=osd.7.0:237, at_version=61'50, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:08.785 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.935863+0000", 2024-09-16T14:49:08.785 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.77848159400000005, 2024-09-16T14:49:08.785 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0078102019999999996, 2024-09-16T14:49:08.785 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.785 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.785 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.785 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.785 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.785 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935863+0000", 2024-09-16T14:49:08.785 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.786 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.786 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.786 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.786 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935863+0000", 2024-09-16T14:49:08.786 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.786 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.786 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.786 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.786 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935862+0000", 2024-09-16T14:49:08.786 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:08.786 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.786 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.786 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.786 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935884+0000", 2024-09-16T14:49:08.787 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2419e-05 2024-09-16T14:49:08.787 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.787 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.787 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.787 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935898+0000", 2024-09-16T14:49:08.787 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3709999999999999e-05 2024-09-16T14:49:08.787 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.787 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.787 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.787 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935906+0000", 2024-09-16T14:49:08.787 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2670000000000006e-06 2024-09-16T14:49:08.787 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.787 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.787 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.788 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935949+0000", 2024-09-16T14:49:08.788 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2707999999999998e-05 2024-09-16T14:49:08.788 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.788 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.788 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.788 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935957+0000", 2024-09-16T14:49:08.788 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8599999999999993e-06 2024-09-16T14:49:08.788 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.788 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.788 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.788 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943638+0000", 2024-09-16T14:49:08.788 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0076813890000000003 2024-09-16T14:49:08.788 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.789 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.789 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.789 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943673+0000", 2024-09-16T14:49:08.789 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.4913999999999998e-05 2024-09-16T14:49:08.789 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.789 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.789 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.789 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.789 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.789 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 61/53 ECSubWriteReply(tid=94, last_complete=61'36, committed=1, applied=1))", 2024-09-16T14:49:08.789 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.937096+0000", 2024-09-16T14:49:08.789 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.77724828800000001, 2024-09-16T14:49:08.789 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0062012990000000004, 2024-09-16T14:49:08.790 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.790 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.790 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.790 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.790 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.790 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937096+0000", 2024-09-16T14:49:08.790 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.790 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.790 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.790 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.790 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937096+0000", 2024-09-16T14:49:08.790 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.790 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.790 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.791 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.791 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937095+0000", 2024-09-16T14:49:08.791 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:08.791 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.791 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.791 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.791 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937102+0000", 2024-09-16T14:49:08.791 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6839999999999999e-06 2024-09-16T14:49:08.791 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.791 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.791 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.791 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937113+0000", 2024-09-16T14:49:08.791 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1209000000000001e-05 2024-09-16T14:49:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937120+0000", 2024-09-16T14:49:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9789999999999996e-06 2024-09-16T14:49:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943282+0000", 2024-09-16T14:49:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0061620670000000002 2024-09-16T14:49:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.792 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943297+0000", 2024-09-16T14:49:08.793 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5313999999999999e-05 2024-09-16T14:49:08.793 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.793 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.793 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.793 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.793 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.793 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 61/27 ECSubWrite(tid=226, reqid=osd.11.0:226, at_version=61'82, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:08.793 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.938771+0000", 2024-09-16T14:49:08.793 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.77557289500000004, 2024-09-16T14:49:08.793 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083068940000000004, 2024-09-16T14:49:08.793 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.793 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.793 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.793 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.794 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.794 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938771+0000", 2024-09-16T14:49:08.794 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.794 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.794 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.794 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.794 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938771+0000", 2024-09-16T14:49:08.794 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.794 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.794 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.794 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.794 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938771+0000", 2024-09-16T14:49:08.794 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.794 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.795 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.795 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.795 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938785+0000", 2024-09-16T14:49:08.795 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4575e-05 2024-09-16T14:49:08.795 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.795 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.795 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.795 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938806+0000", 2024-09-16T14:49:08.795 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0801e-05 2024-09-16T14:49:08.795 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.795 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.795 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.795 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938811+0000", 2024-09-16T14:49:08.796 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.1100000000000002e-06 2024-09-16T14:49:08.796 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.796 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.796 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.796 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938850+0000", 2024-09-16T14:49:08.796 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8909999999999998e-05 2024-09-16T14:49:08.796 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.796 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.796 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.796 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938856+0000", 2024-09-16T14:49:08.796 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.981e-06 2024-09-16T14:49:08.796 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.796 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.796 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.797 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947064+0000", 2024-09-16T14:49:08.797 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082073700000000003 2024-09-16T14:49:08.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.797 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.797 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.797 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947078+0000", 2024-09-16T14:49:08.797 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4779e-05 2024-09-16T14:49:08.797 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.797 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.797 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.797 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.797 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.9s4 61/27 ECSubWrite(tid=52, reqid=osd.8.0:52, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:08.797 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.939040+0000", 2024-09-16T14:49:08.798 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.77530435399999997, 2024-09-16T14:49:08.798 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0080081979999999994, 2024-09-16T14:49:08.798 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.798 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.798 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.798 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.798 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.798 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939040+0000", 2024-09-16T14:49:08.798 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.798 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.798 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.798 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.798 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939040+0000", 2024-09-16T14:49:08.798 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.799 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.799 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.799 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939039+0000", 2024-09-16T14:49:08.799 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:08.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.799 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.799 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.799 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939058+0000", 2024-09-16T14:49:08.799 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9273999999999999e-05 2024-09-16T14:49:08.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.799 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.799 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.800 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939073+0000", 2024-09-16T14:49:08.800 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4375e-05 2024-09-16T14:49:08.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.800 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.800 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.800 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939082+0000", 2024-09-16T14:49:08.800 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3239999999999992e-06 2024-09-16T14:49:08.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.800 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.800 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.800 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939175+0000", 2024-09-16T14:49:08.800 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2423999999999995e-05 2024-09-16T14:49:08.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.800 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939185+0000", 2024-09-16T14:49:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9890000000000004e-06 2024-09-16T14:49:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947024+0000", 2024-09-16T14:49:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0078391329999999999 2024-09-16T14:49:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947048+0000", 2024-09-16T14:49:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4409e-05 2024-09-16T14:49:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.802 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.802 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.802 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.802 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.802 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.9s4 61/27 ECSubWrite(tid=53, reqid=osd.8.0:53, at_version=61'38, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:08.802 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.941071+0000", 2024-09-16T14:49:08.802 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.77327368699999999, 2024-09-16T14:49:08.802 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0069155809999999996, 2024-09-16T14:49:08.802 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.802 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.802 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.802 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.802 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.802 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941071+0000", 2024-09-16T14:49:08.803 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.803 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.803 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.803 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.803 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941071+0000", 2024-09-16T14:49:08.803 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.803 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.803 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.803 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.803 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941070+0000", 2024-09-16T14:49:08.803 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.803 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.803 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.803 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.804 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941089+0000", 2024-09-16T14:49:08.804 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8513999999999999e-05 2024-09-16T14:49:08.804 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.804 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.804 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.804 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941099+0000", 2024-09-16T14:49:08.804 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0049e-05 2024-09-16T14:49:08.804 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.804 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.804 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.804 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941105+0000", 2024-09-16T14:49:08.804 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2040000000000002e-06 2024-09-16T14:49:08.804 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.805 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.805 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.805 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943240+0000", 2024-09-16T14:49:08.805 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021350829999999999 2024-09-16T14:49:08.805 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.805 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.805 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.805 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943249+0000", 2024-09-16T14:49:08.805 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7110000000000001e-06 2024-09-16T14:49:08.805 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.805 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.805 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.805 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947967+0000", 2024-09-16T14:49:08.805 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0047180440000000002 2024-09-16T14:49:08.806 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.806 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.806 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.806 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947986+0000", 2024-09-16T14:49:08.806 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9480999999999999e-05 2024-09-16T14:49:08.806 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.806 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.806 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.806 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.806 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.806 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 61/27 ECSubWrite(tid=228, reqid=osd.11.0:226, at_version=0'0, trim_to=0'0, roll_forward_to=61'82))", 2024-09-16T14:49:08.806 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.949265+0000", 2024-09-16T14:49:08.806 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.76507934099999997, 2024-09-16T14:49:08.806 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.006050377, 2024-09-16T14:49:08.807 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:08.807 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:08.807 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:08.807 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.807 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:08.807 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.949265+0000", 2024-09-16T14:49:08.807 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.807 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.807 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.807 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:08.807 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.949265+0000", 2024-09-16T14:49:08.807 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:08.807 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.807 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.808 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:08.808 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.949264+0000", 2024-09-16T14:49:08.808 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:08.808 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.808 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.808 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:08.808 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.949273+0000", 2024-09-16T14:49:08.808 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0229999999999998e-06 2024-09-16T14:49:08.808 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.808 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.808 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:08.808 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.949283+0000", 2024-09-16T14:49:08.808 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4690000000000003e-06 2024-09-16T14:49:08.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.809 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:08.809 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.949289+0000", 2024-09-16T14:49:08.809 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.5180000000000001e-06 2024-09-16T14:49:08.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.809 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:08.809 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.949344+0000", 2024-09-16T14:49:08.809 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.4496000000000001e-05 2024-09-16T14:49:08.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.809 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:08.809 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.949354+0000", 2024-09-16T14:49:08.810 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8539999999999992e-06 2024-09-16T14:49:08.810 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.810 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.810 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:08.810 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.955291+0000", 2024-09-16T14:49:08.810 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0059375840000000001 2024-09-16T14:49:08.810 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:08.810 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:08.810 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:08.810 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.955315+0000", 2024-09-16T14:49:08.810 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4051999999999999e-05 2024-09-16T14:49:08.810 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.810 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.810 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.811 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:08.811 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:08.811 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:08.812 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:49:08.813 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:08.811+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:08.914 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:08.913+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:08.915 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:08.915 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:08.915 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:08.915 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:08.923 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:49:08.937 INFO:tasks.rados.rados.0.smithi040.stdout:227: done (5 left) 2024-09-16T14:49:08.937 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 40 current snap is 17 2024-09-16T14:49:08.937 INFO:tasks.rados.rados.0.smithi040.stdout:228: seq_num 93 ranges {1474560=802816,2277376=638976,2916352=131072} 2024-09-16T14:49:08.950 INFO:tasks.rados.rados.0.smithi040.stdout:228: writing smithi04040151-40 from 1474560 to 2277376 tid 1 2024-09-16T14:49:08.954 INFO:tasks.rados.rados.0.smithi040.stdout:228: writing smithi04040151-40 from 2277376 to 2916352 tid 2 2024-09-16T14:49:08.955 INFO:tasks.rados.rados.0.smithi040.stdout:228: writing smithi04040151-40 from 2916352 to 3047424 tid 3 2024-09-16T14:49:08.955 INFO:tasks.rados.rados.0.smithi040.stdout:229: snap_remove snap 16 2024-09-16T14:49:09.015 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:09.013+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:09.032 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:09.032 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:09.032 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:09.033 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:09.033 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:09.041 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:49:09.116 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:09.114+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:09.168 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:49:09.185 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:09.185 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:09.185 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:09.185 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:09.185 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.185 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:49:09.185 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:09.185 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 47.042892215999998, 2024-09-16T14:49:09.185 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.223181782, 2024-09-16T14:49:09.185 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.185 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.186 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.186 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:09.186 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:09.186 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 1 2024-09-16T14:49:09.186 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.186 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.186 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.186 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.186 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:09.186 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.186 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.186 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.186 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.186 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:09.187 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.187 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.187 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.187 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.187 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139495+0000", 2024-09-16T14:49:09.187 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:09.187 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.187 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.187 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.187 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139504+0000", 2024-09-16T14:49:09.187 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6119999999999999e-06 2024-09-16T14:49:09.187 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.187 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.188 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.188 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139514+0000", 2024-09-16T14:49:09.188 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:49:09.188 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.188 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.188 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.188 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139533+0000", 2024-09-16T14:49:09.188 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9573e-05 2024-09-16T14:49:09.188 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.188 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.188 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.188 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139585+0000", 2024-09-16T14:49:09.188 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2170999999999999e-05 2024-09-16T14:49:09.189 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.189 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.189 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:09.189 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139594+0000", 2024-09-16T14:49:09.189 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.884e-06 2024-09-16T14:49:09.189 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.189 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.189 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.189 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359233+0000", 2024-09-16T14:49:09.189 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.21963890899999999 2024-09-16T14:49:09.189 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.189 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.189 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359437+0000", 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00020374999999999999 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359611+0000", 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017445899999999999 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.360506+0000", 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00089459299999999995 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.190 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:09.191 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362594+0000", 2024-09-16T14:49:09.191 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002087776 2024-09-16T14:49:09.191 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.191 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.191 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.191 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362636+0000", 2024-09-16T14:49:09.191 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1933e-05 2024-09-16T14:49:09.191 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.191 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.191 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.191 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362679+0000", 2024-09-16T14:49:09.191 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2877999999999998e-05 2024-09-16T14:49:09.191 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.192 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.192 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.192 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.192 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.192 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:09.192 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:49:09.192 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 45.790367832999998, 2024-09-16T14:49:09.192 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0081462830000000007, 2024-09-16T14:49:09.192 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.192 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.192 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.192 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:09.192 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:09.193 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 159 2024-09-16T14:49:09.193 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.193 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.193 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.193 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.193 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:49:09.193 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.193 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.193 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.193 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.193 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:49:09.193 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.193 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.193 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.194 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.194 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392021+0000", 2024-09-16T14:49:09.194 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:09.194 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.194 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.194 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.194 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392023+0000", 2024-09-16T14:49:09.194 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3530000000000002e-06 2024-09-16T14:49:09.194 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.194 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.194 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.194 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392035+0000", 2024-09-16T14:49:09.194 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2207e-05 2024-09-16T14:49:09.194 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.195 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.195 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.195 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392040+0000", 2024-09-16T14:49:09.195 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7450000000000003e-06 2024-09-16T14:49:09.195 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.195 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.195 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.195 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394756+0000", 2024-09-16T14:49:09.195 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0027154010000000001 2024-09-16T14:49:09.195 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.195 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.195 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.195 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394799+0000", 2024-09-16T14:49:09.195 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3921999999999997e-05 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.394867+0000", 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7770999999999999e-05 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396922+0000", 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0020549819999999999 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400094+0000", 2024-09-16T14:49:09.196 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031720979999999999 2024-09-16T14:49:09.197 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.197 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.197 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.197 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400134+0000", 2024-09-16T14:49:09.197 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9449999999999997e-05 2024-09-16T14:49:09.197 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.197 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.197 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.197 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400167+0000", 2024-09-16T14:49:09.197 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3711000000000003e-05 2024-09-16T14:49:09.197 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.197 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.198 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.198 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.198 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.198 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:160 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:09.198 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:49:09.198 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 45.790345600000002, 2024-09-16T14:49:09.198 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082045659999999999, 2024-09-16T14:49:09.198 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.198 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.198 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.198 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:09.198 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:09.198 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 160 2024-09-16T14:49:09.198 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.199 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.199 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.199 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.199 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:49:09.199 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.199 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.199 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.199 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.199 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:49:09.200 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.200 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.200 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.200 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.200 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392043+0000", 2024-09-16T14:49:09.200 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:09.200 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.200 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.200 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.200 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392045+0000", 2024-09-16T14:49:09.200 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.283e-06 2024-09-16T14:49:09.200 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.201 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.201 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.201 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392050+0000", 2024-09-16T14:49:09.201 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1099999999999996e-06 2024-09-16T14:49:09.201 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.201 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.201 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.201 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392052+0000", 2024-09-16T14:49:09.201 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4660000000000002e-06 2024-09-16T14:49:09.201 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.201 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.201 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.201 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395234+0000", 2024-09-16T14:49:09.201 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031823260000000001 2024-09-16T14:49:09.202 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.202 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.202 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.202 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395283+0000", 2024-09-16T14:49:09.202 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8347000000000003e-05 2024-09-16T14:49:09.202 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.202 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.202 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:09.202 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395364+0000", 2024-09-16T14:49:09.202 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0853999999999995e-05 2024-09-16T14:49:09.202 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.202 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.202 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:09.203 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396928+0000", 2024-09-16T14:49:09.203 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015642239999999999 2024-09-16T14:49:09.203 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.203 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.203 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:09.203 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400202+0000", 2024-09-16T14:49:09.203 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032747359999999999 2024-09-16T14:49:09.203 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.203 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.203 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.203 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400227+0000", 2024-09-16T14:49:09.203 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4079000000000001e-05 2024-09-16T14:49:09.203 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.203 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.204 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.204 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400248+0000", 2024-09-16T14:49:09.204 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1444e-05 2024-09-16T14:49:09.204 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.204 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.204 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.204 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.204 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.204 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:161 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:09.204 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:49:09.204 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 45.790334164999997, 2024-09-16T14:49:09.204 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082675720000000008, 2024-09-16T14:49:09.204 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.204 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.205 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.205 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:09.205 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:09.205 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 161 2024-09-16T14:49:09.205 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.205 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.205 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.205 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.205 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:49:09.205 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.205 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.205 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.205 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.205 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:49:09.206 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.206 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.206 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.206 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.206 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392055+0000", 2024-09-16T14:49:09.206 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:09.206 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.206 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.206 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.206 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392122+0000", 2024-09-16T14:49:09.206 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7698000000000001e-05 2024-09-16T14:49:09.206 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.206 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.207 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.207 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392136+0000", 2024-09-16T14:49:09.207 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3464e-05 2024-09-16T14:49:09.207 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.207 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.207 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.207 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392140+0000", 2024-09-16T14:49:09.207 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0790000000000002e-06 2024-09-16T14:49:09.207 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.207 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.207 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.207 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395579+0000", 2024-09-16T14:49:09.207 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034393219999999999 2024-09-16T14:49:09.208 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.208 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.208 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.208 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395624+0000", 2024-09-16T14:49:09.208 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4946e-05 2024-09-16T14:49:09.208 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.208 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.208 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:09.208 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.395692+0000", 2024-09-16T14:49:09.208 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7843999999999996e-05 2024-09-16T14:49:09.208 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.208 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.209 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:09.209 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397834+0000", 2024-09-16T14:49:09.209 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021421529999999999 2024-09-16T14:49:09.209 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.209 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.209 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:09.209 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400281+0000", 2024-09-16T14:49:09.209 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002447238 2024-09-16T14:49:09.209 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.209 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.209 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.209 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400305+0000", 2024-09-16T14:49:09.209 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3428999999999999e-05 2024-09-16T14:49:09.209 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.210 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.210 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.210 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400322+0000", 2024-09-16T14:49:09.210 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7632e-05 2024-09-16T14:49:09.210 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.210 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.210 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.210 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.210 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.210 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:09.210 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:49:09.210 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 45.790244885999996, 2024-09-16T14:49:09.210 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.008260814, 2024-09-16T14:49:09.211 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.211 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.211 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.211 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:09.211 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:09.211 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 162 2024-09-16T14:49:09.211 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.211 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.211 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.211 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.211 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:49:09.211 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.211 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.212 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.212 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.212 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:49:09.212 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.212 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.212 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.212 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.212 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392144+0000", 2024-09-16T14:49:09.212 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:09.212 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.212 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.212 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.212 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392146+0000", 2024-09-16T14:49:09.212 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.526e-06 2024-09-16T14:49:09.213 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.213 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.213 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.213 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392151+0000", 2024-09-16T14:49:09.213 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2980000000000002e-06 2024-09-16T14:49:09.213 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.213 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.213 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.213 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392153+0000", 2024-09-16T14:49:09.213 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5409999999999999e-06 2024-09-16T14:49:09.213 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.213 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.213 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.214 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396103+0000", 2024-09-16T14:49:09.214 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0039497559999999996 2024-09-16T14:49:09.214 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.214 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.214 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.214 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396188+0000", 2024-09-16T14:49:09.214 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5038999999999997e-05 2024-09-16T14:49:09.214 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.214 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.214 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:09.214 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396315+0000", 2024-09-16T14:49:09.214 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000126895 2024-09-16T14:49:09.214 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.215 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.215 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:09.215 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397839+0000", 2024-09-16T14:49:09.215 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0015246859999999999 2024-09-16T14:49:09.215 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.215 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.215 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:09.215 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400356+0000", 2024-09-16T14:49:09.215 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002516524 2024-09-16T14:49:09.215 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.215 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.215 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.215 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400379+0000", 2024-09-16T14:49:09.216 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2793e-05 2024-09-16T14:49:09.216 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.216 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.216 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.216 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400405+0000", 2024-09-16T14:49:09.216 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6140999999999999e-05 2024-09-16T14:49:09.216 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.216 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.216 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.216 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.216 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.216 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:09.216 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:49:09.217 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 45.790233495000003, 2024-09-16T14:49:09.217 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083183530000000006, 2024-09-16T14:49:09.217 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.217 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.217 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.217 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:09.217 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:09.217 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 163 2024-09-16T14:49:09.217 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.217 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.217 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.217 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.217 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:49:09.217 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.218 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.218 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.218 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.218 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392156+0000", 2024-09-16T14:49:09.218 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.218 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.218 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.218 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.218 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392155+0000", 2024-09-16T14:49:09.218 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:09.218 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.218 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.218 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392159+0000", 2024-09-16T14:49:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0459999999999999e-06 2024-09-16T14:49:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392162+0000", 2024-09-16T14:49:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0520000000000002e-06 2024-09-16T14:49:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392165+0000", 2024-09-16T14:49:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.424e-06 2024-09-16T14:49:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.219 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.220 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.220 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396578+0000", 2024-09-16T14:49:09.220 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044136139999999997 2024-09-16T14:49:09.220 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.220 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.220 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.220 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396625+0000", 2024-09-16T14:49:09.220 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6486000000000002e-05 2024-09-16T14:49:09.220 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.220 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.220 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:09.220 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396694+0000", 2024-09-16T14:49:09.220 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9049999999999998e-05 2024-09-16T14:49:09.220 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.221 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.221 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:09.221 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397843+0000", 2024-09-16T14:49:09.221 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011485740000000001 2024-09-16T14:49:09.221 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.221 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.221 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:09.221 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400438+0000", 2024-09-16T14:49:09.221 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025950909999999999 2024-09-16T14:49:09.221 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.221 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.221 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.221 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400458+0000", 2024-09-16T14:49:09.222 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0248e-05 2024-09-16T14:49:09.222 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.222 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.222 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.222 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400474+0000", 2024-09-16T14:49:09.222 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5999999999999999e-05 2024-09-16T14:49:09.222 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.222 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.222 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.222 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.222 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.222 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:09.222 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:49:09.223 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 45.790221897999999, 2024-09-16T14:49:09.223 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083881519999999994, 2024-09-16T14:49:09.223 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.223 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.223 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.223 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:09.223 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:09.223 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 164 2024-09-16T14:49:09.223 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.223 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.223 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.223 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.223 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:49:09.223 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.224 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.224 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.224 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.224 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:49:09.224 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.224 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.224 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.224 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.224 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:49:09.224 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:09.224 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.224 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.224 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.225 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392212+0000", 2024-09-16T14:49:09.225 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5108999999999998e-05 2024-09-16T14:49:09.225 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.225 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.225 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.225 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392216+0000", 2024-09-16T14:49:09.225 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2549999999999999e-06 2024-09-16T14:49:09.225 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.225 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.225 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.225 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392220+0000", 2024-09-16T14:49:09.225 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3850000000000002e-06 2024-09-16T14:49:09.225 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.225 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.226 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.226 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396961+0000", 2024-09-16T14:49:09.226 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.004740976 2024-09-16T14:49:09.226 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.226 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.226 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.226 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397029+0000", 2024-09-16T14:49:09.226 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7889e-05 2024-09-16T14:49:09.226 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.226 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.226 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:09.226 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397131+0000", 2024-09-16T14:49:09.226 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000102315 2024-09-16T14:49:09.227 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.227 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.227 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:09.227 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400259+0000", 2024-09-16T14:49:09.227 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031285229999999998 2024-09-16T14:49:09.227 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.227 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.227 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:09.227 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400501+0000", 2024-09-16T14:49:09.227 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00024211999999999999 2024-09-16T14:49:09.227 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.227 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.227 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.228 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400530+0000", 2024-09-16T14:49:09.228 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8935999999999999e-05 2024-09-16T14:49:09.228 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.228 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.228 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.228 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400555+0000", 2024-09-16T14:49:09.228 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4861000000000001e-05 2024-09-16T14:49:09.228 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.228 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.228 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.228 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.228 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.228 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:09.228 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:49:09.229 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 45.790166133, 2024-09-16T14:49:09.229 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010593673, 2024-09-16T14:49:09.229 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.229 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.229 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.229 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:09.229 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:09.229 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 165 2024-09-16T14:49:09.229 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.229 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.229 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.229 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.229 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:49:09.230 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.230 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.230 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.230 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.230 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:49:09.230 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.230 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.230 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.230 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.230 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:49:09.230 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:09.230 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.230 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.230 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.231 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392226+0000", 2024-09-16T14:49:09.231 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2789999999999999e-06 2024-09-16T14:49:09.231 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.231 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.231 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.231 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392234+0000", 2024-09-16T14:49:09.231 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2369999999999992e-06 2024-09-16T14:49:09.231 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.231 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.231 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.231 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392237+0000", 2024-09-16T14:49:09.231 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8619999999999999e-06 2024-09-16T14:49:09.231 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.231 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397503+0000", 2024-09-16T14:49:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0052656220000000002 2024-09-16T14:49:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397558+0000", 2024-09-16T14:49:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5386999999999997e-05 2024-09-16T14:49:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397628+0000", 2024-09-16T14:49:09.232 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0119999999999999e-05 2024-09-16T14:49:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402533+0000", 2024-09-16T14:49:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049046200000000002 2024-09-16T14:49:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402778+0000", 2024-09-16T14:49:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000245348 2024-09-16T14:49:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.233 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402800+0000", 2024-09-16T14:49:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2277e-05 2024-09-16T14:49:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402817+0000", 2024-09-16T14:49:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6265000000000001e-05 2024-09-16T14:49:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:309 3.ds0 3:b9c0d236:::smithi04040151-19:head [append 0~737280] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:09.234 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:49:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.242974997000001, 2024-09-16T14:49:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01003656, 2024-09-16T14:49:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 309 2024-09-16T14:49:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:49:09.235 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.236 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.236 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.236 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.236 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:49:09.236 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.236 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.236 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.236 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.236 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939413+0000", 2024-09-16T14:49:09.236 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:09.236 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.236 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.236 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.236 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942226+0000", 2024-09-16T14:49:09.237 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0028130450000000001 2024-09-16T14:49:09.237 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.237 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.237 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.237 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942238+0000", 2024-09-16T14:49:09.237 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2639e-05 2024-09-16T14:49:09.237 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.237 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.237 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.237 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942245+0000", 2024-09-16T14:49:09.237 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3840000000000002e-06 2024-09-16T14:49:09.237 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.237 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.237 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.238 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942288+0000", 2024-09-16T14:49:09.238 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3186000000000003e-05 2024-09-16T14:49:09.238 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.238 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.238 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.238 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942368+0000", 2024-09-16T14:49:09.238 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0162999999999998e-05 2024-09-16T14:49:09.238 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.238 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.238 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:09.238 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943660+0000", 2024-09-16T14:49:09.238 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001291698 2024-09-16T14:49:09.238 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.238 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.239 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:09.239 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.947440+0000", 2024-09-16T14:49:09.239 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003780079 2024-09-16T14:49:09.239 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.239 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.239 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.239 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.949391+0000", 2024-09-16T14:49:09.239 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0019512990000000001 2024-09-16T14:49:09.239 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.239 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.239 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.239 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.949451+0000", 2024-09-16T14:49:09.239 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9404000000000002e-05 2024-09-16T14:49:09.239 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.240 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.240 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.240 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.240 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.240 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:312 3.ds0 3:b9c0d236:::smithi04040151-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:09.240 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:49:09.240 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.238995482, 2024-09-16T14:49:09.240 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084967020000000001, 2024-09-16T14:49:09.240 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.240 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:09.240 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.240 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:09.240 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:09.240 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 312 2024-09-16T14:49:09.241 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.241 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.241 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.241 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.241 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:49:09.241 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.241 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.241 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.241 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.241 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:49:09.241 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.241 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.241 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.242 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.242 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943393+0000", 2024-09-16T14:49:09.242 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:09.242 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.242 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.242 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.242 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943396+0000", 2024-09-16T14:49:09.242 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6079999999999998e-06 2024-09-16T14:49:09.242 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.242 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.242 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.242 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943402+0000", 2024-09-16T14:49:09.242 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6100000000000002e-06 2024-09-16T14:49:09.242 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.243 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.243 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.243 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943413+0000", 2024-09-16T14:49:09.243 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1178e-05 2024-09-16T14:49:09.243 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.243 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.243 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.243 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.946818+0000", 2024-09-16T14:49:09.243 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034043049999999998 2024-09-16T14:49:09.243 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.243 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.243 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:09.243 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.946922+0000", 2024-09-16T14:49:09.243 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000104076 2024-09-16T14:49:09.244 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.244 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.244 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.244 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.950275+0000", 2024-09-16T14:49:09.244 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0033530220000000002 2024-09-16T14:49:09.244 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.244 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.244 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.244 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.950357+0000", 2024-09-16T14:49:09.244 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2087000000000002e-05 2024-09-16T14:49:09.244 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.244 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.244 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.951890+0000", 2024-09-16T14:49:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001533333 2024-09-16T14:49:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:390 3.ds0 3:b855c7ac:::smithi04040151-47:head [append 0~0] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:53.864520+0000", 2024-09-16T14:49:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 15.317869286000001, 2024-09-16T14:49:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089883559999999994, 2024-09-16T14:49:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.245 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.246 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:09.246 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:09.246 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 390 2024-09-16T14:49:09.246 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.246 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.246 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.246 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.246 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864520+0000", 2024-09-16T14:49:09.246 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.246 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.246 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.246 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.246 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864520+0000", 2024-09-16T14:49:09.247 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.247 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.247 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.247 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.247 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864519+0000", 2024-09-16T14:49:09.247 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:09.247 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.247 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.247 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.247 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:09.247 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0545e-05 2024-09-16T14:49:09.247 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.247 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.247 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.248 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864536+0000", 2024-09-16T14:49:09.248 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8069999999999996e-06 2024-09-16T14:49:09.248 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.248 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.248 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.248 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864541+0000", 2024-09-16T14:49:09.248 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0900000000000004e-06 2024-09-16T14:49:09.248 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.248 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.248 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.248 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.866852+0000", 2024-09-16T14:49:09.248 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0023115399999999999 2024-09-16T14:49:09.248 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.249 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.249 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.249 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.866946+0000", 2024-09-16T14:49:09.249 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3311000000000002e-05 2024-09-16T14:49:09.249 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.249 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.249 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:09.249 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.867413+0000", 2024-09-16T14:49:09.249 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00046786499999999998 2024-09-16T14:49:09.249 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.249 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.249 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:09.249 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.868091+0000", 2024-09-16T14:49:09.249 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00067759500000000002 2024-09-16T14:49:09.250 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.250 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.250 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.250 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.873468+0000", 2024-09-16T14:49:09.250 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053766930000000001 2024-09-16T14:49:09.250 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.250 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.250 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.250 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.873508+0000", 2024-09-16T14:49:09.250 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0314000000000001e-05 2024-09-16T14:49:09.250 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.250 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:459 3.ds0 3:b9c0d236:::smithi04040151-19:head [create,setxattr bk]W^gTaniZ\\TV]dqj_UT\\QgY_\\pYYUOcYdSe`m_ (1859) in=40b,setxattr XSPcap`qmQ\\ZXYl^_bQYNltYZfOtfbRjqmVk_mOa (419) in=40b,setxattr l_Qd^aXhnapg`O][bWecsVdjlWkdYc^tOnjZh]^m (1577) in=40b,setxattr facegtqpaQroppsVbsrZelUak]RbamP`srOTdms[ (350) in=40b,setxattr QllprTts[liem]moe]Pi`nNtcQko (1402) in=28b,setxattr sjm^Un[ahPriW\\TNkaYS\\NnPZfZ\\Zmq[rQ^TWYZm (1892) in=40b,setxattr kRdb__OpRdYV^sYmYmYdhes^Vbf]QaV`SVZkhRcY (209) in=40b,setxattr b[QTnoQc[_Yc^RmdY[ (1402) in=18b,setxattr ^iZkhfc`oO (1402) in=10b,setxattr d\\QUOkoScOhonabPS_YeNlheOOeOal]aQ^ZW]lc` (2805) in=40b,setxattr SioYcliigTgheX\\jnVqipcelfYhbZRjQRet]mQQU (1365) in=40b,setxattr jWcgaRWbr\\ZQ_XXr_eenPa]sXtbaPlmeshkgjOsm (1192) in=40b,setxattr WmVtOY (1402) in=6b,setxattr ]tk_t\\gnSfccZfYfnWnaqP (1402) in=22b,setxattr dS_dYomnsYPtQarmqoZQkO]Ofk_SfgjgitphTZPg (18524) in=40b,setxattr g_a^Zpt`YVahZiOW]V[jQ\\QccqSTo_ma[\\c\\[aft (454) in=40b,setxattr qmmdmgOR__hN\\UTjpR_gbY^s_pOVSdbTob`k (1402) in=36b,setxattr cW`PPOYrbfXV^hmaWhVNb`qVSXqjcdej`pY][]l^ (665) in=40b,setxattr jjo^VgcaQffrnVNPlnq_iTmcrlnTOenhaZV[_SZf (453) in=40b,setxattr n^nqj]bnVcl\\njtUkprb\\laSh^Rbmam_Z_NkeXqj (595) in=40b] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.993572+0000", 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.18881728, 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010337796999999999, 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 459 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.251 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.252 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.252 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.252 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993572+0000", 2024-09-16T14:49:09.252 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.252 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.252 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.252 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.252 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993572+0000", 2024-09-16T14:49:09.252 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.252 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.252 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.252 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.252 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993569+0000", 2024-09-16T14:49:09.252 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:09.253 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.253 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.253 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.253 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993625+0000", 2024-09-16T14:49:09.253 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5105000000000002e-05 2024-09-16T14:49:09.253 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.253 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.253 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.253 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993647+0000", 2024-09-16T14:49:09.253 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1962000000000001e-05 2024-09-16T14:49:09.253 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.253 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.253 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.253 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993656+0000", 2024-09-16T14:49:09.254 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3179999999999996e-06 2024-09-16T14:49:09.254 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.254 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.254 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.254 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993717+0000", 2024-09-16T14:49:09.254 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.0738e-05 2024-09-16T14:49:09.254 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.254 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.254 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.254 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993841+0000", 2024-09-16T14:49:09.254 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000124537 2024-09-16T14:49:09.254 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.254 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.255 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:09.255 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.994891+0000", 2024-09-16T14:49:09.255 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0010499719999999999 2024-09-16T14:49:09.255 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.255 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.255 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:09.255 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.999326+0000", 2024-09-16T14:49:09.255 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044351240000000004 2024-09-16T14:49:09.255 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.255 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.255 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.255 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003847+0000", 2024-09-16T14:49:09.255 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045207629999999997 2024-09-16T14:49:09.255 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.256 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.256 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.256 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003910+0000", 2024-09-16T14:49:09.256 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2527999999999995e-05 2024-09-16T14:49:09.256 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.256 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.256 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.256 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.256 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.256 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.5s4 51/27 ECSubWrite(tid=60, reqid=client.4500.0:467, at_version=51'19, trim_to=0'0, roll_forward_to=51'16))", 2024-09-16T14:49:09.256 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.997576+0000", 2024-09-16T14:49:09.256 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.184813155000001, 2024-09-16T14:49:09.256 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083578019999999992, 2024-09-16T14:49:09.257 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.257 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:09.257 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.257 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.257 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.257 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997576+0000", 2024-09-16T14:49:09.257 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.257 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.257 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.257 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.257 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997576+0000", 2024-09-16T14:49:09.257 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.257 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.257 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.258 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.258 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997575+0000", 2024-09-16T14:49:09.258 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:09.258 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.258 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.258 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.258 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997593+0000", 2024-09-16T14:49:09.258 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7645000000000001e-05 2024-09-16T14:49:09.258 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.258 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.258 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.258 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997608+0000", 2024-09-16T14:49:09.258 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5315000000000002e-05 2024-09-16T14:49:09.258 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.259 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.259 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.259 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997616+0000", 2024-09-16T14:49:09.259 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.6120000000000001e-06 2024-09-16T14:49:09.259 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.259 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.259 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.259 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997651+0000", 2024-09-16T14:49:09.259 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5256999999999998e-05 2024-09-16T14:49:09.259 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.259 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.259 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:09.259 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997660+0000", 2024-09-16T14:49:09.260 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7320000000000004e-06 2024-09-16T14:49:09.260 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.260 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.260 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:09.260 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.005913+0000", 2024-09-16T14:49:09.260 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082530840000000008 2024-09-16T14:49:09.260 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.260 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.260 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.260 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.005934+0000", 2024-09-16T14:49:09.260 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0812999999999999e-05 2024-09-16T14:49:09.260 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.260 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.260 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.261 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.261 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.261 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:535 3.fs0 3:ff6b916c:::smithi04040151-25:head [append 0~786432] snapc 1e=[1e,1c,1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:09.261 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:09.261 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 1.2510271399999999, 2024-09-16T14:49:09.261 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013876462000000001, 2024-09-16T14:49:09.261 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.261 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.261 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.261 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:09.261 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:09.261 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 535 2024-09-16T14:49:09.261 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.262 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.262 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.262 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.262 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:09.262 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.262 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.262 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.262 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.262 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:09.262 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.262 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.262 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.262 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.262 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931360+0000", 2024-09-16T14:49:09.263 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:09.263 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.263 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.263 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.263 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933014+0000", 2024-09-16T14:49:09.263 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001653715 2024-09-16T14:49:09.263 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.263 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.263 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.263 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933028+0000", 2024-09-16T14:49:09.263 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4239999999999999e-05 2024-09-16T14:49:09.263 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.263 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.263 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.264 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933036+0000", 2024-09-16T14:49:09.264 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8019999999999992e-06 2024-09-16T14:49:09.264 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.264 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.264 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.264 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933084+0000", 2024-09-16T14:49:09.264 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7663000000000001e-05 2024-09-16T14:49:09.264 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.264 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.264 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.264 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933184+0000", 2024-09-16T14:49:09.264 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000100549 2024-09-16T14:49:09.264 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.264 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.265 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:09.265 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.934324+0000", 2024-09-16T14:49:09.265 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011399229999999999 2024-09-16T14:49:09.265 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.265 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.265 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:09.265 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941715+0000", 2024-09-16T14:49:09.265 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.007391023 2024-09-16T14:49:09.265 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.265 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.265 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.265 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.945187+0000", 2024-09-16T14:49:09.265 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034717709999999998 2024-09-16T14:49:09.265 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.266 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.266 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.266 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.945238+0000", 2024-09-16T14:49:09.266 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.1459999999999999e-05 2024-09-16T14:49:09.266 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.266 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.266 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.266 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.266 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.266 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.5s4 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:09.266 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.933072+0000", 2024-09-16T14:49:09.266 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 1.249317147, 2024-09-16T14:49:09.266 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0081398449999999997, 2024-09-16T14:49:09.266 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.267 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:09.267 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.267 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.267 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.267 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933072+0000", 2024-09-16T14:49:09.267 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.267 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.267 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.267 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.267 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933072+0000", 2024-09-16T14:49:09.267 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.267 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.267 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.267 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.268 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933072+0000", 2024-09-16T14:49:09.268 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:09.268 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.268 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.268 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.268 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933097+0000", 2024-09-16T14:49:09.268 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5718e-05 2024-09-16T14:49:09.268 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.268 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.268 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.268 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933107+0000", 2024-09-16T14:49:09.268 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2599999999999994e-06 2024-09-16T14:49:09.268 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.269 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.269 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.269 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933113+0000", 2024-09-16T14:49:09.269 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8639999999999998e-06 2024-09-16T14:49:09.269 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.269 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.269 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.269 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933151+0000", 2024-09-16T14:49:09.269 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.7252000000000002e-05 2024-09-16T14:49:09.269 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.269 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.269 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:09.269 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933160+0000", 2024-09-16T14:49:09.269 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.38e-06 2024-09-16T14:49:09.270 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.270 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.270 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:09.270 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941196+0000", 2024-09-16T14:49:09.270 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0080359190000000007 2024-09-16T14:49:09.270 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.270 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.270 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.270 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941212+0000", 2024-09-16T14:49:09.270 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5692000000000002e-05 2024-09-16T14:49:09.270 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.270 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.270 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.270 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.271 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.271 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:535 3.fs0 3:ff6b916c:::smithi04040151-25:head [append 0~786432] snapc 1e=[1e,1c,1b] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:09.271 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.936346+0000", 2024-09-16T14:49:09.271 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 1.2460428059999999, 2024-09-16T14:49:09.271 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0088507770000000006, 2024-09-16T14:49:09.271 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.271 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "waiting for ondisk", 2024-09-16T14:49:09.271 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.271 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:09.271 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:09.271 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 535 2024-09-16T14:49:09.271 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.271 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.271 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.936346+0000", 2024-09-16T14:49:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.936346+0000", 2024-09-16T14:49:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.936345+0000", 2024-09-16T14:49:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.273 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.273 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.273 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937664+0000", 2024-09-16T14:49:09.273 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0013194000000000001 2024-09-16T14:49:09.273 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.273 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.273 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.273 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937679+0000", 2024-09-16T14:49:09.273 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4588e-05 2024-09-16T14:49:09.273 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.273 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.276 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.276 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937685+0000", 2024-09-16T14:49:09.276 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2389999999999999e-06 2024-09-16T14:49:09.276 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.276 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.276 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.276 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937772+0000", 2024-09-16T14:49:09.277 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6396000000000004e-05 2024-09-16T14:49:09.277 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.277 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.277 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for ondisk", 2024-09-16T14:49:09.277 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937865+0000", 2024-09-16T14:49:09.277 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3029999999999995e-05 2024-09-16T14:49:09.277 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.277 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.277 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.277 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.945197+0000", 2024-09-16T14:49:09.277 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0073322810000000004 2024-09-16T14:49:09.277 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.277 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.278 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.278 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.278 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.278 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:536 3.fs0 3:ff6b916c:::smithi04040151-25:head [append 0~491520] snapc 1e=[1e,1c,1b] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:09.278 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.937695+0000", 2024-09-16T14:49:09.278 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 1.244694497, 2024-09-16T14:49:09.278 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0087034420000000005, 2024-09-16T14:49:09.278 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.278 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.278 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.278 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:09.278 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:09.278 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 536 2024-09-16T14:49:09.278 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.279 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.279 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.279 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.279 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937695+0000", 2024-09-16T14:49:09.279 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.279 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.279 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.279 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.279 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937695+0000", 2024-09-16T14:49:09.279 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.279 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.279 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.279 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.279 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937693+0000", 2024-09-16T14:49:09.280 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:09.280 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.280 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.280 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.280 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938195+0000", 2024-09-16T14:49:09.280 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00050157700000000001 2024-09-16T14:49:09.280 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.280 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.280 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.280 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938204+0000", 2024-09-16T14:49:09.280 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7870000000000007e-06 2024-09-16T14:49:09.280 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.280 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.281 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.281 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938208+0000", 2024-09-16T14:49:09.281 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.318e-06 2024-09-16T14:49:09.281 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.281 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.281 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.281 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938247+0000", 2024-09-16T14:49:09.281 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9214000000000001e-05 2024-09-16T14:49:09.281 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.281 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.281 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.281 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938351+0000", 2024-09-16T14:49:09.281 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000103931 2024-09-16T14:49:09.281 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.282 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.282 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:09.282 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939194+0000", 2024-09-16T14:49:09.282 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00084270199999999997 2024-09-16T14:49:09.282 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.282 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.282 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:09.282 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941717+0000", 2024-09-16T14:49:09.282 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025227100000000001 2024-09-16T14:49:09.282 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.282 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.282 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.282 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.946316+0000", 2024-09-16T14:49:09.282 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045990639999999999 2024-09-16T14:49:09.283 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.283 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.283 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.283 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.946398+0000", 2024-09-16T14:49:09.283 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2218999999999993e-05 2024-09-16T14:49:09.283 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.283 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.283 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.283 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.283 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.283 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:537 3.fs0 3:ff6b916c:::smithi04040151-25:head [append 0~278528] snapc 1e=[1e,1c,1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:09.283 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.938214+0000", 2024-09-16T14:49:09.283 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 1.2441753470000001, 2024-09-16T14:49:09.283 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089476360000000001, 2024-09-16T14:49:09.284 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.284 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.284 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.284 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:09.284 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:09.284 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 537 2024-09-16T14:49:09.284 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.284 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.284 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.284 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.284 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938214+0000", 2024-09-16T14:49:09.284 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.284 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.285 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.285 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.285 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938214+0000", 2024-09-16T14:49:09.285 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.285 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.285 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.285 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.285 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938213+0000", 2024-09-16T14:49:09.285 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:09.285 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.285 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.285 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.285 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938431+0000", 2024-09-16T14:49:09.286 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00021819100000000001 2024-09-16T14:49:09.286 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.286 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.286 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.286 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938440+0000", 2024-09-16T14:49:09.286 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0210000000000005e-06 2024-09-16T14:49:09.286 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.286 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.286 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.286 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938447+0000", 2024-09-16T14:49:09.286 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.5200000000000003e-06 2024-09-16T14:49:09.286 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.286 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.286 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.287 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939886+0000", 2024-09-16T14:49:09.287 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0014394950000000001 2024-09-16T14:49:09.287 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.287 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.287 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.287 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939979+0000", 2024-09-16T14:49:09.287 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2519e-05 2024-09-16T14:49:09.287 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.287 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.287 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:09.287 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.940972+0000", 2024-09-16T14:49:09.287 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00099328099999999994 2024-09-16T14:49:09.287 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.288 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.288 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:09.288 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.944530+0000", 2024-09-16T14:49:09.288 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003557639 2024-09-16T14:49:09.288 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.288 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.288 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.288 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947108+0000", 2024-09-16T14:49:09.288 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025781100000000002 2024-09-16T14:49:09.288 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.288 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.288 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.288 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947161+0000", 2024-09-16T14:49:09.288 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3510000000000001e-05 2024-09-16T14:49:09.289 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.289 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.289 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.289 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.289 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.289 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:538 3.fs0 3:ff6b916c:::smithi04040151-25:head [setxattr _header (57) in=7b] snapc 1e=[1e,1c,1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:09.289 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.938463+0000", 2024-09-16T14:49:09.289 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 1.2439262870000001, 2024-09-16T14:49:09.289 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010050811999999999, 2024-09-16T14:49:09.289 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.289 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:09.289 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.289 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:09.290 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:09.290 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 538 2024-09-16T14:49:09.290 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.290 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.290 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.290 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.290 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938463+0000", 2024-09-16T14:49:09.290 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.290 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.290 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.290 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.290 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938463+0000", 2024-09-16T14:49:09.290 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.290 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.291 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.291 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.291 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938462+0000", 2024-09-16T14:49:09.291 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:09.291 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.291 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.291 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.291 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938466+0000", 2024-09-16T14:49:09.291 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.7979999999999999e-06 2024-09-16T14:49:09.291 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.291 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.291 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.291 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938473+0000", 2024-09-16T14:49:09.291 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2910000000000002e-06 2024-09-16T14:49:09.292 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.292 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.292 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.292 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938478+0000", 2024-09-16T14:49:09.292 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3079999999999998e-06 2024-09-16T14:49:09.292 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.292 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.292 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.292 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941806+0000", 2024-09-16T14:49:09.292 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0033274899999999998 2024-09-16T14:49:09.292 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.292 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.292 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.293 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941906+0000", 2024-09-16T14:49:09.293 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010026600000000001 2024-09-16T14:49:09.293 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.293 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.293 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:09.293 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942348+0000", 2024-09-16T14:49:09.293 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00044172800000000001 2024-09-16T14:49:09.293 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.293 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.293 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:09.293 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.944534+0000", 2024-09-16T14:49:09.293 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021862439999999999 2024-09-16T14:49:09.293 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.293 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.294 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:09.294 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.948427+0000", 2024-09-16T14:49:09.294 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0038925370000000002 2024-09-16T14:49:09.294 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.294 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.294 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.294 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.948514+0000", 2024-09-16T14:49:09.294 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6868999999999998e-05 2024-09-16T14:49:09.294 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.294 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.294 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.294 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.294 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.294 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:539 3.fs0 3:ff6b916c:::smithi04040151-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:09.295 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:09.295 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 1.2439056079999999, 2024-09-16T14:49:09.295 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01551078, 2024-09-16T14:49:09.295 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:09.295 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:09.295 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:09.295 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:09.295 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:09.295 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 539 2024-09-16T14:49:09.295 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.295 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:09.295 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.295 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:09.296 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:09.296 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.296 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.296 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.296 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:09.296 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:09.296 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:09.296 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.296 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.296 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:09.296 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:09.296 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:09.296 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.297 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.297 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:09.297 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938486+0000", 2024-09-16T14:49:09.297 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7779999999999999e-06 2024-09-16T14:49:09.297 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.297 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.297 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:09.297 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938492+0000", 2024-09-16T14:49:09.297 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8120000000000002e-06 2024-09-16T14:49:09.297 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.297 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.297 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:09.297 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938495+0000", 2024-09-16T14:49:09.297 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.948e-06 2024-09-16T14:49:09.298 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.298 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.298 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.298 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942566+0000", 2024-09-16T14:49:09.298 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0040700839999999999 2024-09-16T14:49:09.298 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.298 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.298 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:09.298 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942659+0000", 2024-09-16T14:49:09.298 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3887000000000004e-05 2024-09-16T14:49:09.298 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.298 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.298 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:09.298 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.948875+0000", 2024-09-16T14:49:09.299 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0062154630000000001 2024-09-16T14:49:09.299 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.299 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.299 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:09.299 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.948968+0000", 2024-09-16T14:49:09.299 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2758999999999997e-05 2024-09-16T14:49:09.299 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:09.299 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:09.299 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:09.300 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.953994+0000", 2024-09-16T14:49:09.300 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050265580000000004 2024-09-16T14:49:09.300 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.300 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.300 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.300 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:09.300 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:09.300 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:09.302 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:49:09.302 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:09.300+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:09.304 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:09.305 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:09.321 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:09.321 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:09.337 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:09.337 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:09.353 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:09.353 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:09.369 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:09.369 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:09.385 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:09.385 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:09.402 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:09.402 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:09.403 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:09.401+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:09.416 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:09.416 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:09.416 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:09.416 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:09.418 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:09.418 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:09.425 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:49:09.435 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:09.435 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:09.451 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:09.451 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:09.468 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:09.468 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:09.484 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:09.503 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:09.502+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:09.570 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:09.570 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:09.578 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:49:09.604 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:09.603+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:09.681 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:09.681 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:09.688 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:49:09.732 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:09.731+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:09.791 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:09.791 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:09.799 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:49:09.806 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:09.805+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:09.851 INFO:tasks.rados.rados.0.smithi040.stdout:229: done (6 left) 2024-09-16T14:49:09.851 INFO:tasks.rados.rados.0.smithi040.stdout:230: snap_remove snap 14 2024-09-16T14:49:09.907 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:09.906+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:09.949 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:09.949 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:09.954 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:49:09.958 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:49:10.008 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:10.007+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:10.061 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:10.061 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:10.069 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:49:10.113 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:10.112+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:10.171 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:10.171 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:10.180 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:49:10.210 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:10.208+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:10.311 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:10.310+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:10.323 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:10.323 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:10.332 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:49:10.412 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:10.411+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:10.433 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:10.434 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:10.442 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:49:10.513 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:10.511+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:10.543 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:10.543 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:10.552 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:49:10.614 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:10.612+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:10.653 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:10.653 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:10.661 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:49:10.715 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:10.714+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:10.763 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:10.763 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:10.772 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:49:10.817 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:10.815+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:10.853 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is set 2024-09-16T14:49:10.854 INFO:tasks.rados.rados.0.smithi040.stdout:230: done (6 left) 2024-09-16T14:49:10.854 INFO:tasks.rados.rados.0.smithi040.stdout:231: rollback oid 50 current snap is 17 2024-09-16T14:49:10.855 INFO:tasks.rados.rados.0.smithi040.stdout:231: done (6 left) 2024-09-16T14:49:10.855 INFO:tasks.rados.rados.0.smithi040.stdout:232: read oid 12 snap -1 2024-09-16T14:49:10.855 INFO:tasks.rados.rados.0.smithi040.stdout:232: expect (ObjNum 11 snap 0 seq_num 11) 2024-09-16T14:49:10.855 INFO:tasks.rados.rados.0.smithi040.stdout:233: rmattr oid 44 current snap is 17 2024-09-16T14:49:10.855 INFO:tasks.rados.rados.0.smithi040.stdout:233: done (7 left) 2024-09-16T14:49:10.855 INFO:tasks.rados.rados.0.smithi040.stdout:234: read oid 44 snap -1 2024-09-16T14:49:10.855 INFO:tasks.rados.rados.0.smithi040.stdout:234: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-16T14:49:10.855 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 15 current snap is 17 2024-09-16T14:49:10.855 INFO:tasks.rados.rados.0.smithi040.stdout:235: seq_num 94 ranges {196608=425984,622592=786432,1409024=376832} 2024-09-16T14:49:10.859 INFO:tasks.rados.rados.0.smithi040.stdout:235: writing smithi04040151-15 from 196608 to 622592 tid 1 2024-09-16T14:49:10.864 INFO:tasks.rados.rados.0.smithi040.stdout:235: writing smithi04040151-15 from 622592 to 1409024 tid 2 2024-09-16T14:49:10.867 INFO:tasks.rados.rados.0.smithi040.stdout:235: writing smithi04040151-15 from 1409024 to 1785856 tid 3 2024-09-16T14:49:10.868 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 31 current snap is 17 2024-09-16T14:49:10.868 INFO:tasks.rados.rados.0.smithi040.stdout:236: seq_num 95 ranges {1146880=442368,1589248=540672,2129920=622592} 2024-09-16T14:49:10.877 INFO:tasks.rados.rados.0.smithi040.stdout:236: writing smithi04040151-31 from 1146880 to 1589248 tid 1 2024-09-16T14:49:10.880 INFO:tasks.rados.rados.0.smithi040.stdout:236: writing smithi04040151-31 from 1589248 to 2129920 tid 2 2024-09-16T14:49:10.885 INFO:tasks.rados.rados.0.smithi040.stdout:236: writing smithi04040151-31 from 2129920 to 2752512 tid 3 2024-09-16T14:49:10.885 INFO:tasks.rados.rados.0.smithi040.stdout:237: setattr oid 4 current snap is 17 2024-09-16T14:49:10.885 INFO:tasks.rados.rados.0.smithi040.stdout:220: finishing write tid 1 to smithi04040151-35 2024-09-16T14:49:10.885 INFO:tasks.rados.rados.0.smithi040.stdout:220: finishing write tid 2 to smithi04040151-35 2024-09-16T14:49:10.885 INFO:tasks.rados.rados.0.smithi040.stdout:220: finishing write tid 3 to smithi04040151-35 2024-09-16T14:49:10.885 INFO:tasks.rados.rados.0.smithi040.stdout:220: finishing write tid 4 to smithi04040151-35 2024-09-16T14:49:10.885 INFO:tasks.rados.rados.0.smithi040.stdout:221: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-16T14:49:10.899 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:10.900 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:10.908 INFO:tasks.rados.rados.0.smithi040.stdout:220: finishing write tid 5 to smithi04040151-35 2024-09-16T14:49:10.908 INFO:tasks.rados.rados.0.smithi040.stdout:220: oid 35 updating version 0 to 29 2024-09-16T14:49:10.908 INFO:tasks.rados.rados.0.smithi040.stdout:220: oid 35 version 29 is already newer than 28 2024-09-16T14:49:10.909 INFO:tasks.rados.rados.0.smithi040.stdout:220: oid 35 version 29 is already newer than 26 2024-09-16T14:49:10.909 INFO:tasks.rados.rados.0.smithi040.stdout:220: oid 35 version 29 is already newer than 27 2024-09-16T14:49:10.909 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 35 v 29 (ObjNum 91 snap 17 seq_num 91) dirty exists 2024-09-16T14:49:10.909 INFO:tasks.rados.rados.0.smithi040.stdout:220: left oid 35 (ObjNum 91 snap 17 seq_num 91) 2024-09-16T14:49:10.909 INFO:tasks.rados.rados.0.smithi040.stdout:222: finishing copy_from racing read to smithi04040151-27 2024-09-16T14:49:10.909 INFO:tasks.rados.rados.0.smithi040.stdout:222: finishing copy_from to smithi04040151-27 2024-09-16T14:49:10.909 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 27 v 12 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-16T14:49:10.909 INFO:tasks.rados.rados.0.smithi040.stdout:224: expect (ObjNum 33 snap 0 seq_num 33) 2024-09-16T14:49:10.912 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:49:10.917 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:10.915+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:10.917 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 6] 2024-09-16T14:49:10.917 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:49:10.917 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-09-16T14:49:10.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2024-09-16T14:49:10.926 INFO:tasks.rados.rados.0.smithi040.stdout:226: finishing write tid 1 to smithi04040151-25 2024-09-16T14:49:10.926 INFO:tasks.rados.rados.0.smithi040.stdout:226: finishing write tid 2 to smithi04040151-25 2024-09-16T14:49:10.926 INFO:tasks.rados.rados.0.smithi040.stdout:226: finishing write tid 3 to smithi04040151-25 2024-09-16T14:49:10.927 INFO:tasks.rados.rados.0.smithi040.stdout:226: finishing write tid 4 to smithi04040151-25 2024-09-16T14:49:10.927 INFO:tasks.rados.rados.0.smithi040.stdout:226: finishing write tid 5 to smithi04040151-25 2024-09-16T14:49:10.927 INFO:tasks.rados.rados.0.smithi040.stdout:226: oid 25 updating version 0 to 22 2024-09-16T14:49:10.927 INFO:tasks.rados.rados.0.smithi040.stdout:226: oid 25 updating version 22 to 24 2024-09-16T14:49:10.927 INFO:tasks.rados.rados.0.smithi040.stdout:226: oid 25 updating version 24 to 25 2024-09-16T14:49:10.927 INFO:tasks.rados.rados.0.smithi040.stdout:226: oid 25 version 25 is already newer than 23 2024-09-16T14:49:10.927 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 25 v 25 (ObjNum 92 snap 17 seq_num 92) dirty exists 2024-09-16T14:49:10.927 INFO:tasks.rados.rados.0.smithi040.stdout:226: left oid 25 (ObjNum 92 snap 17 seq_num 92) 2024-09-16T14:49:10.927 INFO:tasks.rados.rados.0.smithi040.stdout:228: finishing write tid 1 to smithi04040151-40 2024-09-16T14:49:10.927 INFO:tasks.rados.rados.0.smithi040.stdout:228: finishing write tid 2 to smithi04040151-40 2024-09-16T14:49:10.927 INFO:tasks.rados.rados.0.smithi040.stdout:228: finishing write tid 3 to smithi04040151-40 2024-09-16T14:49:10.928 INFO:tasks.rados.rados.0.smithi040.stdout:228: finishing write tid 4 to smithi04040151-40 2024-09-16T14:49:10.928 INFO:tasks.rados.rados.0.smithi040.stdout:228: finishing write tid 5 to smithi04040151-40 2024-09-16T14:49:10.928 INFO:tasks.rados.rados.0.smithi040.stdout:228: oid 40 updating version 0 to 86 2024-09-16T14:49:10.928 INFO:tasks.rados.rados.0.smithi040.stdout:228: oid 40 version 86 is already newer than 84 2024-09-16T14:49:10.928 INFO:tasks.rados.rados.0.smithi040.stdout:228: oid 40 version 86 is already newer than 83 2024-09-16T14:49:10.928 INFO:tasks.rados.rados.0.smithi040.stdout:228: oid 40 version 86 is already newer than 85 2024-09-16T14:49:10.928 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 40 v 86 (ObjNum 93 snap 17 seq_num 93) dirty exists 2024-09-16T14:49:10.928 INFO:tasks.rados.rados.0.smithi040.stdout:228: left oid 40 (ObjNum 93 snap 17 seq_num 93) 2024-09-16T14:49:10.928 INFO:tasks.rados.rados.0.smithi040.stdout:232: expect (ObjNum 11 snap 0 seq_num 11) 2024-09-16T14:49:10.932 INFO:tasks.rados.rados.0.smithi040.stdout:234: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-16T14:49:10.954 INFO:tasks.rados.rados.0.smithi040.stdout:235: finishing write tid 1 to smithi04040151-15 2024-09-16T14:49:10.954 INFO:tasks.rados.rados.0.smithi040.stdout:235: finishing write tid 2 to smithi04040151-15 2024-09-16T14:49:10.954 INFO:tasks.rados.rados.0.smithi040.stdout:235: finishing write tid 3 to smithi04040151-15 2024-09-16T14:49:10.954 INFO:tasks.rados.rados.0.smithi040.stdout:235: finishing write tid 4 to smithi04040151-15 2024-09-16T14:49:10.954 INFO:tasks.rados.rados.0.smithi040.stdout:235: finishing write tid 5 to smithi04040151-15 2024-09-16T14:49:10.954 INFO:tasks.rados.rados.0.smithi040.stdout:235: oid 15 updating version 0 to 43 2024-09-16T14:49:10.954 INFO:tasks.rados.rados.0.smithi040.stdout:235: oid 15 version 43 is already newer than 41 2024-09-16T14:49:10.955 INFO:tasks.rados.rados.0.smithi040.stdout:235: oid 15 version 43 is already newer than 42 2024-09-16T14:49:10.955 INFO:tasks.rados.rados.0.smithi040.stdout:235: oid 15 version 43 is already newer than 40 2024-09-16T14:49:10.955 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 15 v 43 (ObjNum 94 snap 17 seq_num 94) dirty exists 2024-09-16T14:49:10.955 INFO:tasks.rados.rados.0.smithi040.stdout:235: left oid 15 (ObjNum 94 snap 17 seq_num 94) 2024-09-16T14:49:10.955 INFO:tasks.rados.rados.0.smithi040.stdout:236: finishing write tid 1 to smithi04040151-31 2024-09-16T14:49:10.955 INFO:tasks.rados.rados.0.smithi040.stdout:236: finishing write tid 2 to smithi04040151-31 2024-09-16T14:49:10.955 INFO:tasks.rados.rados.0.smithi040.stdout:236: finishing write tid 3 to smithi04040151-31 2024-09-16T14:49:10.955 INFO:tasks.rados.rados.0.smithi040.stdout:236: finishing write tid 4 to smithi04040151-31 2024-09-16T14:49:10.955 INFO:tasks.rados.rados.0.smithi040.stdout:236: finishing write tid 5 to smithi04040151-31 2024-09-16T14:49:10.955 INFO:tasks.rados.rados.0.smithi040.stdout:236: oid 31 updating version 0 to 84 2024-09-16T14:49:10.955 INFO:tasks.rados.rados.0.smithi040.stdout:236: oid 31 updating version 84 to 85 2024-09-16T14:49:10.955 INFO:tasks.rados.rados.0.smithi040.stdout:236: oid 31 version 85 is already newer than 82 2024-09-16T14:49:10.956 INFO:tasks.rados.rados.0.smithi040.stdout:236: oid 31 version 85 is already newer than 83 2024-09-16T14:49:10.956 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 31 v 85 (ObjNum 95 snap 17 seq_num 95) dirty exists 2024-09-16T14:49:10.956 INFO:tasks.rados.rados.0.smithi040.stdout:236: left oid 31 (ObjNum 95 snap 17 seq_num 95) 2024-09-16T14:49:10.956 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 4 v 24 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-16T14:49:10.956 INFO:tasks.rados.rados.0.smithi040.stdout:220: done (10 left) 2024-09-16T14:49:10.956 INFO:tasks.rados.rados.0.smithi040.stdout:221: done (9 left) 2024-09-16T14:49:10.956 INFO:tasks.rados.rados.0.smithi040.stdout:222: done (8 left) 2024-09-16T14:49:10.956 INFO:tasks.rados.rados.0.smithi040.stdout:224: done (7 left) 2024-09-16T14:49:10.956 INFO:tasks.rados.rados.0.smithi040.stdout:226: done (6 left) 2024-09-16T14:49:10.956 INFO:tasks.rados.rados.0.smithi040.stdout:228: done (5 left) 2024-09-16T14:49:10.956 INFO:tasks.rados.rados.0.smithi040.stdout:232: done (4 left) 2024-09-16T14:49:10.957 INFO:tasks.rados.rados.0.smithi040.stdout:234: done (3 left) 2024-09-16T14:49:10.957 INFO:tasks.rados.rados.0.smithi040.stdout:235: done (2 left) 2024-09-16T14:49:10.957 INFO:tasks.rados.rados.0.smithi040.stdout:236: done (1 left) 2024-09-16T14:49:10.957 INFO:tasks.rados.rados.0.smithi040.stdout:237: done (0 left) 2024-09-16T14:49:10.957 INFO:tasks.rados.rados.0.smithi040.stdout:238: read oid 18 snap -1 2024-09-16T14:49:10.957 INFO:tasks.rados.rados.0.smithi040.stdout:238: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-16T14:49:10.957 INFO:tasks.rados.rados.0.smithi040.stdout:239: copy_from oid 31 from oid 16 current snap is 17 2024-09-16T14:49:10.957 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 2 current snap is 17 2024-09-16T14:49:10.957 INFO:tasks.rados.rados.0.smithi040.stdout:240: seq_num 96 ranges {32768=458752,491520=688128,1179648=475136} 2024-09-16T14:49:10.958 INFO:tasks.rados.rados.0.smithi040.stdout:240: writing smithi04040151-2 from 32768 to 491520 tid 1 2024-09-16T14:49:10.960 INFO:tasks.rados.rados.0.smithi040.stdout:240: writing smithi04040151-2 from 491520 to 1179648 tid 2 2024-09-16T14:49:10.964 INFO:tasks.rados.rados.0.smithi040.stdout:240: writing smithi04040151-2 from 1179648 to 1654784 tid 3 2024-09-16T14:49:10.964 INFO:tasks.rados.rados.0.smithi040.stdout:241: read oid 38 snap -1 2024-09-16T14:49:10.964 INFO:tasks.rados.rados.0.smithi040.stdout:241: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-16T14:49:10.964 INFO:tasks.rados.rados.0.smithi040.stdout:242: snap_create 2024-09-16T14:49:10.964 INFO:tasks.rados.rados.0.smithi040.stdout:239: finishing copy_from to smithi04040151-31 2024-09-16T14:49:10.964 INFO:tasks.rados.rados.0.smithi040.stdout:239: got expected ENOENT (src dne) 2024-09-16T14:49:10.964 INFO:tasks.rados.rados.0.smithi040.stdout:239: finishing copy_from racing read to smithi04040151-31 2024-09-16T14:49:10.964 INFO:tasks.rados.rados.0.smithi040.stdout:238: expect (ObjNum 9 snap 0 seq_num 9) 2024-09-16T14:49:10.969 INFO:tasks.rados.rados.0.smithi040.stdout:240: finishing write tid 1 to smithi04040151-2 2024-09-16T14:49:10.969 INFO:tasks.rados.rados.0.smithi040.stdout:240: finishing write tid 2 to smithi04040151-2 2024-09-16T14:49:10.969 INFO:tasks.rados.rados.0.smithi040.stdout:240: finishing write tid 3 to smithi04040151-2 2024-09-16T14:49:10.970 INFO:tasks.rados.rados.0.smithi040.stdout:240: finishing write tid 4 to smithi04040151-2 2024-09-16T14:49:10.970 INFO:tasks.rados.rados.0.smithi040.stdout:241: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-16T14:49:10.986 INFO:tasks.rados.rados.0.smithi040.stdout:240: finishing write tid 5 to smithi04040151-2 2024-09-16T14:49:10.986 INFO:tasks.rados.rados.0.smithi040.stdout:240: oid 2 updating version 0 to 47 2024-09-16T14:49:10.986 INFO:tasks.rados.rados.0.smithi040.stdout:240: oid 2 version 47 is already newer than 44 2024-09-16T14:49:10.986 INFO:tasks.rados.rados.0.smithi040.stdout:240: oid 2 version 47 is already newer than 46 2024-09-16T14:49:10.986 INFO:tasks.rados.rados.0.smithi040.stdout:240: oid 2 version 47 is already newer than 45 2024-09-16T14:49:10.986 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 2 v 47 (ObjNum 96 snap 17 seq_num 96) dirty exists 2024-09-16T14:49:10.986 INFO:tasks.rados.rados.0.smithi040.stdout:240: left oid 2 (ObjNum 96 snap 17 seq_num 96) 2024-09-16T14:49:11.018 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:11.016+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:11.057 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:11.057 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:11.066 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:49:11.119 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:11.117+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:11.208 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:11.208 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:11.216 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:49:11.219 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:11.218+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:11.221 INFO:tasks.ceph.mon.a.smithi040.stderr:2024-09-16T14:49:11.220+0000 7f544d3bb640 -1 mon.a@0(leader).osd e64 definitely_dead 0 2024-09-16T14:49:11.321 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:11.319+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:11.323 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:11.323 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:11.332 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:49:11.421 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:11.420+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:11.479 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:11.479 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:11.487 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:49:11.495 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:49:11.522 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:11.521+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:11.595 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:11.595 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:11.604 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:49:11.648 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:11.647+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:11.654 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:11.655 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:11.671 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:11.672 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:11.688 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:11.688 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:11.704 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:11.704 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:11.712 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:11.712 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:11.712 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:11.712 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:11.720 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:11.720 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:11.721 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:49:11.737 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:11.737 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:11.738 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:11.736+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:11.755 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:11.756 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:11.772 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:11.772 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:11.789 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:11.789 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:11.806 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:11.806 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:11.824 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:11.824 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:11.825 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:11.822+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:11.844 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:11.845 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:11.845 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:11.845 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:11.845 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:11.846 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:11.846 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:11.854 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:49:11.941 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:11.940+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:11.947 INFO:tasks.rados.rados.0.smithi040.stdout:238: done (4 left) 2024-09-16T14:49:11.947 INFO:tasks.rados.rados.0.smithi040.stdout:239: done (3 left) 2024-09-16T14:49:11.947 INFO:tasks.rados.rados.0.smithi040.stdout:240: done (2 left) 2024-09-16T14:49:11.947 INFO:tasks.rados.rados.0.smithi040.stdout:241: done (1 left) 2024-09-16T14:49:11.947 INFO:tasks.rados.rados.0.smithi040.stdout:242: done (0 left) 2024-09-16T14:49:11.947 INFO:tasks.rados.rados.0.smithi040.stdout:243: read oid 39 snap -1 2024-09-16T14:49:11.947 INFO:tasks.rados.rados.0.smithi040.stdout:243: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-16T14:49:11.948 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 31 current snap is 18 2024-09-16T14:49:11.948 INFO:tasks.rados.rados.0.smithi040.stdout:244: seq_num 97 ranges {2752512=475136,3227648=442368,3670016=442368,4112384=278528} 2024-09-16T14:49:11.949 INFO:teuthology.orchestra.run.smithi040.stderr:marked in osd.6. 2024-09-16T14:49:11.967 INFO:tasks.rados.rados.0.smithi040.stdout:244: writing smithi04040151-31 from 2752512 to 3227648 tid 1 2024-09-16T14:49:11.970 INFO:tasks.rados.rados.0.smithi040.stdout:244: writing smithi04040151-31 from 3227648 to 3670016 tid 2 2024-09-16T14:49:11.972 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-09-16T14:49:11.974 INFO:tasks.rados.rados.0.smithi040.stdout:244: writing smithi04040151-31 from 3670016 to 4112384 tid 3 2024-09-16T14:49:11.976 INFO:tasks.rados.rados.0.smithi040.stdout:244: writing smithi04040151-31 from 4112384 to 4390912 tid 4 2024-09-16T14:49:11.976 INFO:tasks.rados.rados.0.smithi040.stdout:245: rollback oid 3 current snap is 18 2024-09-16T14:49:11.976 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 3 to 17 2024-09-16T14:49:11.976 INFO:tasks.rados.rados.0.smithi040.stdout:244: finishing write tid 1 to smithi04040151-31 2024-09-16T14:49:11.976 INFO:tasks.rados.rados.0.smithi040.stdout:246: copy_from oid 16 from oid 47 current snap is 18 2024-09-16T14:49:11.976 INFO:tasks.rados.rados.0.smithi040.stdout:247: setattr oid 48 current snap is 18 2024-09-16T14:49:11.976 INFO:tasks.rados.rados.0.smithi040.stdout:244: finishing write tid 2 to smithi04040151-31 2024-09-16T14:49:11.978 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 33 current snap is 18 2024-09-16T14:49:11.978 INFO:tasks.rados.rados.0.smithi040.stdout:248: seq_num 98 ranges {0=491520,491520=589824,1081344=557056,1638400=16384} 2024-09-16T14:49:11.981 INFO:tasks.rados.rados.0.smithi040.stdout:248: writing smithi04040151-33 from 0 to 491520 tid 1 2024-09-16T14:49:11.985 INFO:tasks.rados.rados.0.smithi040.stdout:248: writing smithi04040151-33 from 491520 to 1081344 tid 2 2024-09-16T14:49:11.990 INFO:tasks.rados.rados.0.smithi040.stdout:248: writing smithi04040151-33 from 1081344 to 1638400 tid 3 2024-09-16T14:49:11.990 INFO:tasks.rados.rados.0.smithi040.stdout:248: writing smithi04040151-33 from 1638400 to 1654784 tid 4 2024-09-16T14:49:11.990 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 1 current snap is 18 2024-09-16T14:49:11.990 INFO:tasks.rados.rados.0.smithi040.stdout:244: finishing write tid 3 to smithi04040151-31 2024-09-16T14:49:11.990 INFO:tasks.rados.rados.0.smithi040.stdout:244: finishing write tid 4 to smithi04040151-31 2024-09-16T14:49:11.990 INFO:tasks.rados.rados.0.smithi040.stdout:249: seq_num 99 ranges {32768=507904,540672=606208,1146880=557056} 2024-09-16T14:49:11.993 INFO:tasks.rados.rados.0.smithi040.stdout:249: writing smithi04040151-1 from 32768 to 540672 tid 1 2024-09-16T14:49:11.998 INFO:tasks.rados.rados.0.smithi040.stdout:249: writing smithi04040151-1 from 540672 to 1146880 tid 2 2024-09-16T14:49:12.002 INFO:tasks.rados.rados.0.smithi040.stdout:249: writing smithi04040151-1 from 1146880 to 1703936 tid 3 2024-09-16T14:49:12.003 INFO:tasks.rados.rados.0.smithi040.stdout:250: snap_create 2024-09-16T14:49:12.003 INFO:tasks.rados.rados.0.smithi040.stdout:244: finishing write tid 5 to smithi04040151-31 2024-09-16T14:49:12.003 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 48 v 26 (ObjNum 84 snap 13 seq_num 84) dirty exists 2024-09-16T14:49:12.003 INFO:tasks.rados.rados.0.smithi040.stdout:244: finishing write tid 6 to smithi04040151-31 2024-09-16T14:49:12.003 INFO:tasks.rados.rados.0.smithi040.stdout:244: oid 31 updating version 0 to 91 2024-09-16T14:49:12.003 INFO:tasks.rados.rados.0.smithi040.stdout:244: oid 31 version 91 is already newer than 90 2024-09-16T14:49:12.003 INFO:tasks.rados.rados.0.smithi040.stdout:244: oid 31 version 91 is already newer than 89 2024-09-16T14:49:12.004 INFO:tasks.rados.rados.0.smithi040.stdout:244: oid 31 version 91 is already newer than 88 2024-09-16T14:49:12.004 INFO:tasks.rados.rados.0.smithi040.stdout:244: oid 31 version 91 is already newer than 87 2024-09-16T14:49:12.004 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 31 v 91 (ObjNum 97 snap 18 seq_num 97) dirty exists 2024-09-16T14:49:12.004 INFO:tasks.rados.rados.0.smithi040.stdout:244: left oid 31 (ObjNum 97 snap 18 seq_num 97) 2024-09-16T14:49:12.026 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:12.026 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:12.026 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:12.026 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:12.027 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.027 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.2s5 37/27 ECSubWrite(tid=123, reqid=osd.3.0:123, at_version=37'38, trim_to=0'0, roll_forward_to=37'34))", 2024-09-16T14:49:12.027 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:12.027 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.087798652, 2024-09-16T14:49:12.027 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0094962529999999996, 2024-09-16T14:49:12.027 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.027 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.027 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.027 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.027 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.027 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:12.027 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.028 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.028 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.028 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:12.028 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.028 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.028 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.028 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935902+0000", 2024-09-16T14:49:12.028 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:12.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.028 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.028 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.029 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935938+0000", 2024-09-16T14:49:12.029 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5731000000000001e-05 2024-09-16T14:49:12.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.029 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.029 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.029 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935949+0000", 2024-09-16T14:49:12.029 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1229999999999999e-05 2024-09-16T14:49:12.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.029 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.029 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.029 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935957+0000", 2024-09-16T14:49:12.029 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.473e-06 2024-09-16T14:49:12.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.029 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.030 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.030 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.936913+0000", 2024-09-16T14:49:12.030 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00095599900000000002 2024-09-16T14:49:12.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.030 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.030 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.936923+0000", 2024-09-16T14:49:12.030 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4059999999999994e-06 2024-09-16T14:49:12.030 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.030 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.031 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945371+0000", 2024-09-16T14:49:12.031 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084484279999999991 2024-09-16T14:49:12.031 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.031 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.031 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.031 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945399+0000", 2024-09-16T14:49:12.031 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7716e-05 2024-09-16T14:49:12.031 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.031 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.031 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.031 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.031 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.032 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s1 37/27 ECSubWrite(tid=108, reqid=osd.11.0:108, at_version=37'33, trim_to=0'0, roll_forward_to=35'31))", 2024-09-16T14:49:12.032 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.940384+0000", 2024-09-16T14:49:12.032 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.083317808, 2024-09-16T14:49:12.032 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085853549999999994, 2024-09-16T14:49:12.032 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.032 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.032 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.032 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.032 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.032 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940384+0000", 2024-09-16T14:49:12.032 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.032 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.032 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.033 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.033 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940384+0000", 2024-09-16T14:49:12.033 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.033 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.033 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.033 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.033 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940383+0000", 2024-09-16T14:49:12.033 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.033 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.033 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.033 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.033 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940406+0000", 2024-09-16T14:49:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3003999999999999e-05 2024-09-16T14:49:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940421+0000", 2024-09-16T14:49:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5285e-05 2024-09-16T14:49:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940430+0000", 2024-09-16T14:49:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.439e-06 2024-09-16T14:49:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.034 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.035 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.035 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940486+0000", 2024-09-16T14:49:12.035 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6266e-05 2024-09-16T14:49:12.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.035 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.035 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.035 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940497+0000", 2024-09-16T14:49:12.035 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1231e-05 2024-09-16T14:49:12.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.035 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.035 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.035 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.948934+0000", 2024-09-16T14:49:12.035 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084368450000000001 2024-09-16T14:49:12.035 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.036 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.036 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.036 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.948969+0000", 2024-09-16T14:49:12.036 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.4993999999999997e-05 2024-09-16T14:49:12.036 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.036 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.036 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.036 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.036 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.036 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.0s3 37/27 ECSubWrite(tid=109, reqid=osd.11.0:109, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:12.036 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:12.036 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.083252032000001, 2024-09-16T14:49:12.036 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090738399999999997, 2024-09-16T14:49:12.036 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.037 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.037 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.037 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.037 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.037 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:12.037 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.037 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.037 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.037 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.037 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:12.037 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.037 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.037 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.038 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.038 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:12.038 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.038 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.038 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.038 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.038 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940467+0000", 2024-09-16T14:49:12.038 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8113000000000001e-05 2024-09-16T14:49:12.038 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.038 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.038 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.038 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940475+0000", 2024-09-16T14:49:12.038 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8499999999999994e-06 2024-09-16T14:49:12.038 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940481+0000", 2024-09-16T14:49:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0979999999999999e-06 2024-09-16T14:49:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940569+0000", 2024-09-16T14:49:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8395999999999999e-05 2024-09-16T14:49:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940577+0000", 2024-09-16T14:49:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7500000000000003e-06 2024-09-16T14:49:12.040 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.040 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.040 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.040 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949500+0000", 2024-09-16T14:49:12.040 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089230319999999991 2024-09-16T14:49:12.040 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.040 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.040 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.040 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949523+0000", 2024-09-16T14:49:12.040 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3082e-05 2024-09-16T14:49:12.040 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.040 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.040 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.041 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.041 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.041 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s1 37/27 ECSubWrite(tid=110, reqid=osd.11.0:110, at_version=37'35, trim_to=0'0, roll_forward_to=35'31))", 2024-09-16T14:49:12.041 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.941325+0000", 2024-09-16T14:49:12.041 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.082375978999998, 2024-09-16T14:49:12.041 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.008218075, 2024-09-16T14:49:12.041 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.041 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.041 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.041 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.041 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.041 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.941325+0000", 2024-09-16T14:49:12.041 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.041 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.042 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.042 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.042 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.941325+0000", 2024-09-16T14:49:12.042 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.042 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.042 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.042 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.042 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.941325+0000", 2024-09-16T14:49:12.042 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.042 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.042 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.042 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.042 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.941345+0000", 2024-09-16T14:49:12.043 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0352000000000001e-05 2024-09-16T14:49:12.043 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.043 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.043 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.043 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.941362+0000", 2024-09-16T14:49:12.043 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6820000000000002e-05 2024-09-16T14:49:12.043 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.043 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.043 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.043 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.941373+0000", 2024-09-16T14:49:12.043 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0679000000000001e-05 2024-09-16T14:49:12.043 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.043 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.043 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.044 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942069+0000", 2024-09-16T14:49:12.044 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00069643199999999998 2024-09-16T14:49:12.044 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.044 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.044 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.044 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.942087+0000", 2024-09-16T14:49:12.044 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8187999999999999e-05 2024-09-16T14:49:12.044 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.044 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.044 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.044 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949528+0000", 2024-09-16T14:49:12.044 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074406259999999997 2024-09-16T14:49:12.044 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.045 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.045 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.045 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949544+0000", 2024-09-16T14:49:12.045 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5690000000000001e-05 2024-09-16T14:49:12.045 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.045 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.045 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.045 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.045 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.045 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:302 3.4s0 3:255172f3:::smithi04040151-3:head [create,setxattr qPd\\P_iff[nbYodrbXe[dQ_okmVWqe[RceaqQRNm (630) in=40b,setxattr NWYlZi[dPXnppNo]`fatPqfSaTlsQVetOmrQm^ap (138) in=40b,setxattr iVfcaeRT]TQppeYbq_`lRobpUsaVcWXphplasbgY (1507) in=40b,setxattr YeaeOf[jt`g\\k^]VfXTjOONb`jT[WYUgdiglVjYl (8841) in=40b,setxattr [iqZhSWl_sclXkajVYTXlOX]eeeWkPi\\dOYYt_Wb (1928) in=40b,setxattr \\[gaSPWYO`NQXpTccfbk`SZPaU`lbpTk]a] (1262) in=35b,setxattr ]\\]U^niYqQmjcgO[dsO`YpRZeRV]`X]bU[X^_Qis (1086) in=40b,setxattr VafomYY]aa`a^pbWebSRaithSn_dQjQYq]aTarm\\ (1402) in=40b,setxattr acj^VZhfaaikXrXq]_haYqkl[^_atWUTaloiVj`a (105) in=40b,setxattr VNUlaeZOglfodan^Ycb`dqVeZhSTmO_rabViYVch (70) in=40b,setxattr Qka\\PoUoghRc\\Qcnl`kVbj[rs\\hSdVXmlNqha`ta (594) in=40b,setxattr qsSQhb_apjdTlikdpj\\maaVdS[mVfsh_Vqi[ogWo (771) in=40b,setxattr Tm]oi_Rf_V`pSY[VhYSXSQN^bqjtgmm^fUo^f\\ZV (1087) in=40b,setxattr \\dTq (1262) in=4b,setxattr nettWfThbXj]cUpsYrTjZ^qk^\\[ZoZ[qb]Sf[ZbN (1684) in=40b,setxattr Sa\\hrdN^YombnT\\torZ`N]aRqicS_lP^XgP`e`Oc (1963) in=40b,setxattr dWhoTgYPSTh]jfWhPTs^iVdYbtfhi_WjOdes]NXh (1051) in=40b,setxattr Qbp_QrhXQf`O]blcqeqgRchO]RrhbshoV\\cW[_es (10033) in=40b,setxattr WYbQqgdRjmdbeoaP]U]n]PTWOYkP^lWOanNTnaOr (1998) in=40b,setxattr Rfbdg`[^ZYrN]cl][Ypr^tpXmb_mbh[gf\\idocW] (10804) in=40b] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:12.045 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:12.045 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 27.091286255, 2024-09-16T14:49:12.046 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.009468509, 2024-09-16T14:49:12.046 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.046 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:12.046 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:12.046 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:12.046 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:12.046 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 302 2024-09-16T14:49:12.046 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.046 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.046 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.046 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.046 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:12.046 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.047 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.047 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.047 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:12.047 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.047 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.047 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.047 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932413+0000", 2024-09-16T14:49:12.047 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:12.047 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.047 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.047 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.047 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932459+0000", 2024-09-16T14:49:12.048 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6414e-05 2024-09-16T14:49:12.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.048 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.048 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.048 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932491+0000", 2024-09-16T14:49:12.048 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1656000000000003e-05 2024-09-16T14:49:12.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.048 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.048 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.048 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932499+0000", 2024-09-16T14:49:12.048 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4940000000000003e-06 2024-09-16T14:49:12.048 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.048 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.049 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.049 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932557+0000", 2024-09-16T14:49:12.049 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8044000000000002e-05 2024-09-16T14:49:12.049 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.049 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.049 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:12.049 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932704+0000", 2024-09-16T14:49:12.049 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00014615 2024-09-16T14:49:12.049 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.049 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.049 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.049 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.934021+0000", 2024-09-16T14:49:12.049 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001317052 2024-09-16T14:49:12.049 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.050 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.050 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.050 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941512+0000", 2024-09-16T14:49:12.050 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074917789999999996 2024-09-16T14:49:12.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.050 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.050 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:12.050 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941804+0000", 2024-09-16T14:49:12.050 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029206099999999998 2024-09-16T14:49:12.050 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.050 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.050 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.050 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941884+0000", 2024-09-16T14:49:12.050 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9231000000000001e-05 2024-09-16T14:49:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es2 38/27 ECSubWrite(tid=113, reqid=client.4500.0:298, at_version=38'12, trim_to=0'0, roll_forward_to=37'11))", 2024-09-16T14:49:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.933568+0000", 2024-09-16T14:49:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 27.090133732000002, 2024-09-16T14:49:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089983270000000004, 2024-09-16T14:49:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.051 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.052 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933568+0000", 2024-09-16T14:49:12.052 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.052 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.052 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933568+0000", 2024-09-16T14:49:12.052 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.052 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.052 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933567+0000", 2024-09-16T14:49:12.052 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.052 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.052 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.053 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.053 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933833+0000", 2024-09-16T14:49:12.053 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00026602700000000001 2024-09-16T14:49:12.053 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.053 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.053 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.053 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933846+0000", 2024-09-16T14:49:12.053 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.223e-05 2024-09-16T14:49:12.053 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.053 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.053 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.053 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933854+0000", 2024-09-16T14:49:12.053 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5369999999999997e-06 2024-09-16T14:49:12.054 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.054 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.054 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.054 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933912+0000", 2024-09-16T14:49:12.054 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8103999999999998e-05 2024-09-16T14:49:12.054 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.054 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.054 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.054 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933922+0000", 2024-09-16T14:49:12.054 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.978e-06 2024-09-16T14:49:12.054 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.054 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.054 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.055 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.942550+0000", 2024-09-16T14:49:12.055 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0086275210000000008 2024-09-16T14:49:12.055 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.055 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.055 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.055 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.942566+0000", 2024-09-16T14:49:12.055 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6268000000000001e-05 2024-09-16T14:49:12.055 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.055 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.055 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.055 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.055 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.055 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:365 3.4s0 3:2b1464a3:::smithi04040151-28:head [copy-from ver 15] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:12.056 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:12.056 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.084432858, 2024-09-16T14:49:12.056 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.019409261000000001, 2024-09-16T14:49:12.056 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.056 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:12.056 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:12.056 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:12.056 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:12.056 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 365 2024-09-16T14:49:12.056 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.056 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.056 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.056 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.057 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:12.057 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.057 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.057 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.057 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.057 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:12.057 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.057 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.057 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.057 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.057 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939267+0000", 2024-09-16T14:49:12.057 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:12.058 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.058 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.058 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.058 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939278+0000", 2024-09-16T14:49:12.058 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.093e-05 2024-09-16T14:49:12.058 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.058 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.058 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.058 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939295+0000", 2024-09-16T14:49:12.058 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7161000000000001e-05 2024-09-16T14:49:12.058 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.058 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.058 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.058 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939304+0000", 2024-09-16T14:49:12.059 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.329e-06 2024-09-16T14:49:12.059 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.059 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.059 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.059 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939378+0000", 2024-09-16T14:49:12.059 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3456999999999998e-05 2024-09-16T14:49:12.059 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.059 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.059 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:12.059 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939387+0000", 2024-09-16T14:49:12.059 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1449999999999997e-06 2024-09-16T14:49:12.059 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.059 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.060 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.060 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.941885+0000", 2024-09-16T14:49:12.060 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0024978349999999999 2024-09-16T14:49:12.060 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.060 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.060 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:12.060 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942001+0000", 2024-09-16T14:49:12.060 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011674499999999999 2024-09-16T14:49:12.060 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.060 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.060 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.060 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.953426+0000", 2024-09-16T14:49:12.061 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011424808999999999 2024-09-16T14:49:12.061 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.061 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.061 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.061 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956855+0000", 2024-09-16T14:49:12.061 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0034292160000000001 2024-09-16T14:49:12.061 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.061 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.061 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:12.061 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.958596+0000", 2024-09-16T14:49:12.061 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001740969 2024-09-16T14:49:12.061 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.061 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.062 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.062 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.958678+0000", 2024-09-16T14:49:12.062 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1592000000000001e-05 2024-09-16T14:49:12.062 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.062 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.062 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.062 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.062 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:366 3.4s0 3:2b1464a3:::smithi04040151-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:12.062 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:12.062 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.084383288000001, 2024-09-16T14:49:12.062 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.017101175, 2024-09-16T14:49:12.063 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.063 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:12.063 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:12.063 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:12.063 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:12.063 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 366 2024-09-16T14:49:12.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.063 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.063 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.063 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.063 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:12.064 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.064 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.064 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.064 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:12.064 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.064 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.064 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.064 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:12.064 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.065 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.065 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.065 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939321+0000", 2024-09-16T14:49:12.065 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.49e-06 2024-09-16T14:49:12.065 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.065 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.065 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.065 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939331+0000", 2024-09-16T14:49:12.065 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.006e-05 2024-09-16T14:49:12.065 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.065 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.065 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.065 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939336+0000", 2024-09-16T14:49:12.066 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2959999999999998e-06 2024-09-16T14:49:12.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.066 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.066 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.066 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939454+0000", 2024-09-16T14:49:12.066 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000117726 2024-09-16T14:49:12.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.066 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.066 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:12.066 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939461+0000", 2024-09-16T14:49:12.066 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1940000000000001e-06 2024-09-16T14:49:12.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.066 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.066 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.067 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942256+0000", 2024-09-16T14:49:12.067 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0027949350000000001 2024-09-16T14:49:12.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.067 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.067 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:12.067 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942325+0000", 2024-09-16T14:49:12.067 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.9119e-05 2024-09-16T14:49:12.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.067 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.067 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.067 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956179+0000", 2024-09-16T14:49:12.067 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013853343000000001 2024-09-16T14:49:12.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.068 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.068 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:12.068 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956295+0000", 2024-09-16T14:49:12.068 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000115899 2024-09-16T14:49:12.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.068 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.068 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.068 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956419+0000", 2024-09-16T14:49:12.068 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012467399999999999 2024-09-16T14:49:12.068 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.068 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.068 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.068 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.069 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:391 3.4s0 3:20ea1aa7:::smithi04040151-17:head [append 0~0] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:12.069 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:12.069 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 18.159171241999999, 2024-09-16T14:49:12.069 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011589693999999999, 2024-09-16T14:49:12.069 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.069 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:12.069 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:12.069 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:12.069 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:12.069 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 391 2024-09-16T14:49:12.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.069 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.070 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.070 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.070 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:12.070 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.070 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.070 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.070 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:12.070 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.070 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.070 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.070 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864528+0000", 2024-09-16T14:49:12.070 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:12.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.071 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.071 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.071 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864540+0000", 2024-09-16T14:49:12.071 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2514000000000001e-05 2024-09-16T14:49:12.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.071 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.071 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.071 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864569+0000", 2024-09-16T14:49:12.071 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8716000000000001e-05 2024-09-16T14:49:12.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.071 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.071 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.072 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864580+0000", 2024-09-16T14:49:12.072 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1564e-05 2024-09-16T14:49:12.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.072 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.072 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.072 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864681+0000", 2024-09-16T14:49:12.072 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010063100000000001 2024-09-16T14:49:12.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.072 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.072 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:12.072 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864690+0000", 2024-09-16T14:49:12.072 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6300000000000004e-06 2024-09-16T14:49:12.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.073 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.073 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.073 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.869283+0000", 2024-09-16T14:49:12.073 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.004593186 2024-09-16T14:49:12.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.073 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.073 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:12.073 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.869365+0000", 2024-09-16T14:49:12.073 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.174e-05 2024-09-16T14:49:12.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.073 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.073 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.073 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870206+0000", 2024-09-16T14:49:12.074 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00084176100000000001 2024-09-16T14:49:12.074 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.074 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.074 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.074 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.873261+0000", 2024-09-16T14:49:12.074 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0030546549999999999 2024-09-16T14:49:12.074 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.074 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.074 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:12.074 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.876069+0000", 2024-09-16T14:49:12.074 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0028082210000000001 2024-09-16T14:49:12.074 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.074 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.074 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.075 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.876120+0000", 2024-09-16T14:49:12.075 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.0581999999999998e-05 2024-09-16T14:49:12.075 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.075 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.075 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.075 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.075 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:392 3.4s0 3:20ea1aa7:::smithi04040151-17:head [rollback c] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:12.075 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:12.075 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 18.159080662000001, 2024-09-16T14:49:12.075 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013845265000000001, 2024-09-16T14:49:12.075 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.075 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:12.076 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:12.076 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:12.076 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:12.076 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 392 2024-09-16T14:49:12.076 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.076 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.076 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.076 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.076 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:12.076 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.076 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.076 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.076 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.077 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:12.077 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.077 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.077 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.077 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864620+0000", 2024-09-16T14:49:12.077 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:12.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.077 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.077 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.077 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864626+0000", 2024-09-16T14:49:12.077 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6699999999999999e-06 2024-09-16T14:49:12.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.078 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.078 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.078 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864643+0000", 2024-09-16T14:49:12.078 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7404000000000001e-05 2024-09-16T14:49:12.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.078 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.078 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.078 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864651+0000", 2024-09-16T14:49:12.078 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0860000000000002e-06 2024-09-16T14:49:12.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.078 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.078 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.078 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864759+0000", 2024-09-16T14:49:12.078 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010751000000000001 2024-09-16T14:49:12.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.079 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.079 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:12.079 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864765+0000", 2024-09-16T14:49:12.079 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.737e-06 2024-09-16T14:49:12.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.079 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.079 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.079 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870832+0000", 2024-09-16T14:49:12.079 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060664930000000001 2024-09-16T14:49:12.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.079 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.079 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:12.079 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870972+0000", 2024-09-16T14:49:12.080 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00014033599999999999 2024-09-16T14:49:12.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.080 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.080 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.080 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.871637+0000", 2024-09-16T14:49:12.080 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00066502800000000002 2024-09-16T14:49:12.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.080 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.080 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.080 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.873263+0000", 2024-09-16T14:49:12.080 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0016256409999999999 2024-09-16T14:49:12.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.081 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.081 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:12.081 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.878403+0000", 2024-09-16T14:49:12.081 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0051401600000000004 2024-09-16T14:49:12.081 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.081 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.081 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.081 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.878466+0000", 2024-09-16T14:49:12.081 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.2964000000000002e-05 2024-09-16T14:49:12.081 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.081 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.081 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.081 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.081 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.082 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:393 3.4s0 3:20ea1aa7:::smithi04040151-17:head [append 0~0] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:12.082 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:12.082 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 18.159021068000001, 2024-09-16T14:49:12.082 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.016150277000000001, 2024-09-16T14:49:12.082 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.082 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:12.082 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:12.082 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:12.082 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:12.082 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 393 2024-09-16T14:49:12.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.082 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.082 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.084 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.084 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.084 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864688+0000", 2024-09-16T14:49:12.084 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5259999999999993e-06 2024-09-16T14:49:12.084 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.084 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.084 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.084 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864705+0000", 2024-09-16T14:49:12.084 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6368999999999999e-05 2024-09-16T14:49:12.084 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.084 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.084 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.084 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864712+0000", 2024-09-16T14:49:12.085 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4749999999999996e-06 2024-09-16T14:49:12.085 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.085 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.085 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.085 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864791+0000", 2024-09-16T14:49:12.085 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9159000000000006e-05 2024-09-16T14:49:12.085 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.085 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.085 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:12.085 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864797+0000", 2024-09-16T14:49:12.085 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5400000000000003e-06 2024-09-16T14:49:12.085 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.086 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.086 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.086 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872109+0000", 2024-09-16T14:49:12.086 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073124569999999996 2024-09-16T14:49:12.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.086 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.086 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:12.086 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872237+0000", 2024-09-16T14:49:12.086 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012735099999999999 2024-09-16T14:49:12.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.086 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.086 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.086 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872865+0000", 2024-09-16T14:49:12.086 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00062808000000000004 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.874081+0000", 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001216186 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.880752+0000", 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0066711339999999996 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.880831+0000", 2024-09-16T14:49:12.087 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8681000000000001e-05 2024-09-16T14:49:12.088 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.088 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.088 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.088 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.088 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.088 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s5 51/27 ECSubWrite(tid=55, reqid=osd.5.0:55, at_version=51'16, trim_to=0'0, roll_forward_to=49'13))", 2024-09-16T14:49:12.088 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.879919+0000", 2024-09-16T14:49:12.088 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.143782218, 2024-09-16T14:49:12.088 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084269440000000004, 2024-09-16T14:49:12.088 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.088 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.088 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.088 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.089 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.089 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879919+0000", 2024-09-16T14:49:12.089 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.089 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.089 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.089 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879919+0000", 2024-09-16T14:49:12.089 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.089 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.089 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.089 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879919+0000", 2024-09-16T14:49:12.089 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.090 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.090 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.090 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.090 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879941+0000", 2024-09-16T14:49:12.090 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2299000000000001e-05 2024-09-16T14:49:12.090 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.090 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.090 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.090 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879958+0000", 2024-09-16T14:49:12.090 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7115000000000001e-05 2024-09-16T14:49:12.090 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.090 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.090 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.091 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879966+0000", 2024-09-16T14:49:12.091 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8259999999999993e-06 2024-09-16T14:49:12.091 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.091 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.091 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.091 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880007+0000", 2024-09-16T14:49:12.091 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.1480000000000003e-05 2024-09-16T14:49:12.091 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.091 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.091 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.091 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880016+0000", 2024-09-16T14:49:12.091 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1390000000000001e-06 2024-09-16T14:49:12.092 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.092 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.092 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.888316+0000", 2024-09-16T14:49:12.092 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0082998299999999994 2024-09-16T14:49:12.092 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.092 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.092 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.888346+0000", 2024-09-16T14:49:12.092 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.989e-05 2024-09-16T14:49:12.092 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.092 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.092 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.093 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s1 51/27 ECSubWrite(tid=169, reqid=osd.11.0:169, at_version=51'52, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:12.093 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.880573+0000", 2024-09-16T14:49:12.093 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.143128299000001, 2024-09-16T14:49:12.093 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0082410190000000005, 2024-09-16T14:49:12.093 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.093 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.093 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.093 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.093 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880573+0000", 2024-09-16T14:49:12.093 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.094 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.094 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880573+0000", 2024-09-16T14:49:12.094 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.094 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.094 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880573+0000", 2024-09-16T14:49:12.094 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.094 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.095 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880598+0000", 2024-09-16T14:49:12.095 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5088000000000001e-05 2024-09-16T14:49:12.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.095 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.095 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.095 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880606+0000", 2024-09-16T14:49:12.095 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7919999999999993e-06 2024-09-16T14:49:12.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.095 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.095 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.095 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880612+0000", 2024-09-16T14:49:12.095 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.708e-06 2024-09-16T14:49:12.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.096 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.096 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880693+0000", 2024-09-16T14:49:12.096 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0295000000000003e-05 2024-09-16T14:49:12.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.096 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.096 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.880708+0000", 2024-09-16T14:49:12.096 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5628999999999999e-05 2024-09-16T14:49:12.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.096 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.096 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.888798+0000", 2024-09-16T14:49:12.096 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0080899079999999998 2024-09-16T14:49:12.097 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.097 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.097 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.097 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.888814+0000", 2024-09-16T14:49:12.097 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5889999999999999e-05 2024-09-16T14:49:12.097 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.097 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.097 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.097 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.097 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.097 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.9s5 51/27 ECSubWrite(tid=35, reqid=osd.8.0:35, at_version=51'21, trim_to=0'0, roll_forward_to=47'19))", 2024-09-16T14:49:12.097 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:12.097 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.134575815, 2024-09-16T14:49:12.098 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010449557999999999, 2024-09-16T14:49:12.098 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.098 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.098 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.098 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.098 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.098 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:12.098 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.098 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.098 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.098 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:12.098 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.099 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.099 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889125+0000", 2024-09-16T14:49:12.099 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.099 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.099 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889142+0000", 2024-09-16T14:49:12.099 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7186000000000001e-05 2024-09-16T14:49:12.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.099 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.100 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889152+0000", 2024-09-16T14:49:12.100 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7920000000000004e-06 2024-09-16T14:49:12.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.100 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.100 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889159+0000", 2024-09-16T14:49:12.100 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.0319999999999998e-06 2024-09-16T14:49:12.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.100 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.100 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889203+0000", 2024-09-16T14:49:12.100 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.4264000000000002e-05 2024-09-16T14:49:12.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.101 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.101 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889212+0000", 2024-09-16T14:49:12.101 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6940000000000001e-06 2024-09-16T14:49:12.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.101 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.101 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.899537+0000", 2024-09-16T14:49:12.101 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010325328 2024-09-16T14:49:12.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.101 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.101 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.899575+0000", 2024-09-16T14:49:12.101 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.7741999999999999e-05 2024-09-16T14:49:12.102 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.102 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.102 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.102 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.2s5 51/27 ECSubWrite(tid=175, reqid=client.4500.0:460, at_version=51'53, trim_to=0'0, roll_forward_to=51'51))", 2024-09-16T14:49:12.102 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.994224+0000", 2024-09-16T14:49:12.102 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.029476963, 2024-09-16T14:49:12.102 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085171090000000001, 2024-09-16T14:49:12.102 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.102 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.102 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.102 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.103 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994224+0000", 2024-09-16T14:49:12.103 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.103 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.103 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994224+0000", 2024-09-16T14:49:12.103 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.103 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.103 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994223+0000", 2024-09-16T14:49:12.103 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:12.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.104 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.104 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.104 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994266+0000", 2024-09-16T14:49:12.104 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2602000000000003e-05 2024-09-16T14:49:12.104 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.104 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.104 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.104 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994293+0000", 2024-09-16T14:49:12.104 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7773999999999999e-05 2024-09-16T14:49:12.104 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.104 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.104 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.104 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994304+0000", 2024-09-16T14:49:12.105 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0932e-05 2024-09-16T14:49:12.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.105 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.105 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994371+0000", 2024-09-16T14:49:12.105 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6202000000000001e-05 2024-09-16T14:49:12.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.105 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.105 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994383+0000", 2024-09-16T14:49:12.105 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2685e-05 2024-09-16T14:49:12.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.106 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.106 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002711+0000", 2024-09-16T14:49:12.106 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0083273529999999991 2024-09-16T14:49:12.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.106 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.106 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002742+0000", 2024-09-16T14:49:12.106 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0936000000000003e-05 2024-09-16T14:49:12.106 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.106 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.106 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.106 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds4 51/27 ECSubWrite(tid=211, reqid=client.4500.0:459, at_version=51'31, trim_to=0'0, roll_forward_to=51'29))", 2024-09-16T14:49:12.107 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.994919+0000", 2024-09-16T14:49:12.107 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.028782354000001, 2024-09-16T14:49:12.107 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0087600569999999999, 2024-09-16T14:49:12.107 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.107 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.107 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.107 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.107 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994919+0000", 2024-09-16T14:49:12.107 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.107 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.107 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994919+0000", 2024-09-16T14:49:12.108 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.108 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.108 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994918+0000", 2024-09-16T14:49:12.108 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:12.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.108 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.108 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995629+0000", 2024-09-16T14:49:12.108 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000711245 2024-09-16T14:49:12.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.109 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.109 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995643+0000", 2024-09-16T14:49:12.109 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.398e-05 2024-09-16T14:49:12.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.109 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.109 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995651+0000", 2024-09-16T14:49:12.109 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0479999999999999e-06 2024-09-16T14:49:12.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.110 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.110 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995722+0000", 2024-09-16T14:49:12.110 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.0137999999999998e-05 2024-09-16T14:49:12.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.110 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.110 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995731+0000", 2024-09-16T14:49:12.110 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5279999999999992e-06 2024-09-16T14:49:12.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.110 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.110 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003653+0000", 2024-09-16T14:49:12.110 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0079221599999999993 2024-09-16T14:49:12.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.111 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.111 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003679+0000", 2024-09-16T14:49:12.111 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5882999999999999e-05 2024-09-16T14:49:12.111 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.111 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.111 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.111 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s5 51/27 ECSubWrite(tid=60, reqid=client.4500.0:467, at_version=51'19, trim_to=0'0, roll_forward_to=51'16))", 2024-09-16T14:49:12.111 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:12.111 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.025776609999999, 2024-09-16T14:49:12.111 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0099497439999999999, 2024-09-16T14:49:12.111 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.112 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.112 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.112 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.112 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:12.112 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.112 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.112 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:12.112 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.113 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.113 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997924+0000", 2024-09-16T14:49:12.113 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:12.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.113 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.113 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997944+0000", 2024-09-16T14:49:12.113 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0404e-05 2024-09-16T14:49:12.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.113 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.113 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997957+0000", 2024-09-16T14:49:12.114 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2123000000000001e-05 2024-09-16T14:49:12.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.114 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.114 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997966+0000", 2024-09-16T14:49:12.114 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6549999999999999e-06 2024-09-16T14:49:12.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.114 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.114 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002158+0000", 2024-09-16T14:49:12.114 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0041915930000000004 2024-09-16T14:49:12.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.115 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.115 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002168+0000", 2024-09-16T14:49:12.115 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0071000000000001e-05 2024-09-16T14:49:12.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.115 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.115 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007845+0000", 2024-09-16T14:49:12.115 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056765690000000002 2024-09-16T14:49:12.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.115 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.115 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007875+0000", 2024-09-16T14:49:12.115 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0077e-05 2024-09-16T14:49:12.115 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.116 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.116 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.116 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s5 51/27 ECSubWrite(tid=62, reqid=client.4500.0:468, at_version=51'20, trim_to=0'0, roll_forward_to=51'16))", 2024-09-16T14:49:12.116 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.999395+0000", 2024-09-16T14:49:12.116 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.024306660000001, 2024-09-16T14:49:12.116 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084961840000000004, 2024-09-16T14:49:12.116 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.116 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.116 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.116 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.117 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999395+0000", 2024-09-16T14:49:12.117 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.117 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.117 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999395+0000", 2024-09-16T14:49:12.117 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.117 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.117 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999394+0000", 2024-09-16T14:49:12.117 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:12.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.118 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.118 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999413+0000", 2024-09-16T14:49:12.118 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9508999999999999e-05 2024-09-16T14:49:12.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.118 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.118 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999436+0000", 2024-09-16T14:49:12.118 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2512999999999999e-05 2024-09-16T14:49:12.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.118 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.118 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999445+0000", 2024-09-16T14:49:12.118 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4390000000000006e-06 2024-09-16T14:49:12.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.119 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.119 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003294+0000", 2024-09-16T14:49:12.119 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0038483250000000001 2024-09-16T14:49:12.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.119 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.119 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003304+0000", 2024-09-16T14:49:12.119 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:49:12.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.119 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.119 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007879+0000", 2024-09-16T14:49:12.120 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0045754569999999998 2024-09-16T14:49:12.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.120 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.120 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007891+0000", 2024-09-16T14:49:12.120 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1953000000000001e-05 2024-09-16T14:49:12.120 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.120 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.120 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.120 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:506 3.4s0 3:255172f3:::smithi04040151-3:head [rollback 13] snapc 1b=[1b,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-16T14:49:12.120 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:12.121 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.12933898, 2024-09-16T14:49:12.121 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0092517710000000006, 2024-09-16T14:49:12.121 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.121 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:12.121 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:12.121 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:12.121 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:12.121 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 506 2024-09-16T14:49:12.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.121 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.121 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.121 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:12.121 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.122 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.122 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:12.122 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.122 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.122 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:12.122 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.122 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.123 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894365+0000", 2024-09-16T14:49:12.123 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.571e-06 2024-09-16T14:49:12.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.123 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.123 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894370+0000", 2024-09-16T14:49:12.123 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8279999999999999e-06 2024-09-16T14:49:12.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.123 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.123 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894374+0000", 2024-09-16T14:49:12.123 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5920000000000001e-06 2024-09-16T14:49:12.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.124 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.124 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894483+0000", 2024-09-16T14:49:12.124 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010895699999999999 2024-09-16T14:49:12.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.124 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:12.124 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894491+0000", 2024-09-16T14:49:12.124 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9389999999999997e-06 2024-09-16T14:49:12.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.124 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.124 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.897713+0000", 2024-09-16T14:49:12.124 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0032221060000000002 2024-09-16T14:49:12.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.125 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:12.125 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.897829+0000", 2024-09-16T14:49:12.125 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000116082 2024-09-16T14:49:12.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.125 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.125 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.898559+0000", 2024-09-16T14:49:12.125 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00072999800000000004 2024-09-16T14:49:12.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.125 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.125 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.901829+0000", 2024-09-16T14:49:12.126 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0032702159999999998 2024-09-16T14:49:12.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.126 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:12.126 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903546+0000", 2024-09-16T14:49:12.126 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001716827 2024-09-16T14:49:12.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.126 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.126 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903614+0000", 2024-09-16T14:49:12.126 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8058999999999993e-05 2024-09-16T14:49:12.126 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.126 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.127 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.127 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.127 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:507 3.4s0 3:255172f3:::smithi04040151-3:head [append 0~0] snapc 1b=[1b,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-16T14:49:12.127 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:12.127 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.1293070049999994, 2024-09-16T14:49:12.127 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0095393700000000001, 2024-09-16T14:49:12.127 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.127 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:12.127 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:12.127 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:12.127 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:12.127 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 507 2024-09-16T14:49:12.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.128 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.128 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.128 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:12.128 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.128 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.128 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:12.128 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.128 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.129 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:12.129 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.129 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.129 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894397+0000", 2024-09-16T14:49:12.129 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.774e-06 2024-09-16T14:49:12.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.129 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.129 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894403+0000", 2024-09-16T14:49:12.129 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1079999999999998e-06 2024-09-16T14:49:12.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.130 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.130 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894406+0000", 2024-09-16T14:49:12.130 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.337e-06 2024-09-16T14:49:12.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.130 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.130 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894536+0000", 2024-09-16T14:49:12.130 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012969199999999999 2024-09-16T14:49:12.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.130 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:12.130 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894545+0000", 2024-09-16T14:49:12.130 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7499999999999992e-06 2024-09-16T14:49:12.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.131 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.131 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899040+0000", 2024-09-16T14:49:12.131 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044951289999999996 2024-09-16T14:49:12.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.131 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:12.131 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899164+0000", 2024-09-16T14:49:12.131 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012377699999999999 2024-09-16T14:49:12.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.131 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.131 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899721+0000", 2024-09-16T14:49:12.131 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00055683800000000004 2024-09-16T14:49:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.901911+0000", 2024-09-16T14:49:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0021908650000000002 2024-09-16T14:49:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903859+0000", 2024-09-16T14:49:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001947383 2024-09-16T14:49:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.132 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903934+0000", 2024-09-16T14:49:12.133 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5056999999999996e-05 2024-09-16T14:49:12.133 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.133 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.133 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.133 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.133 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.133 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:12.134 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:12.024+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:12.135 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:49:12.135 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:12.133+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:12.236 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:12.235+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:12.282 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:12.282 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:12.282 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:12.282 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:12.290 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:49:12.337 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:12.335+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:12.436 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:12.436 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:12.436 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:12.436 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:12.437 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:12.438 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:12.436+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:12.445 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:49:12.539 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:12.538+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:12.553 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:12.553 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:12.553 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:12.553 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:12.553 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.553 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=128, reqid=client.4500.0:369, at_version=42'38, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:12.553 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:12.553 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.591008340999998, 2024-09-16T14:49:12.553 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060794769999999998, 2024-09-16T14:49:12.553 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.554 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.554 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.554 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.554 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.554 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:12.554 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.554 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.554 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.554 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.554 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:12.554 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.554 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.554 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.554 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.555 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959976+0000", 2024-09-16T14:49:12.555 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:12.555 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.555 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.555 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.555 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960436+0000", 2024-09-16T14:49:12.555 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00046022400000000002 2024-09-16T14:49:12.555 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.555 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.555 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.555 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960455+0000", 2024-09-16T14:49:12.555 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8162999999999999e-05 2024-09-16T14:49:12.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.556 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.556 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960468+0000", 2024-09-16T14:49:12.556 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3135999999999999e-05 2024-09-16T14:49:12.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.556 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.556 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960527+0000", 2024-09-16T14:49:12.556 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8859e-05 2024-09-16T14:49:12.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.556 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.557 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960533+0000", 2024-09-16T14:49:12.557 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:49:12.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.557 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.557 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.557 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.966018+0000", 2024-09-16T14:49:12.557 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0054845069999999996 2024-09-16T14:49:12.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.557 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.557 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.557 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.966057+0000", 2024-09-16T14:49:12.557 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.8927999999999997e-05 2024-09-16T14:49:12.557 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=129, reqid=client.4500.0:370, at_version=42'39, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.589752582999999, 2024-09-16T14:49:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074279639999999996, 2024-09-16T14:49:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.558 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:12.559 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.559 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.559 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:12.559 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.559 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.559 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961232+0000", 2024-09-16T14:49:12.559 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:12.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.559 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.560 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962118+0000", 2024-09-16T14:49:12.560 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00088592399999999996 2024-09-16T14:49:12.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.560 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.560 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962132+0000", 2024-09-16T14:49:12.560 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4127e-05 2024-09-16T14:49:12.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.560 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.560 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962142+0000", 2024-09-16T14:49:12.560 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0181e-05 2024-09-16T14:49:12.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.561 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.561 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962192+0000", 2024-09-16T14:49:12.561 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9826e-05 2024-09-16T14:49:12.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.561 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.561 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962202+0000", 2024-09-16T14:49:12.561 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8400000000000007e-06 2024-09-16T14:49:12.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.561 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.561 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968622+0000", 2024-09-16T14:49:12.561 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00642021 2024-09-16T14:49:12.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.562 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.562 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968661+0000", 2024-09-16T14:49:12.562 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.8670000000000001e-05 2024-09-16T14:49:12.562 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.562 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.562 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.562 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=130, reqid=client.4500.0:371, at_version=42'40, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:12.562 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:12.562 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.588417327999998, 2024-09-16T14:49:12.563 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0061144900000000002, 2024-09-16T14:49:12.563 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.563 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.563 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.563 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.563 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.563 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:12.563 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.563 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.563 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.563 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:12.563 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.564 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.564 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962567+0000", 2024-09-16T14:49:12.564 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.564 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.564 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962595+0000", 2024-09-16T14:49:12.564 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7824999999999999e-05 2024-09-16T14:49:12.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.565 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.565 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962611+0000", 2024-09-16T14:49:12.565 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:49:12.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.565 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.565 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962620+0000", 2024-09-16T14:49:12.565 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8259999999999999e-06 2024-09-16T14:49:12.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.565 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.565 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.963285+0000", 2024-09-16T14:49:12.565 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00066560100000000004 2024-09-16T14:49:12.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.566 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.566 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.566 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.963295+0000", 2024-09-16T14:49:12.566 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0036e-05 2024-09-16T14:49:12.566 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.566 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.566 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.566 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968667+0000", 2024-09-16T14:49:12.566 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0053715960000000002 2024-09-16T14:49:12.566 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.566 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968683+0000", 2024-09-16T14:49:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5522000000000001e-05 2024-09-16T14:49:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 43/27 ECSubWrite(tid=42, reqid=osd.1.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=43'30))", 2024-09-16T14:49:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:49.851949+0000", 2024-09-16T14:49:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 22.699035946999999, 2024-09-16T14:49:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0057984890000000004, 2024-09-16T14:49:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.567 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.568 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.568 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.568 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851949+0000", 2024-09-16T14:49:12.568 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.568 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.568 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.568 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851949+0000", 2024-09-16T14:49:12.568 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.568 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.568 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.569 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851949+0000", 2024-09-16T14:49:12.569 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:12.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.569 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.569 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.569 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851962+0000", 2024-09-16T14:49:12.569 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3189e-05 2024-09-16T14:49:12.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.569 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.569 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.569 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851979+0000", 2024-09-16T14:49:12.570 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6724000000000001e-05 2024-09-16T14:49:12.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.570 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.570 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.851987+0000", 2024-09-16T14:49:12.570 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.568e-06 2024-09-16T14:49:12.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.570 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.570 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.852055+0000", 2024-09-16T14:49:12.570 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.7519000000000002e-05 2024-09-16T14:49:12.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.570 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.571 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.852065+0000", 2024-09-16T14:49:12.571 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0111e-05 2024-09-16T14:49:12.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.571 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.571 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.571 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.857719+0000", 2024-09-16T14:49:12.571 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056545429999999997 2024-09-16T14:49:12.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.571 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.571 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.571 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.857748+0000", 2024-09-16T14:49:12.571 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8609e-05 2024-09-16T14:49:12.571 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.572 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.572 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.572 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 43/27 ECSubWrite(tid=133, reqid=osd.11.0:133, at_version=43'42, trim_to=0'0, roll_forward_to=42'40))", 2024-09-16T14:49:12.572 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:12.572 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 22.697209533999999, 2024-09-16T14:49:12.572 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073093740000000004, 2024-09-16T14:49:12.572 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.572 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.572 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.573 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.573 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:12.573 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.573 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.573 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:12.573 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.573 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.573 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:12.573 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:12.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.574 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.574 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854515+0000", 2024-09-16T14:49:12.574 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00073950999999999997 2024-09-16T14:49:12.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.574 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.574 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854527+0000", 2024-09-16T14:49:12.574 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2012e-05 2024-09-16T14:49:12.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.574 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.574 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854535+0000", 2024-09-16T14:49:12.575 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:49:12.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.575 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.575 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854952+0000", 2024-09-16T14:49:12.575 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00041779399999999998 2024-09-16T14:49:12.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.575 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.575 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854961+0000", 2024-09-16T14:49:12.575 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7749999999999998e-06 2024-09-16T14:49:12.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.575 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.576 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.861052+0000", 2024-09-16T14:49:12.576 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006090608 2024-09-16T14:49:12.576 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.576 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.576 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.576 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.861085+0000", 2024-09-16T14:49:12.576 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3442999999999998e-05 2024-09-16T14:49:12.576 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.576 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.576 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.576 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.576 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.576 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 51/27 ECSubWrite(tid=169, reqid=osd.11.0:169, at_version=51'52, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:12.576 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:12.577 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.669799613, 2024-09-16T14:49:12.577 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006781848, 2024-09-16T14:49:12.577 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.577 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.577 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.577 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.577 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.577 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:12.577 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.577 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.577 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.577 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881185+0000", 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881208+0000", 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2823000000000002e-05 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.578 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881218+0000", 2024-09-16T14:49:12.579 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3929999999999997e-06 2024-09-16T14:49:12.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.579 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.579 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881225+0000", 2024-09-16T14:49:12.579 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1629999999999999e-06 2024-09-16T14:49:12.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.579 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.579 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881279+0000", 2024-09-16T14:49:12.579 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4147000000000002e-05 2024-09-16T14:49:12.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.580 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.580 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881288+0000", 2024-09-16T14:49:12.580 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4630000000000007e-06 2024-09-16T14:49:12.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.580 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.580 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.580 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.887941+0000", 2024-09-16T14:49:12.580 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0066530210000000003 2024-09-16T14:49:12.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.581 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.581 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.581 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.887968+0000", 2024-09-16T14:49:12.581 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6228999999999999e-05 2024-09-16T14:49:12.581 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.581 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.581 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.581 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.581 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.581 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 51/27 ECSubWrite(tid=173, reqid=osd.11.0:173, at_version=51'54, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:12.581 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:12.581 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.669648386, 2024-09-16T14:49:12.581 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0080949799999999999, 2024-09-16T14:49:12.581 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.582 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.583 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.583 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881336+0000", 2024-09-16T14:49:12.583 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.583 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.583 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.583 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881356+0000", 2024-09-16T14:49:12.583 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9334000000000002e-05 2024-09-16T14:49:12.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.584 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.584 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.584 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881366+0000", 2024-09-16T14:49:12.584 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0742e-05 2024-09-16T14:49:12.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.585 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.585 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.585 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881374+0000", 2024-09-16T14:49:12.585 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5020000000000003e-06 2024-09-16T14:49:12.586 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.586 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.586 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.586 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.882087+0000", 2024-09-16T14:49:12.586 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00071271300000000002 2024-09-16T14:49:12.586 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.586 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.586 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.586 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.882096+0000", 2024-09-16T14:49:12.586 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4550000000000001e-06 2024-09-16T14:49:12.586 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.586 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.586 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.586 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889403+0000", 2024-09-16T14:49:12.587 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073067749999999997 2024-09-16T14:49:12.587 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.587 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.587 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.587 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889432+0000", 2024-09-16T14:49:12.587 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9065999999999999e-05 2024-09-16T14:49:12.587 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.587 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.587 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.588 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.588 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 52/27 ECSubWrite(tid=50, reqid=osd.1.0:50, at_version=52'40, trim_to=0'0, roll_forward_to=51'36))", 2024-09-16T14:49:12.588 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:12.588 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 13.668445517, 2024-09-16T14:49:12.588 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060730380000000002, 2024-09-16T14:49:12.588 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.588 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.588 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.588 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.588 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.588 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:12.588 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.589 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.589 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.589 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.589 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:12.589 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.589 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.589 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.589 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.589 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882539+0000", 2024-09-16T14:49:12.589 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:12.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.590 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.590 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.590 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882566+0000", 2024-09-16T14:49:12.590 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7101999999999999e-05 2024-09-16T14:49:12.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.590 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.590 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.590 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882580+0000", 2024-09-16T14:49:12.590 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4605e-05 2024-09-16T14:49:12.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.590 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.590 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.590 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882589+0000", 2024-09-16T14:49:12.591 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1020000000000003e-06 2024-09-16T14:49:12.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.591 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.591 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.884101+0000", 2024-09-16T14:49:12.591 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001511601 2024-09-16T14:49:12.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.591 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.591 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.884112+0000", 2024-09-16T14:49:12.591 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1055e-05 2024-09-16T14:49:12.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.591 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.592 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888581+0000", 2024-09-16T14:49:12.592 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044684240000000004 2024-09-16T14:49:12.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.592 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.592 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888613+0000", 2024-09-16T14:49:12.592 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2354000000000002e-05 2024-09-16T14:49:12.592 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.592 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.592 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.592 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=235, reqid=osd.7.0:235, at_version=61'21, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:12.593 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:12.593 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.6194595100000004, 2024-09-16T14:49:12.593 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0080251469999999998, 2024-09-16T14:49:12.593 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.593 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.593 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.593 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.593 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:12.593 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.593 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.593 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:12.594 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.594 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.594 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931525+0000", 2024-09-16T14:49:12.594 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.594 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.594 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931556+0000", 2024-09-16T14:49:12.594 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0713000000000002e-05 2024-09-16T14:49:12.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.595 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.595 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931577+0000", 2024-09-16T14:49:12.595 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0879000000000001e-05 2024-09-16T14:49:12.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.595 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.595 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931599+0000", 2024-09-16T14:49:12.595 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2107999999999999e-05 2024-09-16T14:49:12.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.596 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.596 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932715+0000", 2024-09-16T14:49:12.596 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0011159119999999999 2024-09-16T14:49:12.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.596 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.596 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932725+0000", 2024-09-16T14:49:12.596 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8239999999999995e-06 2024-09-16T14:49:12.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.596 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.596 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939520+0000", 2024-09-16T14:49:12.596 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00679556 2024-09-16T14:49:12.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.597 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.597 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939551+0000", 2024-09-16T14:49:12.597 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0582e-05 2024-09-16T14:49:12.597 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.597 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.597 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.597 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 61/27 ECSubWrite(tid=63, reqid=osd.1.0:63, at_version=61'50, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:12.597 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:12.597 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.61933872, 2024-09-16T14:49:12.598 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090916260000000002, 2024-09-16T14:49:12.598 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.598 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.598 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.598 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.598 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:12.598 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.598 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.598 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:12.598 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.599 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.599 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931646+0000", 2024-09-16T14:49:12.599 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:12.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.599 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.599 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931665+0000", 2024-09-16T14:49:12.599 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8459999999999999e-05 2024-09-16T14:49:12.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.599 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.600 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931677+0000", 2024-09-16T14:49:12.600 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2150000000000001e-05 2024-09-16T14:49:12.600 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.600 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.600 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931684+0000", 2024-09-16T14:49:12.600 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3599999999999998e-06 2024-09-16T14:49:12.600 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.600 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.600 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931754+0000", 2024-09-16T14:49:12.600 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.9733999999999993e-05 2024-09-16T14:49:12.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.601 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.601 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931766+0000", 2024-09-16T14:49:12.601 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1873e-05 2024-09-16T14:49:12.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.601 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.601 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940712+0000", 2024-09-16T14:49:12.602 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089461669999999997 2024-09-16T14:49:12.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.602 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.602 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940738+0000", 2024-09-16T14:49:12.602 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6092999999999999e-05 2024-09-16T14:49:12.602 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.602 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.602 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.602 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:12.603 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:12.603 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.6189387069999999, 2024-09-16T14:49:12.603 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010994122, 2024-09-16T14:49:12.603 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.603 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.603 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.603 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.603 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:12.603 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.603 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.604 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:12.604 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.604 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.604 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:12.604 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:12.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.604 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.604 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932066+0000", 2024-09-16T14:49:12.604 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9802000000000001e-05 2024-09-16T14:49:12.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.605 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.605 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932076+0000", 2024-09-16T14:49:12.605 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8470000000000008e-06 2024-09-16T14:49:12.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.605 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.605 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932086+0000", 2024-09-16T14:49:12.605 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7010000000000008e-06 2024-09-16T14:49:12.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.605 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.605 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933307+0000", 2024-09-16T14:49:12.606 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0012207450000000001 2024-09-16T14:49:12.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.606 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.606 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933319+0000", 2024-09-16T14:49:12.606 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2473000000000001e-05 2024-09-16T14:49:12.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.606 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.606 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943013+0000", 2024-09-16T14:49:12.607 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0096943700000000008 2024-09-16T14:49:12.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.607 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.607 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943041+0000", 2024-09-16T14:49:12.607 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7376999999999999e-05 2024-09-16T14:49:12.607 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.607 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.607 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.607 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:12.607 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:12.607 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.6178138789999998, 2024-09-16T14:49:12.608 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0099360779999999992, 2024-09-16T14:49:12.608 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.608 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.608 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.608 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.608 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:12.608 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.608 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.608 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:12.609 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.609 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.609 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933171+0000", 2024-09-16T14:49:12.609 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.609 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.609 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933205+0000", 2024-09-16T14:49:12.609 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3630999999999997e-05 2024-09-16T14:49:12.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.610 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.610 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933214+0000", 2024-09-16T14:49:12.610 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0620000000000007e-06 2024-09-16T14:49:12.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.610 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.610 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933222+0000", 2024-09-16T14:49:12.610 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:12.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.610 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.611 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933271+0000", 2024-09-16T14:49:12.611 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9251e-05 2024-09-16T14:49:12.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.611 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.611 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933280+0000", 2024-09-16T14:49:12.611 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5419999999999994e-06 2024-09-16T14:49:12.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.611 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.611 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943089+0000", 2024-09-16T14:49:12.612 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0098082529999999994 2024-09-16T14:49:12.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.612 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.612 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943108+0000", 2024-09-16T14:49:12.612 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8916e-05 2024-09-16T14:49:12.612 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.612 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.612 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.612 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s2 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:12.612 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:12.612 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.6176753499999998, 2024-09-16T14:49:12.613 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01234993, 2024-09-16T14:49:12.613 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.613 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.613 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.613 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.613 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:12.613 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.613 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.613 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:12.613 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.614 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.614 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:12.614 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.614 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.614 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933339+0000", 2024-09-16T14:49:12.614 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9107000000000001e-05 2024-09-16T14:49:12.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.615 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.615 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933347+0000", 2024-09-16T14:49:12.615 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6219999999999998e-06 2024-09-16T14:49:12.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.615 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.615 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933354+0000", 2024-09-16T14:49:12.615 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.72e-06 2024-09-16T14:49:12.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.615 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.615 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933438+0000", 2024-09-16T14:49:12.615 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.3666999999999997e-05 2024-09-16T14:49:12.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.616 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.616 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933453+0000", 2024-09-16T14:49:12.616 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4814000000000001e-05 2024-09-16T14:49:12.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.616 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.616 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945638+0000", 2024-09-16T14:49:12.616 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012185367000000001 2024-09-16T14:49:12.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.617 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.617 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945660+0000", 2024-09-16T14:49:12.617 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2033999999999999e-05 2024-09-16T14:49:12.617 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.617 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.617 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.617 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:12.617 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:12.618 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.6176302289999995, 2024-09-16T14:49:12.618 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012249365, 2024-09-16T14:49:12.618 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.618 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.618 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.618 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.618 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:12.618 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.618 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.618 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:12.619 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.619 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.619 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933354+0000", 2024-09-16T14:49:12.619 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:12.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.619 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.619 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933377+0000", 2024-09-16T14:49:12.619 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3087000000000001e-05 2024-09-16T14:49:12.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.620 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.620 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933393+0000", 2024-09-16T14:49:12.620 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5231e-05 2024-09-16T14:49:12.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.620 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.620 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933400+0000", 2024-09-16T14:49:12.620 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.6190000000000002e-06 2024-09-16T14:49:12.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.620 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.620 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933470+0000", 2024-09-16T14:49:12.620 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.936e-05 2024-09-16T14:49:12.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.621 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.621 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933479+0000", 2024-09-16T14:49:12.621 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2609999999999999e-06 2024-09-16T14:49:12.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.621 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.621 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945575+0000", 2024-09-16T14:49:12.621 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012095896 2024-09-16T14:49:12.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.621 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.621 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945605+0000", 2024-09-16T14:49:12.622 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9765e-05 2024-09-16T14:49:12.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.622 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.622 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:12.622 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:12.622 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.6175879349999995, 2024-09-16T14:49:12.622 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013525466, 2024-09-16T14:49:12.622 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.622 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.622 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.623 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.623 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:12.623 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.623 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.623 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:12.623 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.623 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.623 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:12.623 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:12.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.624 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.624 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933451+0000", 2024-09-16T14:49:12.624 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4162000000000001e-05 2024-09-16T14:49:12.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.624 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.624 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933459+0000", 2024-09-16T14:49:12.624 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1110000000000008e-06 2024-09-16T14:49:12.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.624 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.625 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:12.625 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5050000000000004e-06 2024-09-16T14:49:12.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.625 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.625 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.936478+0000", 2024-09-16T14:49:12.625 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0030122790000000001 2024-09-16T14:49:12.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.625 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.625 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.936488+0000", 2024-09-16T14:49:12.625 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0197000000000001e-05 2024-09-16T14:49:12.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.626 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.626 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946886+0000", 2024-09-16T14:49:12.626 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010397912 2024-09-16T14:49:12.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.626 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.626 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946923+0000", 2024-09-16T14:49:12.626 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6699999999999998e-05 2024-09-16T14:49:12.626 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.626 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.626 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.627 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:12.627 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:12.627 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.6164304739999995, 2024-09-16T14:49:12.627 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0083397039999999999, 2024-09-16T14:49:12.627 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.627 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.627 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.627 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.627 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:12.627 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.628 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.628 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:12.628 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.628 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.628 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.628 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934554+0000", 2024-09-16T14:49:12.628 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:12.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.628 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.628 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.628 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935547+0000", 2024-09-16T14:49:12.628 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00099309300000000001 2024-09-16T14:49:12.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.629 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.629 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.629 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935565+0000", 2024-09-16T14:49:12.629 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8335999999999999e-05 2024-09-16T14:49:12.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.629 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.629 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.629 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935578+0000", 2024-09-16T14:49:12.629 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2831e-05 2024-09-16T14:49:12.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.629 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.629 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.629 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935653+0000", 2024-09-16T14:49:12.630 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4411000000000003e-05 2024-09-16T14:49:12.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.630 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.630 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935663+0000", 2024-09-16T14:49:12.630 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0869999999999999e-05 2024-09-16T14:49:12.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.630 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.630 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942652+0000", 2024-09-16T14:49:12.630 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006988129 2024-09-16T14:49:12.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.631 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.631 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942895+0000", 2024-09-16T14:49:12.631 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00024308899999999999 2024-09-16T14:49:12.631 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.631 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.631 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.631 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.631 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s2 61/53 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=61'38, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:12.631 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:12.631 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.6130915940000001, 2024-09-16T14:49:12.631 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090892060000000007, 2024-09-16T14:49:12.632 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.632 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.632 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.632 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.632 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.632 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:12.632 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.632 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.632 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.632 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:12.632 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.633 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.633 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.633 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937893+0000", 2024-09-16T14:49:12.633 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:12.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.633 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.633 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.633 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937915+0000", 2024-09-16T14:49:12.633 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2566e-05 2024-09-16T14:49:12.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937929+0000", 2024-09-16T14:49:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3314e-05 2024-09-16T14:49:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937938+0000", 2024-09-16T14:49:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2050000000000008e-06 2024-09-16T14:49:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.634 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940150+0000", 2024-09-16T14:49:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0022119040000000002 2024-09-16T14:49:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940158+0000", 2024-09-16T14:49:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2930000000000001e-06 2024-09-16T14:49:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946972+0000", 2024-09-16T14:49:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0068139150000000002 2024-09-16T14:49:12.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946983+0000", 2024-09-16T14:49:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0907000000000001e-05 2024-09-16T14:49:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 61/27 ECSubWrite(tid=226, reqid=osd.11.0:226, at_version=61'82, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:12.636 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.6125689789999997, 2024-09-16T14:49:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085452879999999998, 2024-09-16T14:49:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.637 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938432+0000", 2024-09-16T14:49:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5931e-05 2024-09-16T14:49:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.638 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.639 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938444+0000", 2024-09-16T14:49:12.639 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1933000000000001e-05 2024-09-16T14:49:12.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.639 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.639 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938449+0000", 2024-09-16T14:49:12.639 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2070000000000003e-06 2024-09-16T14:49:12.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.639 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.639 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938492+0000", 2024-09-16T14:49:12.639 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2827999999999997e-05 2024-09-16T14:49:12.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.640 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.640 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938500+0000", 2024-09-16T14:49:12.640 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7600000000000008e-06 2024-09-16T14:49:12.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.640 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.640 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946932+0000", 2024-09-16T14:49:12.640 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084318099999999997 2024-09-16T14:49:12.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.640 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.640 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946962+0000", 2024-09-16T14:49:12.641 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9603000000000001e-05 2024-09-16T14:49:12.641 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.641 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.641 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.641 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=238, reqid=client.4500.0:536, at_version=61'23, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:12.641 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:12.641 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.6117460069999998, 2024-09-16T14:49:12.641 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0064046049999999998, 2024-09-16T14:49:12.641 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.641 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.641 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.642 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.642 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:12.642 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.642 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.642 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:12.642 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.642 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.642 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939238+0000", 2024-09-16T14:49:12.643 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:12.643 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.643 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.643 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939537+0000", 2024-09-16T14:49:12.643 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029911299999999999 2024-09-16T14:49:12.643 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.643 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.643 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939551+0000", 2024-09-16T14:49:12.643 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4003e-05 2024-09-16T14:49:12.643 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.644 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.644 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939561+0000", 2024-09-16T14:49:12.644 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4269999999999997e-06 2024-09-16T14:49:12.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.644 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.644 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.644 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939777+0000", 2024-09-16T14:49:12.644 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00021644599999999999 2024-09-16T14:49:12.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.644 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.644 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.644 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939790+0000", 2024-09-16T14:49:12.644 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2381999999999999e-05 2024-09-16T14:49:12.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.645 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.645 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945601+0000", 2024-09-16T14:49:12.645 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0058117139999999999 2024-09-16T14:49:12.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.645 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.645 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945644+0000", 2024-09-16T14:49:12.645 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.265e-05 2024-09-16T14:49:12.645 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.645 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.645 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=239, reqid=client.4500.0:537, at_version=61'24, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.940949+0000", 2024-09-16T14:49:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.610036708, 2024-09-16T14:49:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0059680189999999998, 2024-09-16T14:49:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940949+0000", 2024-09-16T14:49:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:12.647 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940949+0000", 2024-09-16T14:49:12.647 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:12.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.647 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:12.647 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940948+0000", 2024-09-16T14:49:12.647 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:12.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.647 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:12.647 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.941054+0000", 2024-09-16T14:49:12.647 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000106405 2024-09-16T14:49:12.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.648 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:12.648 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.941074+0000", 2024-09-16T14:49:12.648 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9813000000000002e-05 2024-09-16T14:49:12.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.648 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:12.648 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.941083+0000", 2024-09-16T14:49:12.648 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6850000000000007e-06 2024-09-16T14:49:12.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.648 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:12.648 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.941156+0000", 2024-09-16T14:49:12.648 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3003999999999994e-05 2024-09-16T14:49:12.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.649 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.649 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:12.649 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.941165+0000", 2024-09-16T14:49:12.649 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6299999999999993e-06 2024-09-16T14:49:12.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.649 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.649 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:12.649 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946887+0000", 2024-09-16T14:49:12.649 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.005722024 2024-09-16T14:49:12.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:12.649 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:12.649 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:12.649 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946917+0000", 2024-09-16T14:49:12.650 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9252e-05 2024-09-16T14:49:12.650 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.650 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.650 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.650 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:12.650 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:12.650 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:12.651 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:49:12.652 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:12.650+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:12.753 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:12.751+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:12.799 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:12.799 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:12.799 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:12.799 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:12.807 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:49:12.853 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:12.852+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:12.888 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:49:12.914 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:12.914 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:12.914 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:12.915 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:12.915 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:12.923 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:49:12.954 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:12.952+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:12.965 INFO:tasks.rados.rados.0.smithi040.stdout:244: done (7 left) 2024-09-16T14:49:12.965 INFO:tasks.rados.rados.0.smithi040.stdout:247: done (6 left) 2024-09-16T14:49:12.965 INFO:tasks.rados.rados.0.smithi040.stdout:250: done (5 left) 2024-09-16T14:49:12.965 INFO:tasks.rados.rados.0.smithi040.stdout:251: rmattr oid 23 current snap is 19 2024-09-16T14:49:12.965 INFO:tasks.rados.rados.0.smithi040.stdout:251: done (5 left) 2024-09-16T14:49:12.966 INFO:tasks.rados.rados.0.smithi040.stdout:252: rmattr oid 44 current snap is 19 2024-09-16T14:49:12.966 INFO:tasks.rados.rados.0.smithi040.stdout:252: done (5 left) 2024-09-16T14:49:12.966 INFO:tasks.rados.rados.0.smithi040.stdout:253: snap_create 2024-09-16T14:49:12.966 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:12.960+0000 7f4085d96640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:12.966 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:12.960+0000 7f4085d96640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:12.966 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:12.964+0000 7f4081d8e640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:12.987 INFO:tasks.rados.rados.0.smithi040.stdout:243: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-16T14:49:13.015 INFO:tasks.rados.rados.0.smithi040.stdout:245: finishing rollback tid 0 to smithi04040151-3 2024-09-16T14:49:13.015 INFO:tasks.rados.rados.0.smithi040.stdout:249: finishing write tid 1 to smithi04040151-1 2024-09-16T14:49:13.015 INFO:tasks.rados.rados.0.smithi040.stdout:246: finishing copy_from racing read to smithi04040151-16 2024-09-16T14:49:13.015 INFO:tasks.rados.rados.0.smithi040.stdout:245: finishing rollback tid 1 to smithi04040151-3 2024-09-16T14:49:13.015 INFO:tasks.rados.rados.0.smithi040.stdout:246: finishing copy_from to smithi04040151-16 2024-09-16T14:49:13.015 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 16 v 27 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-16T14:49:13.016 INFO:tasks.rados.rados.0.smithi040.stdout:245: finishing rollback tid 2 to smithi04040151-3 2024-09-16T14:49:13.016 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 3 v 58 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-16T14:49:13.016 INFO:tasks.rados.rados.0.smithi040.stdout:249: finishing write tid 2 to smithi04040151-1 2024-09-16T14:49:13.016 INFO:tasks.rados.rados.0.smithi040.stdout:249: finishing write tid 3 to smithi04040151-1 2024-09-16T14:49:13.016 INFO:tasks.rados.rados.0.smithi040.stdout:249: finishing write tid 4 to smithi04040151-1 2024-09-16T14:49:13.016 INFO:tasks.rados.rados.0.smithi040.stdout:249: finishing write tid 5 to smithi04040151-1 2024-09-16T14:49:13.016 INFO:tasks.rados.rados.0.smithi040.stdout:249: oid 1 updating version 0 to 54 2024-09-16T14:49:13.016 INFO:tasks.rados.rados.0.smithi040.stdout:249: oid 1 updating version 54 to 55 2024-09-16T14:49:13.016 INFO:tasks.rados.rados.0.smithi040.stdout:249: oid 1 version 55 is already newer than 53 2024-09-16T14:49:13.016 INFO:tasks.rados.rados.0.smithi040.stdout:249: oid 1 version 55 is already newer than 52 2024-09-16T14:49:13.016 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 1 v 55 (ObjNum 99 snap 18 seq_num 99) dirty exists 2024-09-16T14:49:13.016 INFO:tasks.rados.rados.0.smithi040.stdout:249: left oid 1 (ObjNum 99 snap 18 seq_num 99) 2024-09-16T14:49:13.016 INFO:tasks.rados.rados.0.smithi040.stdout:248: finishing write tid 1 to smithi04040151-33 2024-09-16T14:49:13.017 INFO:tasks.rados.rados.0.smithi040.stdout:248: finishing write tid 2 to smithi04040151-33 2024-09-16T14:49:13.017 INFO:tasks.rados.rados.0.smithi040.stdout:248: finishing write tid 3 to smithi04040151-33 2024-09-16T14:49:13.019 INFO:tasks.rados.rados.0.smithi040.stdout:248: finishing write tid 4 to smithi04040151-33 2024-09-16T14:49:13.020 INFO:tasks.rados.rados.0.smithi040.stdout:248: finishing write tid 5 to smithi04040151-33 2024-09-16T14:49:13.025 INFO:tasks.rados.rados.0.smithi040.stdout:248: finishing write tid 6 to smithi04040151-33 2024-09-16T14:49:13.025 INFO:tasks.rados.rados.0.smithi040.stdout:248: oid 33 updating version 0 to 24 2024-09-16T14:49:13.025 INFO:tasks.rados.rados.0.smithi040.stdout:248: oid 33 updating version 24 to 27 2024-09-16T14:49:13.025 INFO:tasks.rados.rados.0.smithi040.stdout:248: oid 33 version 27 is already newer than 26 2024-09-16T14:49:13.025 INFO:tasks.rados.rados.0.smithi040.stdout:248: oid 33 version 27 is already newer than 25 2024-09-16T14:49:13.025 INFO:tasks.rados.rados.0.smithi040.stdout:248: oid 33 version 27 is already newer than 23 2024-09-16T14:49:13.025 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 33 v 27 (ObjNum 98 snap 18 seq_num 98) dirty exists 2024-09-16T14:49:13.026 INFO:tasks.rados.rados.0.smithi040.stdout:248: left oid 33 (ObjNum 98 snap 18 seq_num 98) 2024-09-16T14:49:13.055 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:13.053+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:13.100 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:13.100 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:13.100 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:13.101 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:13.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.101 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:303 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:13.101 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:13.101 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.165783711, 2024-09-16T14:49:13.101 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014763235, 2024-09-16T14:49:13.101 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.101 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:13.101 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:13.101 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:13.101 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:13.101 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 303 2024-09-16T14:49:13.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.102 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.102 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.102 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:13.102 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.102 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.102 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:13.102 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.102 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.102 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932467+0000", 2024-09-16T14:49:13.103 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:13.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.103 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.103 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932474+0000", 2024-09-16T14:49:13.103 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3189999999999997e-06 2024-09-16T14:49:13.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.103 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.103 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932490+0000", 2024-09-16T14:49:13.103 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:49:13.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.104 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.104 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932498+0000", 2024-09-16T14:49:13.104 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7789999999999996e-06 2024-09-16T14:49:13.104 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.104 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.104 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.104 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932544+0000", 2024-09-16T14:49:13.104 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6486000000000002e-05 2024-09-16T14:49:13.104 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.104 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.104 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:13.104 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932550+0000", 2024-09-16T14:49:13.105 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.3149999999999999e-06 2024-09-16T14:49:13.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.105 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.105 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.939655+0000", 2024-09-16T14:49:13.105 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071050039999999998 2024-09-16T14:49:13.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.105 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:13.105 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.939762+0000", 2024-09-16T14:49:13.105 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000106885 2024-09-16T14:49:13.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.105 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.106 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.947232+0000", 2024-09-16T14:49:13.106 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074701619999999998 2024-09-16T14:49:13.106 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.106 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.106 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.106 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:304 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:13.106 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:13.106 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.16574597, 2024-09-16T14:49:13.106 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015391726, 2024-09-16T14:49:13.106 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.106 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:13.106 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:13.107 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:13.107 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:13.107 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 304 2024-09-16T14:49:13.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.107 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.107 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.107 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:13.107 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.107 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.107 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:13.108 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.108 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.108 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:13.108 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.108 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.108 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932510+0000", 2024-09-16T14:49:13.108 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.1069999999999998e-06 2024-09-16T14:49:13.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.108 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.109 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932516+0000", 2024-09-16T14:49:13.109 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4699999999999999e-06 2024-09-16T14:49:13.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.109 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.109 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932522+0000", 2024-09-16T14:49:13.109 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4750000000000001e-06 2024-09-16T14:49:13.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.109 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.109 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932610+0000", 2024-09-16T14:49:13.109 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8623999999999997e-05 2024-09-16T14:49:13.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.110 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:13.110 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932616+0000", 2024-09-16T14:49:13.110 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.9649999999999996e-06 2024-09-16T14:49:13.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.110 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.110 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940205+0000", 2024-09-16T14:49:13.110 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0075882570000000002 2024-09-16T14:49:13.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.110 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:13.110 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940285+0000", 2024-09-16T14:49:13.111 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0783000000000002e-05 2024-09-16T14:49:13.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.111 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.111 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.947898+0000", 2024-09-16T14:49:13.111 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0076124820000000003 2024-09-16T14:49:13.111 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.111 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.111 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.111 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:305 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:13.111 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:13.112 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.165665180000001, 2024-09-16T14:49:13.112 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015660833999999998, 2024-09-16T14:49:13.112 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.112 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:13.112 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:13.112 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:13.112 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:13.112 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 305 2024-09-16T14:49:13.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.112 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.112 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.112 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:13.113 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.113 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.113 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:13.113 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.113 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.113 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932586+0000", 2024-09-16T14:49:13.113 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932591+0000", 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2470000000000002e-06 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932602+0000", 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1396000000000001e-05 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932608+0000", 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.7169999999999996e-06 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.115 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.115 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932646+0000", 2024-09-16T14:49:13.115 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.7978999999999997e-05 2024-09-16T14:49:13.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.115 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:13.115 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932652+0000", 2024-09-16T14:49:13.115 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0310000000000004e-06 2024-09-16T14:49:13.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.115 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.115 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941247+0000", 2024-09-16T14:49:13.115 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085954180000000005 2024-09-16T14:49:13.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.116 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:13.116 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941327+0000", 2024-09-16T14:49:13.116 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9354e-05 2024-09-16T14:49:13.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.116 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.116 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.948248+0000", 2024-09-16T14:49:13.116 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0069212400000000004 2024-09-16T14:49:13.116 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.116 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.116 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.117 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:403 3.2s0 3:421e4b92:::smithi04040151-11:head [copy-from ver 2] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:13.117 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:13.117 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.214687283, 2024-09-16T14:49:13.117 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015904270000000002, 2024-09-16T14:49:13.117 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.117 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:13.117 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:13.117 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:13.117 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:13.117 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 403 2024-09-16T14:49:13.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.117 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.118 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.118 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:13.118 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.118 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.118 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:13.118 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.118 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.118 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883563+0000", 2024-09-16T14:49:13.118 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:13.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.119 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.119 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883573+0000", 2024-09-16T14:49:13.119 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0583e-05 2024-09-16T14:49:13.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.119 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.119 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883593+0000", 2024-09-16T14:49:13.119 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0052e-05 2024-09-16T14:49:13.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.119 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.120 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883603+0000", 2024-09-16T14:49:13.120 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4409999999999999e-06 2024-09-16T14:49:13.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.120 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.120 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883667+0000", 2024-09-16T14:49:13.120 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4671000000000005e-05 2024-09-16T14:49:13.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.120 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:13.120 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883775+0000", 2024-09-16T14:49:13.120 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000108068 2024-09-16T14:49:13.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.896809+0000", 2024-09-16T14:49:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01303355 2024-09-16T14:49:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897717+0000", 2024-09-16T14:49:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00090801599999999999 2024-09-16T14:49:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.899408+0000", 2024-09-16T14:49:13.121 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0016911459999999999 2024-09-16T14:49:13.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.122 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.122 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.899469+0000", 2024-09-16T14:49:13.122 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0968999999999999e-05 2024-09-16T14:49:13.122 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.122 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.122 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.122 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:404 3.2s0 3:421e4b92:::smithi04040151-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:13.122 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:13.122 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.214633985999999, 2024-09-16T14:49:13.123 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0139831, 2024-09-16T14:49:13.123 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.123 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:13.123 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:13.123 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:13.123 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:13.123 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 404 2024-09-16T14:49:13.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.123 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.123 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.123 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:13.124 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.124 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.124 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:13.124 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.124 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.124 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883617+0000", 2024-09-16T14:49:13.124 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:13.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.124 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.125 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883622+0000", 2024-09-16T14:49:13.125 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7489999999999998e-06 2024-09-16T14:49:13.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.125 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.125 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883631+0000", 2024-09-16T14:49:13.125 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6489999999999997e-06 2024-09-16T14:49:13.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.125 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.125 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883639+0000", 2024-09-16T14:49:13.125 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2120000000000003e-06 2024-09-16T14:49:13.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.126 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.126 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883930+0000", 2024-09-16T14:49:13.126 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029138200000000002 2024-09-16T14:49:13.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.126 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:13.126 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883961+0000", 2024-09-16T14:49:13.126 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0190999999999999e-05 2024-09-16T14:49:13.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.126 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.126 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897411+0000", 2024-09-16T14:49:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013450037999999999 2024-09-16T14:49:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897506+0000", 2024-09-16T14:49:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5785000000000005e-05 2024-09-16T14:49:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897601+0000", 2024-09-16T14:49:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4868e-05 2024-09-16T14:49:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:461 3.2s0 3:421e4b92:::smithi04040151-11:head [rollback 10] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.104643942999999, 2024-09-16T14:49:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011419016000000001, 2024-09-16T14:49:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 461 2024-09-16T14:49:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993611+0000", 2024-09-16T14:49:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6440000000000002e-06 2024-09-16T14:49:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993619+0000", 2024-09-16T14:49:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.199e-06 2024-09-16T14:49:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993624+0000", 2024-09-16T14:49:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.8790000000000002e-06 2024-09-16T14:49:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994734+0000", 2024-09-16T14:49:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0011103339999999999 2024-09-16T14:49:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994830+0000", 2024-09-16T14:49:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6125000000000005e-05 2024-09-16T14:49:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995335+0000", 2024-09-16T14:49:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00050461799999999997 2024-09-16T14:49:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.001584+0000", 2024-09-16T14:49:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0062489609999999999 2024-09-16T14:49:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004966+0000", 2024-09-16T14:49:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.003382674 2024-09-16T14:49:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.005027+0000", 2024-09-16T14:49:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0974000000000003e-05 2024-09-16T14:49:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:470 3.2s0 3:4ee78709:::smithi04040151-40:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.102389478999999, 2024-09-16T14:49:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011056359999999999, 2024-09-16T14:49:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 470 2024-09-16T14:49:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995862+0000", 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995865+0000", 2024-09-16T14:49:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2440000000000002e-06 2024-09-16T14:49:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995872+0000", 2024-09-16T14:49:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5640000000000002e-06 2024-09-16T14:49:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995877+0000", 2024-09-16T14:49:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1680000000000003e-06 2024-09-16T14:49:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.996768+0000", 2024-09-16T14:49:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00089129699999999999 2024-09-16T14:49:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.996867+0000", 2024-09-16T14:49:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8259999999999998e-05 2024-09-16T14:49:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.006919+0000", 2024-09-16T14:49:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010052284 2024-09-16T14:49:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:471 3.2s0 3:4ee78709:::smithi04040151-40:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.102370753000001, 2024-09-16T14:49:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012013398999999999, 2024-09-16T14:49:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 471 2024-09-16T14:49:13.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.139 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.139 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.139 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:13.139 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.139 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.139 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:13.139 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.139 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.140 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:13.140 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.140 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.140 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995884+0000", 2024-09-16T14:49:13.140 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:49:13.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.140 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.140 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995891+0000", 2024-09-16T14:49:13.140 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.72e-06 2024-09-16T14:49:13.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.141 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.141 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995896+0000", 2024-09-16T14:49:13.141 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1889999999999998e-06 2024-09-16T14:49:13.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.141 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.141 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997239+0000", 2024-09-16T14:49:13.141 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0013430460000000001 2024-09-16T14:49:13.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.141 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:13.141 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997352+0000", 2024-09-16T14:49:13.141 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011287500000000001 2024-09-16T14:49:13.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.142 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.142 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007895+0000", 2024-09-16T14:49:13.142 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010543113999999999 2024-09-16T14:49:13.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.142 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.142 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s4 52/27 ECSubWrite(tid=183, reqid=osd.11.0:183, at_version=52'60, trim_to=0'0, roll_forward_to=51'56))", 2024-09-16T14:49:13.142 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:13.142 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.210981882, 2024-09-16T14:49:13.142 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011901715, 2024-09-16T14:49:13.143 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.143 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.143 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.143 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.143 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:13.143 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.143 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.143 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:13.143 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.144 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.144 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:13.144 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.144 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.144 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887292+0000", 2024-09-16T14:49:13.144 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1920999999999999e-05 2024-09-16T14:49:13.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.144 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.144 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887303+0000", 2024-09-16T14:49:13.145 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1472e-05 2024-09-16T14:49:13.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.145 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.145 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887311+0000", 2024-09-16T14:49:13.145 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:13.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.145 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.145 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887360+0000", 2024-09-16T14:49:13.145 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9002999999999998e-05 2024-09-16T14:49:13.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.145 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.146 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887369+0000", 2024-09-16T14:49:13.146 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9299999999999992e-06 2024-09-16T14:49:13.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.146 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.146 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.899136+0000", 2024-09-16T14:49:13.146 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011767123000000001 2024-09-16T14:49:13.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.146 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.146 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.899172+0000", 2024-09-16T14:49:13.146 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6161000000000002e-05 2024-09-16T14:49:13.147 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.147 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.147 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.147 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds5 52/27 ECSubWrite(tid=214, reqid=osd.7.0:214, at_version=52'35, trim_to=0'0, roll_forward_to=51'31))", 2024-09-16T14:49:13.147 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:13.147 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.210202718, 2024-09-16T14:49:13.147 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010846751, 2024-09-16T14:49:13.147 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.148 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.148 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.148 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.148 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:13.148 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.148 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.148 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:13.148 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.149 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.149 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:13.149 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.149 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.149 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888072+0000", 2024-09-16T14:49:13.149 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3008000000000001e-05 2024-09-16T14:49:13.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.149 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.149 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888082+0000", 2024-09-16T14:49:13.149 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0129000000000001e-05 2024-09-16T14:49:13.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.150 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.150 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888090+0000", 2024-09-16T14:49:13.150 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7510000000000008e-06 2024-09-16T14:49:13.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.150 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.150 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889906+0000", 2024-09-16T14:49:13.150 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001816603 2024-09-16T14:49:13.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.150 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.150 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889931+0000", 2024-09-16T14:49:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4165e-05 2024-09-16T14:49:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.898870+0000", 2024-09-16T14:49:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089394299999999999 2024-09-16T14:49:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.898896+0000", 2024-09-16T14:49:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6188999999999999e-05 2024-09-16T14:49:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.151 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.152 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s4 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:13.152 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.931930+0000", 2024-09-16T14:49:13.152 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.166322085, 2024-09-16T14:49:13.152 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014515299000000001, 2024-09-16T14:49:13.152 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.152 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.152 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.152 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.152 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931930+0000", 2024-09-16T14:49:13.152 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.153 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.153 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931930+0000", 2024-09-16T14:49:13.153 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.153 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.153 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931929+0000", 2024-09-16T14:49:13.153 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:13.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.153 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.153 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931950+0000", 2024-09-16T14:49:13.154 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0661000000000001e-05 2024-09-16T14:49:13.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.154 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.154 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931962+0000", 2024-09-16T14:49:13.154 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1949e-05 2024-09-16T14:49:13.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.154 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.154 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931970+0000", 2024-09-16T14:49:13.154 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9510000000000006e-06 2024-09-16T14:49:13.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.154 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.155 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932022+0000", 2024-09-16T14:49:13.155 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1990000000000002e-05 2024-09-16T14:49:13.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.155 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.155 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932032+0000", 2024-09-16T14:49:13.155 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0474e-05 2024-09-16T14:49:13.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.155 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.155 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946420+0000", 2024-09-16T14:49:13.155 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0143874 2024-09-16T14:49:13.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.156 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.156 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946445+0000", 2024-09-16T14:49:13.156 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5627e-05 2024-09-16T14:49:13.156 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.156 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.156 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.156 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.156 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds5 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:13.156 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932528+0000", 2024-09-16T14:49:13.156 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.165723818, 2024-09-16T14:49:13.156 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011203032999999999, 2024-09-16T14:49:13.156 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.157 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.157 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.157 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.157 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932528+0000", 2024-09-16T14:49:13.157 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.157 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.157 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932528+0000", 2024-09-16T14:49:13.157 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932527+0000", 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932559+0000", 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1553000000000001e-05 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932575+0000", 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6188000000000002e-05 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932600+0000", 2024-09-16T14:49:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4505e-05 2024-09-16T14:49:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932684+0000", 2024-09-16T14:49:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4512999999999995e-05 2024-09-16T14:49:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932694+0000", 2024-09-16T14:49:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5510000000000005e-06 2024-09-16T14:49:13.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943713+0000", 2024-09-16T14:49:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011019340000000001 2024-09-16T14:49:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943731+0000", 2024-09-16T14:49:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8260000000000001e-05 2024-09-16T14:49:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.as6 61/53 ECSubWrite(tid=225, reqid=osd.11.0:225, at_version=61'26, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932886+0000", 2024-09-16T14:49:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.1653665919999998, 2024-09-16T14:49:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014177248, 2024-09-16T14:49:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932886+0000", 2024-09-16T14:49:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932886+0000", 2024-09-16T14:49:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932885+0000", 2024-09-16T14:49:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932916+0000", 2024-09-16T14:49:13.162 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1257999999999998e-05 2024-09-16T14:49:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932934+0000", 2024-09-16T14:49:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7958999999999999e-05 2024-09-16T14:49:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932942+0000", 2024-09-16T14:49:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9189999999999999e-06 2024-09-16T14:49:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.163 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933003+0000", 2024-09-16T14:49:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1301000000000001e-05 2024-09-16T14:49:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933023+0000", 2024-09-16T14:49:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0258000000000002e-05 2024-09-16T14:49:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946638+0000", 2024-09-16T14:49:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013614924 2024-09-16T14:49:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947063+0000", 2024-09-16T14:49:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00042440400000000002 2024-09-16T14:49:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs1 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933121+0000", 2024-09-16T14:49:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.1651315709999999, 2024-09-16T14:49:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013854730000000001, 2024-09-16T14:49:13.165 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933121+0000", 2024-09-16T14:49:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933121+0000", 2024-09-16T14:49:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.166 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933120+0000", 2024-09-16T14:49:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933143+0000", 2024-09-16T14:49:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3048999999999999e-05 2024-09-16T14:49:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933158+0000", 2024-09-16T14:49:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.543e-05 2024-09-16T14:49:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933166+0000", 2024-09-16T14:49:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0839999999999992e-06 2024-09-16T14:49:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933239+0000", 2024-09-16T14:49:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2447999999999995e-05 2024-09-16T14:49:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933250+0000", 2024-09-16T14:49:13.168 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1143e-05 2024-09-16T14:49:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946560+0000", 2024-09-16T14:49:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013309642999999999 2024-09-16T14:49:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946975+0000", 2024-09-16T14:49:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00041567600000000002 2024-09-16T14:49:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s1 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933175+0000", 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.1650772629999997, 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014250087999999999, 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933175+0000", 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.170 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933175+0000", 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933174+0000", 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933196+0000", 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1577999999999999e-05 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.171 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933210+0000", 2024-09-16T14:49:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3864e-05 2024-09-16T14:49:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933217+0000", 2024-09-16T14:49:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1999999999999997e-06 2024-09-16T14:49:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933306+0000", 2024-09-16T14:49:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8985000000000003e-05 2024-09-16T14:49:13.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933315+0000", 2024-09-16T14:49:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5730000000000008e-06 2024-09-16T14:49:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947397+0000", 2024-09-16T14:49:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014082643000000001 2024-09-16T14:49:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.173 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947425+0000", 2024-09-16T14:49:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7793e-05 2024-09-16T14:49:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs1 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933463+0000", 2024-09-16T14:49:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.1647889859999996, 2024-09-16T14:49:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014016617, 2024-09-16T14:49:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.174 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933463+0000", 2024-09-16T14:49:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933463+0000", 2024-09-16T14:49:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933462+0000", 2024-09-16T14:49:13.175 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933487+0000", 2024-09-16T14:49:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4287e-05 2024-09-16T14:49:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933501+0000", 2024-09-16T14:49:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4202999999999999e-05 2024-09-16T14:49:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.176 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933509+0000", 2024-09-16T14:49:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5979999999999999e-06 2024-09-16T14:49:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940824+0000", 2024-09-16T14:49:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073150840000000003 2024-09-16T14:49:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940836+0000", 2024-09-16T14:49:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2002e-05 2024-09-16T14:49:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947393+0000", 2024-09-16T14:49:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006557223 2024-09-16T14:49:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947480+0000", 2024-09-16T14:49:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6948999999999997e-05 2024-09-16T14:49:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.178 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s1 61/53 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=61'38, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.937911+0000", 2024-09-16T14:49:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.1603411049999997, 2024-09-16T14:49:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012160381, 2024-09-16T14:49:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937911+0000", 2024-09-16T14:49:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937911+0000", 2024-09-16T14:49:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937910+0000", 2024-09-16T14:49:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937945+0000", 2024-09-16T14:49:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.4999999999999997e-05 2024-09-16T14:49:13.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937979+0000", 2024-09-16T14:49:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3309999999999998e-05 2024-09-16T14:49:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937988+0000", 2024-09-16T14:49:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8509999999999994e-06 2024-09-16T14:49:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.181 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943764+0000", 2024-09-16T14:49:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0057755419999999998 2024-09-16T14:49:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943771+0000", 2024-09-16T14:49:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8399999999999997e-06 2024-09-16T14:49:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.950049+0000", 2024-09-16T14:49:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0062787390000000002 2024-09-16T14:49:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.182 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.950071+0000", 2024-09-16T14:49:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1940999999999999e-05 2024-09-16T14:49:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s4 61/27 ECSubWrite(tid=229, reqid=osd.11.0:229, at_version=61'84, trim_to=0'0, roll_forward_to=61'82))", 2024-09-16T14:49:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.953930+0000", 2024-09-16T14:49:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.1443217209999998, 2024-09-16T14:49:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012298072, 2024-09-16T14:49:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.183 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953930+0000", 2024-09-16T14:49:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953930+0000", 2024-09-16T14:49:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.184 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953930+0000", 2024-09-16T14:49:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953948+0000", 2024-09-16T14:49:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8253999999999998e-05 2024-09-16T14:49:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953964+0000", 2024-09-16T14:49:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5101e-05 2024-09-16T14:49:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.185 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953971+0000", 2024-09-16T14:49:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8250000000000005e-06 2024-09-16T14:49:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954019+0000", 2024-09-16T14:49:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7250000000000003e-05 2024-09-16T14:49:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954029+0000", 2024-09-16T14:49:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0154e-05 2024-09-16T14:49:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966203+0000", 2024-09-16T14:49:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01217391 2024-09-16T14:49:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966228+0000", 2024-09-16T14:49:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5799000000000002e-05 2024-09-16T14:49:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es6 61/27 ECSubWrite(tid=230, reqid=osd.11.0:230, at_version=61'35, trim_to=0'0, roll_forward_to=60'33))", 2024-09-16T14:49:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.953982+0000", 2024-09-16T14:49:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.1442701690000003, 2024-09-16T14:49:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012264608, 2024-09-16T14:49:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953982+0000", 2024-09-16T14:49:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.188 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953982+0000", 2024-09-16T14:49:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953982+0000", 2024-09-16T14:49:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954000+0000", 2024-09-16T14:49:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8175000000000001e-05 2024-09-16T14:49:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954009+0000", 2024-09-16T14:49:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2280000000000004e-06 2024-09-16T14:49:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954016+0000", 2024-09-16T14:49:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6839999999999999e-06 2024-09-16T14:49:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954084+0000", 2024-09-16T14:49:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8365e-05 2024-09-16T14:49:13.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954094+0000", 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0147e-05 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966228+0000", 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012133961 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966247+0000", 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8386e-05 2024-09-16T14:49:13.191 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es6 61/27 ECSubWrite(tid=231, reqid=osd.11.0:231, at_version=61'37, trim_to=0'0, roll_forward_to=60'33))", 2024-09-16T14:49:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.954093+0000", 2024-09-16T14:49:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.1441592700000003, 2024-09-16T14:49:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012660772000000001, 2024-09-16T14:49:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:13.192 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954093+0000", 2024-09-16T14:49:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954093+0000", 2024-09-16T14:49:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954092+0000", 2024-09-16T14:49:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.193 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954118+0000", 2024-09-16T14:49:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5442000000000001e-05 2024-09-16T14:49:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954127+0000", 2024-09-16T14:49:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9560000000000003e-06 2024-09-16T14:49:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954133+0000", 2024-09-16T14:49:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5810000000000002e-06 2024-09-16T14:49:13.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954527+0000", 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000393161 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954536+0000", 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5699999999999999e-06 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966735+0000", 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012198413999999999 2024-09-16T14:49:13.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:13.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:13.196 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:13.196 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966754+0000", 2024-09-16T14:49:13.196 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9082000000000001e-05 2024-09-16T14:49:13.196 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.196 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.196 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.196 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:13.196 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:13.196 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:13.198 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:49:13.198 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:13.196+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:13.299 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:13.297+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:13.301 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:13.301 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:13.302 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:13.302 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:13.310 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:49:13.399 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:13.399+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:13.412 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:13.412 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:13.412 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:13.412 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:13.412 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:13.421 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:49:13.500 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:13.499+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:13.525 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:13.525 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:13.525 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:13.525 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:13.525 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.525 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs2 37/27 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:13.525 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:13.525 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.602128082, 2024-09-16T14:49:13.525 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010231874, 2024-09-16T14:49:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.526 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.527 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.527 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:13.527 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.527 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.527 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.527 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.527 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920725+0000", 2024-09-16T14:49:13.527 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2198999999999999e-05 2024-09-16T14:49:13.527 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.527 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.527 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.527 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920735+0000", 2024-09-16T14:49:13.527 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0122000000000001e-05 2024-09-16T14:49:13.528 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.528 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.528 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.528 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920742+0000", 2024-09-16T14:49:13.528 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7179999999999999e-06 2024-09-16T14:49:13.528 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.528 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.528 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.528 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921079+0000", 2024-09-16T14:49:13.528 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00033676000000000002 2024-09-16T14:49:13.528 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.528 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.529 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.529 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921088+0000", 2024-09-16T14:49:13.529 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9290000000000004e-06 2024-09-16T14:49:13.529 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.529 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.529 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.529 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930916+0000", 2024-09-16T14:49:13.529 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0098279749999999992 2024-09-16T14:49:13.529 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.529 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.529 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.529 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930935+0000", 2024-09-16T14:49:13.529 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9510000000000001e-05 2024-09-16T14:49:13.529 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.530 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.530 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.530 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.530 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.530 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.fs7 37/27 ECSubWrite(tid=189, reqid=osd.7.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'12))", 2024-09-16T14:49:13.530 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:13.530 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.601940382999999, 2024-09-16T14:49:13.530 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010007947999999999, 2024-09-16T14:49:13.530 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.530 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.530 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.530 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.530 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.530 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:13.531 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.531 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.531 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.531 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.531 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:13.531 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.531 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.531 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.531 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.531 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920890+0000", 2024-09-16T14:49:13.531 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:13.531 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.531 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.531 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.532 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920900+0000", 2024-09-16T14:49:13.532 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9839999999999996e-06 2024-09-16T14:49:13.532 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.532 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.532 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.532 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920910+0000", 2024-09-16T14:49:13.532 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4220000000000006e-06 2024-09-16T14:49:13.532 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.532 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.532 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.532 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920917+0000", 2024-09-16T14:49:13.532 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5349999999999999e-06 2024-09-16T14:49:13.533 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.533 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.533 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.533 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920967+0000", 2024-09-16T14:49:13.533 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9348000000000003e-05 2024-09-16T14:49:13.533 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.533 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.533 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.533 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920976+0000", 2024-09-16T14:49:13.533 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8810000000000002e-06 2024-09-16T14:49:13.533 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.533 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.533 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.534 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930871+0000", 2024-09-16T14:49:13.534 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0098950500000000007 2024-09-16T14:49:13.534 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.534 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.534 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.534 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930899+0000", 2024-09-16T14:49:13.534 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.847e-05 2024-09-16T14:49:13.534 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.534 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.534 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.534 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.534 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.534 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s3 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:13.534 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.600943341000001, 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011669621999999999, 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.535 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921887+0000", 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921911+0000", 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4012999999999999e-05 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921923+0000", 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1642e-05 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.536 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921932+0000", 2024-09-16T14:49:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0210000000000005e-06 2024-09-16T14:49:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921988+0000", 2024-09-16T14:49:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5822999999999997e-05 2024-09-16T14:49:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921997+0000", 2024-09-16T14:49:13.537 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7900000000000005e-06 2024-09-16T14:49:13.538 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.538 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.538 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.538 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933539+0000", 2024-09-16T14:49:13.538 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011542248 2024-09-16T14:49:13.538 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.538 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.538 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.538 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933558+0000", 2024-09-16T14:49:13.538 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8964999999999999e-05 2024-09-16T14:49:13.538 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.538 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.538 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.538 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.539 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.539 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds6 37/27 ECSubWrite(tid=190, reqid=osd.7.0:188, at_version=0'0, trim_to=0'0, roll_forward_to=37'20))", 2024-09-16T14:49:13.539 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:13.539 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.600861740999999, 2024-09-16T14:49:13.539 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0090799120000000007, 2024-09-16T14:49:13.539 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.539 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.539 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.539 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.539 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.539 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:13.539 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.539 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.540 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.540 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.540 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:13.540 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.540 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.540 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.540 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.540 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921969+0000", 2024-09-16T14:49:13.540 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:13.540 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.540 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.540 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.540 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921983+0000", 2024-09-16T14:49:13.540 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4403000000000001e-05 2024-09-16T14:49:13.541 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.541 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.541 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.541 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921996+0000", 2024-09-16T14:49:13.541 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.259e-05 2024-09-16T14:49:13.541 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.541 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.541 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.541 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922005+0000", 2024-09-16T14:49:13.541 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9080000000000001e-06 2024-09-16T14:49:13.541 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.541 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.541 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.541 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922073+0000", 2024-09-16T14:49:13.542 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8127999999999996e-05 2024-09-16T14:49:13.542 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.542 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.542 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.542 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922085+0000", 2024-09-16T14:49:13.542 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1712999999999999e-05 2024-09-16T14:49:13.542 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.542 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.542 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.542 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931031+0000", 2024-09-16T14:49:13.542 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089464669999999996 2024-09-16T14:49:13.542 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.542 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.542 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.543 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931050+0000", 2024-09-16T14:49:13.543 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8471999999999999e-05 2024-09-16T14:49:13.543 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.543 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.543 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.543 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.543 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.543 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.6s2 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:49:13.543 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.923513+0000", 2024-09-16T14:49:13.543 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.599318786000001, 2024-09-16T14:49:13.543 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0075220499999999997, 2024-09-16T14:49:13.543 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.543 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.543 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.544 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.544 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.544 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923513+0000", 2024-09-16T14:49:13.544 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.544 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.544 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.544 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.544 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923513+0000", 2024-09-16T14:49:13.544 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.544 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.544 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.544 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.544 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923512+0000", 2024-09-16T14:49:13.545 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:13.545 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.545 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.545 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.545 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923534+0000", 2024-09-16T14:49:13.545 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1892999999999999e-05 2024-09-16T14:49:13.545 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.545 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.545 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.545 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923551+0000", 2024-09-16T14:49:13.545 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7127999999999999e-05 2024-09-16T14:49:13.545 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.545 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923559+0000", 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6400000000000003e-06 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923615+0000", 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5560000000000003e-05 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923627+0000", 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2296999999999999e-05 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.546 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931010+0000", 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.007382258 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931035+0000", 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5194e-05 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as1 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:13.548 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.598766491999999, 2024-09-16T14:49:13.548 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0095052339999999996, 2024-09-16T14:49:13.548 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.548 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.548 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.548 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.548 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.548 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:13.548 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.548 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.548 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.548 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.548 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:13.548 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.549 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.549 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.549 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.549 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:13.549 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:13.549 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.549 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.549 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.549 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924089+0000", 2024-09-16T14:49:13.549 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3768999999999999e-05 2024-09-16T14:49:13.549 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.549 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.549 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.549 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924101+0000", 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2294e-05 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924109+0000", 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7389999999999999e-06 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924163+0000", 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.4052999999999999e-05 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.550 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924172+0000", 2024-09-16T14:49:13.551 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7189999999999996e-06 2024-09-16T14:49:13.551 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.551 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.551 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.551 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933561+0000", 2024-09-16T14:49:13.551 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0093884659999999998 2024-09-16T14:49:13.551 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.551 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.551 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.551 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933570+0000", 2024-09-16T14:49:13.551 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4120000000000007e-06 2024-09-16T14:49:13.551 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.551 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.552 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.552 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.552 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.552 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.es3 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:13.552 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:13.552 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.598709057000001, 2024-09-16T14:49:13.552 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0094028089999999998, 2024-09-16T14:49:13.552 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.552 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.552 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.552 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.552 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.552 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:13.552 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.553 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.553 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.553 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.553 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:13.553 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.553 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.553 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.553 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.553 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:13.553 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.553 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.553 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.553 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.554 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924142+0000", 2024-09-16T14:49:13.554 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0162000000000001e-05 2024-09-16T14:49:13.554 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.554 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.554 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.554 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924150+0000", 2024-09-16T14:49:13.554 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0919999999999998e-06 2024-09-16T14:49:13.554 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.554 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.554 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.554 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924157+0000", 2024-09-16T14:49:13.554 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6069999999999996e-06 2024-09-16T14:49:13.554 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.554 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.555 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.555 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924215+0000", 2024-09-16T14:49:13.555 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8192000000000001e-05 2024-09-16T14:49:13.555 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.555 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.555 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.555 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924225+0000", 2024-09-16T14:49:13.555 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9760000000000007e-06 2024-09-16T14:49:13.555 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.555 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.555 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.555 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933494+0000", 2024-09-16T14:49:13.555 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0092692720000000003 2024-09-16T14:49:13.555 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.556 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.556 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.556 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933525+0000", 2024-09-16T14:49:13.556 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0976999999999998e-05 2024-09-16T14:49:13.556 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.556 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.556 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.556 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.556 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.556 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s3 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:13.556 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932044+0000", 2024-09-16T14:49:13.556 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.5907875369999998, 2024-09-16T14:49:13.556 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0086059629999999995, 2024-09-16T14:49:13.556 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.557 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.557 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.557 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.557 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.557 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932044+0000", 2024-09-16T14:49:13.557 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.557 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.557 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.557 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.557 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932044+0000", 2024-09-16T14:49:13.557 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.557 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.557 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.557 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.558 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932043+0000", 2024-09-16T14:49:13.558 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.558 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.558 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.558 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.558 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932067+0000", 2024-09-16T14:49:13.558 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3501000000000001e-05 2024-09-16T14:49:13.558 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.558 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.558 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.558 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932078+0000", 2024-09-16T14:49:13.558 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1151999999999999e-05 2024-09-16T14:49:13.558 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.559 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.559 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.559 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932085+0000", 2024-09-16T14:49:13.559 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4019999999999996e-06 2024-09-16T14:49:13.559 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.559 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.559 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.559 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932134+0000", 2024-09-16T14:49:13.559 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8698e-05 2024-09-16T14:49:13.559 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.559 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.559 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.559 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932144+0000", 2024-09-16T14:49:13.559 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7440000000000002e-06 2024-09-16T14:49:13.560 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.560 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.560 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.560 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.940624+0000", 2024-09-16T14:49:13.560 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084803870000000007 2024-09-16T14:49:13.560 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.560 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.560 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.560 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.940650+0000", 2024-09-16T14:49:13.560 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5769999999999999e-05 2024-09-16T14:49:13.560 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.560 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.561 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.561 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.561 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.561 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds6 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:13.561 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932207+0000", 2024-09-16T14:49:13.561 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.5906244760000003, 2024-09-16T14:49:13.561 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0085983610000000005, 2024-09-16T14:49:13.561 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.561 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.561 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.561 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.561 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.561 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932207+0000", 2024-09-16T14:49:13.561 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932207+0000", 2024-09-16T14:49:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932206+0000", 2024-09-16T14:49:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932229+0000", 2024-09-16T14:49:13.563 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3074e-05 2024-09-16T14:49:13.563 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.563 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.563 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.563 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932241+0000", 2024-09-16T14:49:13.563 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1505000000000001e-05 2024-09-16T14:49:13.563 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.563 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.563 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.563 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932249+0000", 2024-09-16T14:49:13.563 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7400000000000004e-06 2024-09-16T14:49:13.563 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.563 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.563 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.564 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932295+0000", 2024-09-16T14:49:13.564 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5908999999999997e-05 2024-09-16T14:49:13.564 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.564 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.564 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.564 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932304+0000", 2024-09-16T14:49:13.564 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1249999999999999e-06 2024-09-16T14:49:13.564 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.564 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.564 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.564 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.940790+0000", 2024-09-16T14:49:13.564 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084867380000000006 2024-09-16T14:49:13.564 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.565 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.565 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.565 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.940805+0000", 2024-09-16T14:49:13.565 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4934e-05 2024-09-16T14:49:13.565 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.565 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.565 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.565 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.565 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.565 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as1 61/53 ECSubWrite(tid=225, reqid=osd.11.0:225, at_version=61'26, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:13.565 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932623+0000", 2024-09-16T14:49:13.565 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.5902086769999997, 2024-09-16T14:49:13.565 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089969119999999993, 2024-09-16T14:49:13.565 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.566 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.566 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.566 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.566 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.566 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932623+0000", 2024-09-16T14:49:13.566 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.566 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.566 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.566 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.566 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932623+0000", 2024-09-16T14:49:13.566 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.566 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.566 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.566 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.567 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932622+0000", 2024-09-16T14:49:13.567 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.567 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.567 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.567 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.567 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932643+0000", 2024-09-16T14:49:13.567 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0611e-05 2024-09-16T14:49:13.567 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.567 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.567 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.567 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932656+0000", 2024-09-16T14:49:13.567 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3613e-05 2024-09-16T14:49:13.567 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.568 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.568 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.568 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932663+0000", 2024-09-16T14:49:13.568 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0149999999999998e-06 2024-09-16T14:49:13.568 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.568 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.568 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.568 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932790+0000", 2024-09-16T14:49:13.568 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00012615099999999999 2024-09-16T14:49:13.568 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.568 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.568 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.568 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932803+0000", 2024-09-16T14:49:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3355e-05 2024-09-16T14:49:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941602+0000", 2024-09-16T14:49:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0087990859999999994 2024-09-16T14:49:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941620+0000", 2024-09-16T14:49:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7710000000000002e-05 2024-09-16T14:49:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.569 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.570 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.570 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.570 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.cs2 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:13.570 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932695+0000", 2024-09-16T14:49:13.570 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.5901360479999997, 2024-09-16T14:49:13.570 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089530259999999993, 2024-09-16T14:49:13.570 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.570 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.570 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.570 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.570 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.570 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932695+0000", 2024-09-16T14:49:13.570 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932695+0000", 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932695+0000", 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932711+0000", 2024-09-16T14:49:13.571 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6248000000000001e-05 2024-09-16T14:49:13.572 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.572 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.572 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.572 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932721+0000", 2024-09-16T14:49:13.572 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7489999999999993e-06 2024-09-16T14:49:13.572 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.572 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.572 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.572 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932728+0000", 2024-09-16T14:49:13.572 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:49:13.572 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.572 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.572 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.573 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932815+0000", 2024-09-16T14:49:13.573 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6984999999999995e-05 2024-09-16T14:49:13.573 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.573 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.573 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.573 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932825+0000", 2024-09-16T14:49:13.573 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0459999999999994e-06 2024-09-16T14:49:13.573 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.573 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.573 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.573 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941636+0000", 2024-09-16T14:49:13.573 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0088109880000000005 2024-09-16T14:49:13.573 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.573 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941649+0000", 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2989e-05 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.5s6 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932903+0000", 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.589928177, 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0087662160000000003, 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.574 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.575 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.575 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.575 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932903+0000", 2024-09-16T14:49:13.575 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.575 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.575 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.575 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.575 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932903+0000", 2024-09-16T14:49:13.575 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.575 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.575 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.575 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.575 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932903+0000", 2024-09-16T14:49:13.575 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.576 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.576 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.576 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.576 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932917+0000", 2024-09-16T14:49:13.576 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4443e-05 2024-09-16T14:49:13.576 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.576 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.576 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.576 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932927+0000", 2024-09-16T14:49:13.576 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8910000000000007e-06 2024-09-16T14:49:13.576 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.576 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.576 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.577 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932934+0000", 2024-09-16T14:49:13.577 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.985e-06 2024-09-16T14:49:13.577 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.577 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.577 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.577 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932978+0000", 2024-09-16T14:49:13.577 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3730000000000003e-05 2024-09-16T14:49:13.577 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.577 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.577 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.577 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932987+0000", 2024-09-16T14:49:13.577 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3940000000000002e-06 2024-09-16T14:49:13.577 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.577 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941659+0000", 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0086717840000000001 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941670+0000", 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0654e-05 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.cs2 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:13.578 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.933500+0000", 2024-09-16T14:49:13.579 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.5893313259999999, 2024-09-16T14:49:13.579 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0087068370000000003, 2024-09-16T14:49:13.579 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.579 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.579 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.579 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.579 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.579 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933500+0000", 2024-09-16T14:49:13.579 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.579 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.579 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.579 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.579 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933500+0000", 2024-09-16T14:49:13.579 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.580 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.580 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.580 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933499+0000", 2024-09-16T14:49:13.580 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:13.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.580 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.580 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.581 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933519+0000", 2024-09-16T14:49:13.581 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9932000000000002e-05 2024-09-16T14:49:13.581 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.581 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.581 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.581 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933541+0000", 2024-09-16T14:49:13.581 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1698000000000001e-05 2024-09-16T14:49:13.581 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.581 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.581 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.581 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933554+0000", 2024-09-16T14:49:13.581 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2995e-05 2024-09-16T14:49:13.581 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.581 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.582 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.582 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935629+0000", 2024-09-16T14:49:13.582 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002075061 2024-09-16T14:49:13.582 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.582 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.582 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.582 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935637+0000", 2024-09-16T14:49:13.582 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4079999999999998e-06 2024-09-16T14:49:13.582 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.582 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.582 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.582 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942191+0000", 2024-09-16T14:49:13.583 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0065539630000000003 2024-09-16T14:49:13.583 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.583 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.583 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.583 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942207+0000", 2024-09-16T14:49:13.583 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5622999999999999e-05 2024-09-16T14:49:13.583 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.583 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.583 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.583 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.583 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.583 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:594 3.3s0 3:c9d06351:::smithi04040151-33:head [append 0~491520] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:13.583 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.981536+0000", 2024-09-16T14:49:13.584 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 1.5412952849999999, 2024-09-16T14:49:13.584 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0295208730000001, 2024-09-16T14:49:13.584 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.584 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:13.584 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:13.584 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:13.584 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:13.584 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 594 2024-09-16T14:49:13.584 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.584 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.584 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.584 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.584 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.981536+0000", 2024-09-16T14:49:13.584 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.585 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.585 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.585 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.585 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.981536+0000", 2024-09-16T14:49:13.585 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.585 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.585 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.585 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.585 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.981535+0000", 2024-09-16T14:49:13.585 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:13.585 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.585 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.585 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.586 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982118+0000", 2024-09-16T14:49:13.586 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00058284500000000002 2024-09-16T14:49:13.586 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.586 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.586 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.586 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982129+0000", 2024-09-16T14:49:13.586 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1644999999999999e-05 2024-09-16T14:49:13.586 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.586 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.586 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.586 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982134+0000", 2024-09-16T14:49:13.586 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.152e-06 2024-09-16T14:49:13.586 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.586 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.587 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.587 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982188+0000", 2024-09-16T14:49:13.587 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3829999999999998e-05 2024-09-16T14:49:13.587 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.587 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.587 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:13.587 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982192+0000", 2024-09-16T14:49:13.587 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5980000000000001e-06 2024-09-16T14:49:13.587 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.587 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.587 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.587 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980648+0000", 2024-09-16T14:49:13.587 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99845620300000004 2024-09-16T14:49:13.587 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.588 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.588 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:13.588 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980918+0000", 2024-09-16T14:49:13.588 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00026981999999999999 2024-09-16T14:49:13.588 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.588 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.588 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.588 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.002491+0000", 2024-09-16T14:49:13.588 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.021573093000000002 2024-09-16T14:49:13.588 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.588 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.588 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:13.588 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.002715+0000", 2024-09-16T14:49:13.589 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00022431799999999999 2024-09-16T14:49:13.589 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.589 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.589 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.589 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004518+0000", 2024-09-16T14:49:13.589 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001802339 2024-09-16T14:49:13.589 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.589 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.589 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.589 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009950+0000", 2024-09-16T14:49:13.589 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0054329260000000002 2024-09-16T14:49:13.589 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.590 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.590 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:13.590 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.010633+0000", 2024-09-16T14:49:13.590 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00068230700000000005 2024-09-16T14:49:13.590 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.590 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.590 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.590 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.011057+0000", 2024-09-16T14:49:13.590 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000424313 2024-09-16T14:49:13.590 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.590 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.590 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.590 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.591 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.591 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:595 3.3s0 3:c9d06351:::smithi04040151-33:head [append 0~589824] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:13.591 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.985758+0000", 2024-09-16T14:49:13.591 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 1.5370733570000001, 2024-09-16T14:49:13.591 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0310086730000001, 2024-09-16T14:49:13.591 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.591 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:13.591 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:13.591 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:13.591 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:13.591 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 595 2024-09-16T14:49:13.591 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.591 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.592 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.592 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.592 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.985758+0000", 2024-09-16T14:49:13.592 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.592 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.592 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.592 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.592 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.985758+0000", 2024-09-16T14:49:13.592 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.592 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.592 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.592 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.592 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.985757+0000", 2024-09-16T14:49:13.593 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:13.593 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.593 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.593 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.593 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986224+0000", 2024-09-16T14:49:13.593 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00046679099999999998 2024-09-16T14:49:13.593 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.593 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.593 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.593 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986234+0000", 2024-09-16T14:49:13.593 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0614999999999999e-05 2024-09-16T14:49:13.593 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.593 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.594 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.594 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986239+0000", 2024-09-16T14:49:13.594 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0000000000000004e-06 2024-09-16T14:49:13.594 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.594 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.594 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.594 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986272+0000", 2024-09-16T14:49:13.594 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2861999999999998e-05 2024-09-16T14:49:13.594 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.594 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.594 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:13.594 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986274+0000", 2024-09-16T14:49:13.594 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.013e-06 2024-09-16T14:49:13.595 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.595 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.595 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.595 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.981881+0000", 2024-09-16T14:49:13.595 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99560661699999997 2024-09-16T14:49:13.595 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.595 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.595 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:13.595 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.981947+0000", 2024-09-16T14:49:13.595 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6390000000000006e-05 2024-09-16T14:49:13.595 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.595 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.596 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.596 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005200+0000", 2024-09-16T14:49:13.596 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.023253165999999999 2024-09-16T14:49:13.596 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.596 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.596 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:13.596 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005354+0000", 2024-09-16T14:49:13.596 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000153611 2024-09-16T14:49:13.596 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.596 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.596 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.596 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007291+0000", 2024-09-16T14:49:13.597 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001936716 2024-09-16T14:49:13.597 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.597 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.597 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.597 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009953+0000", 2024-09-16T14:49:13.597 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0026618190000000002 2024-09-16T14:49:13.597 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.597 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.597 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:13.597 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.016469+0000", 2024-09-16T14:49:13.597 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0065166629999999998 2024-09-16T14:49:13.597 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.598 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.598 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.598 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.016767+0000", 2024-09-16T14:49:13.598 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00029760600000000001 2024-09-16T14:49:13.598 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.598 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.598 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.598 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.598 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.598 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:596 3.3s0 3:c9d06351:::smithi04040151-33:head [append 0~557056] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:13.598 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.989693+0000", 2024-09-16T14:49:13.598 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 1.5331388019999999, 2024-09-16T14:49:13.598 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.027510892, 2024-09-16T14:49:13.598 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 596 2024-09-16T14:49:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.989693+0000", 2024-09-16T14:49:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.599 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.600 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.989693+0000", 2024-09-16T14:49:13.600 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.600 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.600 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.600 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.600 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.989692+0000", 2024-09-16T14:49:13.600 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.600 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.600 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.600 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.600 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990180+0000", 2024-09-16T14:49:13.600 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00048803300000000002 2024-09-16T14:49:13.601 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.601 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.601 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.601 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990189+0000", 2024-09-16T14:49:13.601 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0440000000000001e-06 2024-09-16T14:49:13.601 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.601 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.601 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.601 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990194+0000", 2024-09-16T14:49:13.601 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8369999999999996e-06 2024-09-16T14:49:13.601 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.601 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.601 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.601 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990239+0000", 2024-09-16T14:49:13.602 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4898999999999998e-05 2024-09-16T14:49:13.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.602 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.602 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:13.602 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990241+0000", 2024-09-16T14:49:13.602 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:49:13.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.602 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.602 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.602 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.981982+0000", 2024-09-16T14:49:13.602 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99174135900000004 2024-09-16T14:49:13.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.602 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.603 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:13.603 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982045+0000", 2024-09-16T14:49:13.603 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2890000000000003e-05 2024-09-16T14:49:13.603 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.603 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.603 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.603 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007979+0000", 2024-09-16T14:49:13.603 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.025933905 2024-09-16T14:49:13.603 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.603 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.603 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:13.603 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.008137+0000", 2024-09-16T14:49:13.603 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015834399999999999 2024-09-16T14:49:13.603 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.604 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.604 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.604 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009359+0000", 2024-09-16T14:49:13.604 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0012212499999999999 2024-09-16T14:49:13.604 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.604 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.604 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.604 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014320+0000", 2024-09-16T14:49:13.604 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049609850000000002 2024-09-16T14:49:13.604 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.604 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.604 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:13.604 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.017003+0000", 2024-09-16T14:49:13.604 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002683389 2024-09-16T14:49:13.605 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.605 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.605 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.605 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.017204+0000", 2024-09-16T14:49:13.605 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00020068700000000001 2024-09-16T14:49:13.605 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.605 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.605 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.605 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.605 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.605 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:597 3.3s0 3:c9d06351:::smithi04040151-33:head [append 0~16384] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:13.605 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.990199+0000", 2024-09-16T14:49:13.605 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 1.5326324570000001, 2024-09-16T14:49:13.605 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.029187326, 2024-09-16T14:49:13.606 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.606 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:13.606 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:13.606 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:13.606 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:13.606 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 597 2024-09-16T14:49:13.606 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.606 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.606 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.606 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.606 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990199+0000", 2024-09-16T14:49:13.606 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.606 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.606 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.607 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.607 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990199+0000", 2024-09-16T14:49:13.607 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.607 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.607 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.607 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.607 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990199+0000", 2024-09-16T14:49:13.607 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.607 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.607 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.607 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.607 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990216+0000", 2024-09-16T14:49:13.607 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7176999999999998e-05 2024-09-16T14:49:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990220+0000", 2024-09-16T14:49:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4739999999999999e-06 2024-09-16T14:49:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990224+0000", 2024-09-16T14:49:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3799999999999998e-06 2024-09-16T14:49:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.609 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990262+0000", 2024-09-16T14:49:13.609 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8461999999999999e-05 2024-09-16T14:49:13.609 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.609 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.609 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:13.609 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990264+0000", 2024-09-16T14:49:13.609 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6330000000000001e-06 2024-09-16T14:49:13.609 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.609 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.609 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.609 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982110+0000", 2024-09-16T14:49:13.609 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99184635300000001 2024-09-16T14:49:13.609 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.609 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982190+0000", 2024-09-16T14:49:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0255999999999999e-05 2024-09-16T14:49:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.011434+0000", 2024-09-16T14:49:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.029243331000000001 2024-09-16T14:49:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.011595+0000", 2024-09-16T14:49:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00016157699999999999 2024-09-16T14:49:13.610 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012441+0000", 2024-09-16T14:49:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00084541900000000003 2024-09-16T14:49:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.016197+0000", 2024-09-16T14:49:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003756073 2024-09-16T14:49:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.019019+0000", 2024-09-16T14:49:13.611 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00282168 2024-09-16T14:49:13.612 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.612 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.612 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.612 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.019386+0000", 2024-09-16T14:49:13.612 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00036785099999999999 2024-09-16T14:49:13.612 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.612 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.612 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.612 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.612 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.612 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:598 3.3s0 3:c9d06351:::smithi04040151-33:head [setxattr _header (57) in=7b] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:13.612 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.990235+0000", 2024-09-16T14:49:13.613 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 1.5325968969999999, 2024-09-16T14:49:13.613 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.029738257, 2024-09-16T14:49:13.613 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.613 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:13.613 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:13.613 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:13.613 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:13.613 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 598 2024-09-16T14:49:13.613 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.613 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.613 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.613 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.613 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990235+0000", 2024-09-16T14:49:13.613 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.614 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.614 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.614 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.614 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990235+0000", 2024-09-16T14:49:13.614 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.614 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.614 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.614 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.614 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990234+0000", 2024-09-16T14:49:13.614 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.614 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.614 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.614 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990237+0000", 2024-09-16T14:49:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:49:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990240+0000", 2024-09-16T14:49:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.625e-06 2024-09-16T14:49:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990243+0000", 2024-09-16T14:49:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.9000000000000002e-06 2024-09-16T14:49:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.616 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.616 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990290+0000", 2024-09-16T14:49:13.616 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7154000000000003e-05 2024-09-16T14:49:13.616 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.616 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.616 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:13.616 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990292+0000", 2024-09-16T14:49:13.616 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.237e-06 2024-09-16T14:49:13.616 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.616 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.616 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.616 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982229+0000", 2024-09-16T14:49:13.616 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.991937069 2024-09-16T14:49:13.617 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.617 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.617 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:13.617 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982316+0000", 2024-09-16T14:49:13.617 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7495000000000002e-05 2024-09-16T14:49:13.617 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.617 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.617 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.617 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012980+0000", 2024-09-16T14:49:13.617 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.030663620999999999 2024-09-16T14:49:13.617 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.617 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.617 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:13.617 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013119+0000", 2024-09-16T14:49:13.618 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00013920800000000001 2024-09-16T14:49:13.618 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.618 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.618 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:13.618 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013756+0000", 2024-09-16T14:49:13.618 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00063710700000000004 2024-09-16T14:49:13.618 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.618 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.618 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:13.618 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.016198+0000", 2024-09-16T14:49:13.618 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002442222 2024-09-16T14:49:13.618 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.618 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.618 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:13.619 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.019618+0000", 2024-09-16T14:49:13.619 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003420083 2024-09-16T14:49:13.619 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.619 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.619 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.619 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.019973+0000", 2024-09-16T14:49:13.619 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00035448499999999998 2024-09-16T14:49:13.619 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.619 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.619 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.619 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.619 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.619 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:599 3.3s0 3:c9d06351:::smithi04040151-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:13.619 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.990261+0000", 2024-09-16T14:49:13.620 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 1.5325703019999999, 2024-09-16T14:49:13.620 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.034782844, 2024-09-16T14:49:13.620 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.620 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:13.620 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:13.620 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:13.620 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:13.620 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 599 2024-09-16T14:49:13.620 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.620 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.620 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.620 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.620 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990261+0000", 2024-09-16T14:49:13.620 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.621 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.621 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.621 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.621 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990261+0000", 2024-09-16T14:49:13.621 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.621 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.621 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.621 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.621 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990261+0000", 2024-09-16T14:49:13.621 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:13.621 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.621 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.621 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.621 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990263+0000", 2024-09-16T14:49:13.622 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8059999999999999e-06 2024-09-16T14:49:13.622 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.622 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.622 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.622 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990267+0000", 2024-09-16T14:49:13.622 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8850000000000001e-06 2024-09-16T14:49:13.622 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.622 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.622 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.622 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990269+0000", 2024-09-16T14:49:13.622 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7839999999999999e-06 2024-09-16T14:49:13.622 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.622 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.623 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.623 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990311+0000", 2024-09-16T14:49:13.623 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1971000000000002e-05 2024-09-16T14:49:13.623 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.623 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.623 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:13.623 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990312+0000", 2024-09-16T14:49:13.623 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2e-07 2024-09-16T14:49:13.623 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.623 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.623 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.623 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982653+0000", 2024-09-16T14:49:13.623 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99234119499999995 2024-09-16T14:49:13.623 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.624 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.624 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:13.624 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982733+0000", 2024-09-16T14:49:13.624 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9537999999999996e-05 2024-09-16T14:49:13.624 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.624 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.624 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.624 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014037+0000", 2024-09-16T14:49:13.624 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.031304580999999998 2024-09-16T14:49:13.624 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.624 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.624 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:13.624 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014200+0000", 2024-09-16T14:49:13.624 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00016317999999999999 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.020590+0000", 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0063896650000000001 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.020693+0000", 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010270800000000001 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.025044+0000", 2024-09-16T14:49:13.625 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.004351293 2024-09-16T14:49:13.626 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.626 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.626 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.626 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.626 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.626 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDPGRecoveryDeleteReply(3.3s0 e66,65 3:c09a69fe:::smithi04040151-5:2,35'7)", 2024-09-16T14:49:13.626 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.005793+0000", 2024-09-16T14:49:13.626 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.51703827000000002, 2024-09-16T14:49:13.626 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0091592329999999993, 2024-09-16T14:49:13.626 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:13.626 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:13.626 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:13.626 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.627 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:13.627 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005793+0000", 2024-09-16T14:49:13.627 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.627 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.627 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.627 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:13.627 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005793+0000", 2024-09-16T14:49:13.627 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:13.627 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.627 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.627 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:13.627 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005792+0000", 2024-09-16T14:49:13.627 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:13.627 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.628 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.628 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:13.628 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005800+0000", 2024-09-16T14:49:13.628 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0309999999999998e-06 2024-09-16T14:49:13.628 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.628 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.628 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:13.628 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005813+0000", 2024-09-16T14:49:13.628 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.287e-05 2024-09-16T14:49:13.628 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.628 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.628 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:13.628 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005822+0000", 2024-09-16T14:49:13.628 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3920000000000003e-06 2024-09-16T14:49:13.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.629 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.629 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:13.629 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014351+0000", 2024-09-16T14:49:13.629 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0085288429999999995 2024-09-16T14:49:13.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:13.629 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:13.629 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:13.629 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014952+0000", 2024-09-16T14:49:13.629 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00060190500000000002 2024-09-16T14:49:13.629 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.629 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.629 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.629 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:13.630 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:13.630 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:13.631 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:49:13.632 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:13.630+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:13.732 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:13.730+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:13.733 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:13.733 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:13.734 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:13.734 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:13.742 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:49:13.833 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:13.832+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:13.857 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:49:13.886 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:13.886 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:13.886 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:13.886 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:13.886 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:13.894 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:49:13.934 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:13.932+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:13.960 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is unset 2024-09-16T14:49:13.961 INFO:tasks.rados.rados.0.smithi040.stdout:243: done (5 left) 2024-09-16T14:49:13.961 INFO:tasks.rados.rados.0.smithi040.stdout:245: done (4 left) 2024-09-16T14:49:13.961 INFO:tasks.rados.rados.0.smithi040.stdout:246: done (3 left) 2024-09-16T14:49:13.961 INFO:tasks.rados.rados.0.smithi040.stdout:248: done (2 left) 2024-09-16T14:49:13.961 INFO:tasks.rados.rados.0.smithi040.stdout:249: done (1 left) 2024-09-16T14:49:13.961 INFO:tasks.rados.rados.0.smithi040.stdout:253: done (0 left) 2024-09-16T14:49:13.961 INFO:tasks.rados.rados.0.smithi040.stdout:254: delete oid 10 current snap is 20 2024-09-16T14:49:13.965 INFO:tasks.rados.rados.0.smithi040.stdout:254: done (0 left) 2024-09-16T14:49:13.965 INFO:tasks.rados.rados.0.smithi040.stdout:255: read oid 30 snap 18 2024-09-16T14:49:13.966 INFO:tasks.rados.rados.0.smithi040.stdout:255: expect deleted 2024-09-16T14:49:13.966 INFO:tasks.rados.rados.0.smithi040.stdout:256: snap_remove snap 17 2024-09-16T14:49:13.990 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:13.990 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:13.999 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s4 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921852+0000", 2024-09-16T14:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 30.075946994999999, 2024-09-16T14:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0065836089999999998, 2024-09-16T14:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:14.000 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921852+0000", 2024-09-16T14:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921852+0000", 2024-09-16T14:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921851+0000", 2024-09-16T14:49:14.001 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921890+0000", 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8668000000000003e-05 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921906+0000", 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6772000000000001e-05 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921920+0000", 2024-09-16T14:49:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3525e-05 2024-09-16T14:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923071+0000", 2024-09-16T14:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001150812 2024-09-16T14:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923079+0000", 2024-09-16T14:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6119999999999999e-06 2024-09-16T14:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.003 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.928412+0000", 2024-09-16T14:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053327419999999997 2024-09-16T14:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.928435+0000", 2024-09-16T14:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3419999999999999e-05 2024-09-16T14:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:449 3.5s0 3:a7111aa9:::smithi04040151-45:head [stat,delete] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:14.004 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.121412386999999, 2024-09-16T14:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013571421, 2024-09-16T14:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 449 2024-09-16T14:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.005 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876385+0000", 2024-09-16T14:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.006 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876396+0000", 2024-09-16T14:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1094000000000001e-05 2024-09-16T14:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876412+0000", 2024-09-16T14:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6563e-05 2024-09-16T14:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876421+0000", 2024-09-16T14:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5189999999999992e-06 2024-09-16T14:49:14.007 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876477+0000", 2024-09-16T14:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.6679999999999999e-05 2024-09-16T14:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876487+0000", 2024-09-16T14:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4509999999999998e-06 2024-09-16T14:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.008 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.878486+0000", 2024-09-16T14:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001999635 2024-09-16T14:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.878592+0000", 2024-09-16T14:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000105398 2024-09-16T14:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.889958+0000", 2024-09-16T14:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01136595 2024-09-16T14:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.009 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:466 3.5s0 3:a29e72cc:::smithi04040151-39:head [append 0~0] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995540+0000", 2024-09-16T14:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.002258806, 2024-09-16T14:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084481160000000003, 2024-09-16T14:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 466 2024-09-16T14:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.010 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995540+0000", 2024-09-16T14:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995540+0000", 2024-09-16T14:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995538+0000", 2024-09-16T14:49:14.011 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995548+0000", 2024-09-16T14:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1570000000000006e-06 2024-09-16T14:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995560+0000", 2024-09-16T14:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2527999999999999e-05 2024-09-16T14:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.012 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995567+0000", 2024-09-16T14:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2030000000000003e-06 2024-09-16T14:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995620+0000", 2024-09-16T14:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2516000000000003e-05 2024-09-16T14:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995718+0000", 2024-09-16T14:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.781e-05 2024-09-16T14:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.013 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.996361+0000", 2024-09-16T14:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00064323600000000005 2024-09-16T14:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.999417+0000", 2024-09-16T14:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0030556120000000001 2024-09-16T14:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003945+0000", 2024-09-16T14:49:14.014 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045287950000000004 2024-09-16T14:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003988+0000", 2024-09-16T14:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2758e-05 2024-09-16T14:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:467 3.5s0 3:a29e72cc:::smithi04040151-39:head [rollback 14] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.002219829000001, 2024-09-16T14:49:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012554012, 2024-09-16T14:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 467 2024-09-16T14:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.016 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995581+0000", 2024-09-16T14:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.9009999999999998e-06 2024-09-16T14:49:14.017 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995587+0000", 2024-09-16T14:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8850000000000004e-06 2024-09-16T14:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995592+0000", 2024-09-16T14:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9749999999999998e-06 2024-09-16T14:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.996914+0000", 2024-09-16T14:49:14.018 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00132168 2024-09-16T14:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997029+0000", 2024-09-16T14:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000115203 2024-09-16T14:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997570+0000", 2024-09-16T14:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00054107999999999999 2024-09-16T14:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.019 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003265+0000", 2024-09-16T14:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0056945679999999997 2024-09-16T14:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008067+0000", 2024-09-16T14:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048018039999999998 2024-09-16T14:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008133+0000", 2024-09-16T14:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6365999999999994e-05 2024-09-16T14:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.020 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:468 3.5s0 3:a29e72cc:::smithi04040151-39:head [append 0~0] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.002050769, 2024-09-16T14:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012562663999999999, 2024-09-16T14:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:14.021 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 468 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.022 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995747+0000", 2024-09-16T14:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995752+0000", 2024-09-16T14:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.916e-06 2024-09-16T14:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995762+0000", 2024-09-16T14:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4609999999999997e-06 2024-09-16T14:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.023 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995768+0000", 2024-09-16T14:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8109999999999997e-06 2024-09-16T14:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.998577+0000", 2024-09-16T14:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0028088369999999998 2024-09-16T14:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.998680+0000", 2024-09-16T14:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010318399999999999 2024-09-16T14:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.024 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.999182+0000", 2024-09-16T14:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00050101099999999999 2024-09-16T14:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.005964+0000", 2024-09-16T14:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0067821490000000003 2024-09-16T14:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008246+0000", 2024-09-16T14:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0022827670000000002 2024-09-16T14:49:14.025 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008311+0000", 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.4269000000000004e-05 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 52/27 ECSubWrite(tid=185, reqid=osd.11.0:185, at_version=52'62, trim_to=0'0, roll_forward_to=52'60))", 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:58.900768+0000", 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 15.097030740999999, 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083307420000000004, 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.026 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:14.027 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.027 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.027 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.027 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900768+0000", 2024-09-16T14:49:14.027 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.027 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.027 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.027 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.027 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900768+0000", 2024-09-16T14:49:14.027 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.027 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.027 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.027 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.027 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900767+0000", 2024-09-16T14:49:14.028 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:14.028 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.028 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.028 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.028 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900790+0000", 2024-09-16T14:49:14.028 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2812000000000001e-05 2024-09-16T14:49:14.028 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.028 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.028 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.028 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900801+0000", 2024-09-16T14:49:14.028 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0954999999999999e-05 2024-09-16T14:49:14.028 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.028 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.029 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.029 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900810+0000", 2024-09-16T14:49:14.029 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5019999999999992e-06 2024-09-16T14:49:14.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.029 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.029 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.029 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900851+0000", 2024-09-16T14:49:14.029 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1845999999999999e-05 2024-09-16T14:49:14.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.029 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.029 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.029 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.900860+0000", 2024-09-16T14:49:14.029 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4630000000000001e-06 2024-09-16T14:49:14.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.030 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.030 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.030 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909074+0000", 2024-09-16T14:49:14.030 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082140349999999997 2024-09-16T14:49:14.030 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.030 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.030 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.030 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909099+0000", 2024-09-16T14:49:14.030 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4808e-05 2024-09-16T14:49:14.030 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.030 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.030 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.030 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.030 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds1 52/27 ECSubWrite(tid=215, reqid=osd.7.0:214, at_version=0'0, trim_to=0'0, roll_forward_to=52'35))", 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:58.901786+0000", 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 15.096012808999999, 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0073522930000000002, 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901786+0000", 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.031 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901786+0000", 2024-09-16T14:49:14.032 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.032 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.032 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.032 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.032 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901785+0000", 2024-09-16T14:49:14.032 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:14.032 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.032 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.032 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.032 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901796+0000", 2024-09-16T14:49:14.032 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0475000000000001e-05 2024-09-16T14:49:14.032 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.032 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901808+0000", 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2755000000000001e-05 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901817+0000", 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7409999999999998e-06 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901871+0000", 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3940000000000002e-05 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.033 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.034 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.034 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.901880+0000", 2024-09-16T14:49:14.034 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2539999999999998e-06 2024-09-16T14:49:14.034 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.034 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.034 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.034 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909107+0000", 2024-09-16T14:49:14.034 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00722684 2024-09-16T14:49:14.034 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.034 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.034 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.034 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.909138+0000", 2024-09-16T14:49:14.034 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.112e-05 2024-09-16T14:49:14.034 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.035 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.035 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.035 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.035 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.035 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:14.035 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932515+0000", 2024-09-16T14:49:14.035 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 6.0652840360000004, 2024-09-16T14:49:14.035 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010770217, 2024-09-16T14:49:14.035 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.035 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:14.035 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.035 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.035 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.036 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932515+0000", 2024-09-16T14:49:14.036 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.036 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.036 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.036 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.036 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932515+0000", 2024-09-16T14:49:14.036 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.036 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.036 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.036 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.036 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932514+0000", 2024-09-16T14:49:14.036 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:14.036 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.036 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.037 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.037 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932539+0000", 2024-09-16T14:49:14.037 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4494e-05 2024-09-16T14:49:14.037 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.037 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.037 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.037 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932549+0000", 2024-09-16T14:49:14.037 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0028e-05 2024-09-16T14:49:14.037 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.037 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.037 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.037 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932556+0000", 2024-09-16T14:49:14.037 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.6920000000000002e-06 2024-09-16T14:49:14.037 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.038 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.038 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.038 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932618+0000", 2024-09-16T14:49:14.038 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.1600000000000007e-05 2024-09-16T14:49:14.038 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.038 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.038 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.038 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932626+0000", 2024-09-16T14:49:14.038 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4139999999999994e-06 2024-09-16T14:49:14.038 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.038 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.038 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.038 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943268+0000", 2024-09-16T14:49:14.038 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010641695 2024-09-16T14:49:14.039 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.039 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.039 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.039 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943285+0000", 2024-09-16T14:49:14.039 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6929e-05 2024-09-16T14:49:14.039 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.039 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.039 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.039 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.039 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.039 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds1 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:14.039 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:14.039 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 6.0650313560000004, 2024-09-16T14:49:14.039 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010600091000000001, 2024-09-16T14:49:14.040 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.040 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:14.040 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.040 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.040 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.040 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:14.040 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.040 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.040 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.040 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.040 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:14.040 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.040 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.040 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.041 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.041 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:14.041 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:14.041 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.041 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.041 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.041 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932785+0000", 2024-09-16T14:49:14.041 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7829000000000002e-05 2024-09-16T14:49:14.041 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.041 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.041 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.041 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932794+0000", 2024-09-16T14:49:14.041 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5179999999999993e-06 2024-09-16T14:49:14.041 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.042 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.042 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.042 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932801+0000", 2024-09-16T14:49:14.042 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.793e-06 2024-09-16T14:49:14.042 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.042 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.042 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.042 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933189+0000", 2024-09-16T14:49:14.042 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00038759100000000001 2024-09-16T14:49:14.042 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.042 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.042 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.042 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933197+0000", 2024-09-16T14:49:14.042 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5290000000000008e-06 2024-09-16T14:49:14.043 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.043 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.043 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.043 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943351+0000", 2024-09-16T14:49:14.043 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010153673 2024-09-16T14:49:14.043 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.043 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.043 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.043 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943368+0000", 2024-09-16T14:49:14.043 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6538999999999999e-05 2024-09-16T14:49:14.043 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.043 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.043 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.043 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.044 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.044 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/27 ECSubWrite(tid=63, reqid=osd.1.0:63, at_version=61'50, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:14.044 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:14.044 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 6.0649786209999998, 2024-09-16T14:49:14.044 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010214164, 2024-09-16T14:49:14.044 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.044 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:14.044 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.044 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.044 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.044 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:14.044 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.044 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.044 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.045 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.045 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:14.045 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.045 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.045 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.045 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.045 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:14.045 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:14.045 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.045 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.045 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.045 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932838+0000", 2024-09-16T14:49:14.045 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8576e-05 2024-09-16T14:49:14.045 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.046 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.046 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.046 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932847+0000", 2024-09-16T14:49:14.046 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6519999999999995e-06 2024-09-16T14:49:14.046 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.046 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.046 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.046 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932855+0000", 2024-09-16T14:49:14.046 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7600000000000002e-06 2024-09-16T14:49:14.046 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.046 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.046 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.046 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932929+0000", 2024-09-16T14:49:14.046 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4134999999999994e-05 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932937+0000", 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.1519999999999992e-06 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941947+0000", 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0090100430000000006 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943034+0000", 2024-09-16T14:49:14.047 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001087111 2024-09-16T14:49:14.048 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.048 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.048 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.048 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.048 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as2 61/53 ECSubWrite(tid=225, reqid=osd.11.0:225, at_version=61'26, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:14.048 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:14.048 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 6.0648846340000002, 2024-09-16T14:49:14.048 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010782797, 2024-09-16T14:49:14.048 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.048 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:14.048 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.048 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.049 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.049 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:14.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.049 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.049 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.049 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:14.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.049 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.049 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.049 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:14.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:14.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.050 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.050 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.050 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932954+0000", 2024-09-16T14:49:14.050 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0166000000000001e-05 2024-09-16T14:49:14.050 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.050 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.050 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.050 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932980+0000", 2024-09-16T14:49:14.050 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5843e-05 2024-09-16T14:49:14.050 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.050 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.050 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.050 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932987+0000", 2024-09-16T14:49:14.050 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0079999999999996e-06 2024-09-16T14:49:14.051 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.051 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.051 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.051 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933034+0000", 2024-09-16T14:49:14.051 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6934999999999997e-05 2024-09-16T14:49:14.051 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.051 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.051 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.051 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933042+0000", 2024-09-16T14:49:14.051 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8429999999999999e-06 2024-09-16T14:49:14.051 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.051 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.051 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.051 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943652+0000", 2024-09-16T14:49:14.052 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010609288999999999 2024-09-16T14:49:14.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.052 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.052 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.052 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943697+0000", 2024-09-16T14:49:14.052 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5296999999999997e-05 2024-09-16T14:49:14.052 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.052 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.052 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.052 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.052 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:14.052 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:14.052 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 6.0648345490000004, 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010665579999999999, 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.053 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.054 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.054 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:14.054 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:14.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.054 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.054 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.054 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932995+0000", 2024-09-16T14:49:14.054 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1084000000000003e-05 2024-09-16T14:49:14.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.054 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.054 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.054 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933003+0000", 2024-09-16T14:49:14.054 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2199999999999992e-06 2024-09-16T14:49:14.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.055 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.055 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.055 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933010+0000", 2024-09-16T14:49:14.055 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.4359999999999999e-06 2024-09-16T14:49:14.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.055 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.055 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.055 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935460+0000", 2024-09-16T14:49:14.055 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0024503350000000001 2024-09-16T14:49:14.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.055 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.055 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.055 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935469+0000", 2024-09-16T14:49:14.055 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.602e-06 2024-09-16T14:49:14.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.056 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.056 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.056 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943610+0000", 2024-09-16T14:49:14.056 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0081417390000000003 2024-09-16T14:49:14.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.056 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.056 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.056 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943630+0000", 2024-09-16T14:49:14.056 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9584000000000001e-05 2024-09-16T14:49:14.056 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.056 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.056 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.057 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.057 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds1 61/27 ECSubWrite(tid=237, reqid=osd.7.0:237, at_version=61'50, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:14.057 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.935863+0000", 2024-09-16T14:49:14.057 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 6.0619361119999997, 2024-09-16T14:49:14.057 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0078102019999999996, 2024-09-16T14:49:14.057 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.057 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:14.057 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.057 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.057 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.057 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935863+0000", 2024-09-16T14:49:14.057 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.057 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.058 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.058 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.058 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935863+0000", 2024-09-16T14:49:14.058 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.058 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.058 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.058 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.058 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935862+0000", 2024-09-16T14:49:14.058 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:14.058 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.058 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.058 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.058 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935884+0000", 2024-09-16T14:49:14.058 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2419e-05 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935898+0000", 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3709999999999999e-05 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935906+0000", 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2670000000000006e-06 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935949+0000", 2024-09-16T14:49:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2707999999999998e-05 2024-09-16T14:49:14.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.060 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.060 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.060 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935957+0000", 2024-09-16T14:49:14.060 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8599999999999993e-06 2024-09-16T14:49:14.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.060 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.060 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.060 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943638+0000", 2024-09-16T14:49:14.060 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0076813890000000003 2024-09-16T14:49:14.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.060 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.060 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.061 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943673+0000", 2024-09-16T14:49:14.061 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.4913999999999998e-05 2024-09-16T14:49:14.061 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.061 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.061 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.061 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.061 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.061 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 61/27 ECSubWrite(tid=226, reqid=osd.11.0:226, at_version=61'82, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:14.061 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.938771+0000", 2024-09-16T14:49:14.061 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 6.0590274129999999, 2024-09-16T14:49:14.061 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083068940000000004, 2024-09-16T14:49:14.061 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.061 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:14.062 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.062 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.062 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.062 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938771+0000", 2024-09-16T14:49:14.062 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.062 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.062 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.062 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938771+0000", 2024-09-16T14:49:14.062 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.062 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.062 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.062 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938771+0000", 2024-09-16T14:49:14.063 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:14.063 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.063 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.063 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.063 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938785+0000", 2024-09-16T14:49:14.063 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4575e-05 2024-09-16T14:49:14.063 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.063 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.063 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.063 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938806+0000", 2024-09-16T14:49:14.063 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0801e-05 2024-09-16T14:49:14.063 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.063 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.063 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.064 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938811+0000", 2024-09-16T14:49:14.064 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.1100000000000002e-06 2024-09-16T14:49:14.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.064 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.064 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.064 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938850+0000", 2024-09-16T14:49:14.064 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8909999999999998e-05 2024-09-16T14:49:14.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.064 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.064 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.064 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938856+0000", 2024-09-16T14:49:14.064 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.981e-06 2024-09-16T14:49:14.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.064 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.065 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.065 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947064+0000", 2024-09-16T14:49:14.065 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0082073700000000003 2024-09-16T14:49:14.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.065 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.065 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.065 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947078+0000", 2024-09-16T14:49:14.065 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4779e-05 2024-09-16T14:49:14.065 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.065 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.065 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.065 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.065 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.9s4 61/27 ECSubWrite(tid=52, reqid=osd.8.0:52, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.939040+0000", 2024-09-16T14:49:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 6.0587588720000003, 2024-09-16T14:49:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0080081979999999994, 2024-09-16T14:49:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939040+0000", 2024-09-16T14:49:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.066 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939040+0000", 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939039+0000", 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939058+0000", 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9273999999999999e-05 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.067 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939073+0000", 2024-09-16T14:49:14.068 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4375e-05 2024-09-16T14:49:14.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.068 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.068 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.068 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939082+0000", 2024-09-16T14:49:14.068 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3239999999999992e-06 2024-09-16T14:49:14.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.068 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.068 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.068 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939175+0000", 2024-09-16T14:49:14.068 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2423999999999995e-05 2024-09-16T14:49:14.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.068 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.068 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.069 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939185+0000", 2024-09-16T14:49:14.069 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9890000000000004e-06 2024-09-16T14:49:14.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.069 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.069 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.069 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947024+0000", 2024-09-16T14:49:14.069 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0078391329999999999 2024-09-16T14:49:14.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.069 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.069 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.069 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947048+0000", 2024-09-16T14:49:14.069 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4409e-05 2024-09-16T14:49:14.069 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.069 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.070 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.070 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.070 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.070 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.9s4 61/27 ECSubWrite(tid=53, reqid=osd.8.0:53, at_version=61'38, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:14.070 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.941071+0000", 2024-09-16T14:49:14.070 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 6.0567282049999998, 2024-09-16T14:49:14.070 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0069155809999999996, 2024-09-16T14:49:14.070 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.070 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:14.070 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.070 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.070 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.070 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941071+0000", 2024-09-16T14:49:14.071 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.071 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.071 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.071 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.071 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941071+0000", 2024-09-16T14:49:14.071 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.071 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.071 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.071 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.071 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941070+0000", 2024-09-16T14:49:14.071 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:14.071 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.071 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.072 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.072 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941089+0000", 2024-09-16T14:49:14.072 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8513999999999999e-05 2024-09-16T14:49:14.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.072 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.072 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.072 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941099+0000", 2024-09-16T14:49:14.072 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0049e-05 2024-09-16T14:49:14.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.072 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.072 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.072 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941105+0000", 2024-09-16T14:49:14.072 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2040000000000002e-06 2024-09-16T14:49:14.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.073 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.073 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.073 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943240+0000", 2024-09-16T14:49:14.073 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021350829999999999 2024-09-16T14:49:14.073 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.073 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.073 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.073 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943249+0000", 2024-09-16T14:49:14.073 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7110000000000001e-06 2024-09-16T14:49:14.073 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.073 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.073 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.073 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947967+0000", 2024-09-16T14:49:14.073 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0047180440000000002 2024-09-16T14:49:14.074 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.074 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.074 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.074 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947986+0000", 2024-09-16T14:49:14.074 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9480999999999999e-05 2024-09-16T14:49:14.074 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.074 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.074 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.074 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.074 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.074 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:579 3.5s0 3:a29e72cc:::smithi04040151-39:head [read 0~671744,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:14.074 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.946788+0000", 2024-09-16T14:49:14.074 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 2.0510110529999999, 2024-09-16T14:49:14.074 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.037038522, 2024-09-16T14:49:14.075 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.075 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:14.075 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:14.075 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:14.075 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:14.075 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 579 2024-09-16T14:49:14.075 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.075 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.075 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.075 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.075 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946788+0000", 2024-09-16T14:49:14.075 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.075 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.075 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.076 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.076 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946788+0000", 2024-09-16T14:49:14.076 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.076 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.076 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.076 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.076 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946786+0000", 2024-09-16T14:49:14.076 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:14.076 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.076 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.076 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.076 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946795+0000", 2024-09-16T14:49:14.076 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9549999999999998e-06 2024-09-16T14:49:14.077 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.077 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.077 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.077 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946806+0000", 2024-09-16T14:49:14.077 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1174e-05 2024-09-16T14:49:14.077 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.077 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.077 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.077 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946816+0000", 2024-09-16T14:49:14.077 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1949999999999992e-06 2024-09-16T14:49:14.077 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.077 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.077 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.947936+0000", 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011208679999999999 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.947940+0000", 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.365e-06 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.976737+0000", 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.028796684 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.078 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:14.079 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.977002+0000", 2024-09-16T14:49:14.079 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00026509800000000002 2024-09-16T14:49:14.079 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.079 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.079 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.079 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.983826+0000", 2024-09-16T14:49:14.079 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0068246660000000001 2024-09-16T14:49:14.079 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.079 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.079 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.079 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.079 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.079 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:580 3.5s0 3:a29e72cc:::smithi04040151-39:head [read 0~671744,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:14.080 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.946829+0000", 2024-09-16T14:49:14.080 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 2.0509700849999999, 2024-09-16T14:49:14.080 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0381929240000001, 2024-09-16T14:49:14.080 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.080 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:14.080 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:14.080 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:14.080 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:14.080 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 580 2024-09-16T14:49:14.080 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.080 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.080 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.080 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.081 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946829+0000", 2024-09-16T14:49:14.081 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.081 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.081 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.081 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946829+0000", 2024-09-16T14:49:14.081 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.081 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.081 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.081 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946828+0000", 2024-09-16T14:49:14.081 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:14.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.081 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.082 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.082 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946832+0000", 2024-09-16T14:49:14.082 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9009999999999996e-06 2024-09-16T14:49:14.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.082 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.082 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.082 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946839+0000", 2024-09-16T14:49:14.082 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9619999999999996e-06 2024-09-16T14:49:14.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.082 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.082 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.082 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946846+0000", 2024-09-16T14:49:14.082 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7789999999999998e-06 2024-09-16T14:49:14.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.083 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.083 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.083 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.948182+0000", 2024-09-16T14:49:14.083 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0013367609999999999 2024-09-16T14:49:14.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.083 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.083 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:14.083 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.948185+0000", 2024-09-16T14:49:14.083 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.357e-06 2024-09-16T14:49:14.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.083 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.083 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.083 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.979390+0000", 2024-09-16T14:49:14.083 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0312049299999999 2024-09-16T14:49:14.084 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.084 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.084 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:14.084 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.979672+0000", 2024-09-16T14:49:14.084 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00028239299999999998 2024-09-16T14:49:14.084 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.084 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.084 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.084 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.985022+0000", 2024-09-16T14:49:14.084 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053494620000000001 2024-09-16T14:49:14.084 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.084 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.084 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.084 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.085 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.085 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:581 3.5s0 3:a29e72cc:::smithi04040151-39:head [read 0~671744,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:14.085 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.946851+0000", 2024-09-16T14:49:14.085 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 2.0509477860000001, 2024-09-16T14:49:14.085 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0393245289999999, 2024-09-16T14:49:14.085 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.085 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:14.085 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:14.085 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:14.085 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:14.085 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 581 2024-09-16T14:49:14.085 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.085 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.086 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.086 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.086 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946851+0000", 2024-09-16T14:49:14.086 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.086 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.086 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.086 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.086 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946851+0000", 2024-09-16T14:49:14.086 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.086 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.086 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.086 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.086 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946851+0000", 2024-09-16T14:49:14.086 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:14.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.087 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.087 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.087 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946854+0000", 2024-09-16T14:49:14.087 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.937e-06 2024-09-16T14:49:14.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.087 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.087 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.087 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946860+0000", 2024-09-16T14:49:14.087 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.1120000000000001e-06 2024-09-16T14:49:14.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.087 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.087 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.087 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946864+0000", 2024-09-16T14:49:14.088 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.634e-06 2024-09-16T14:49:14.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.088 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.088 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.088 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.948218+0000", 2024-09-16T14:49:14.088 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0013536209999999999 2024-09-16T14:49:14.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.088 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.088 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:14.088 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.948220+0000", 2024-09-16T14:49:14.088 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.866e-06 2024-09-16T14:49:14.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.088 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.089 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.089 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980141+0000", 2024-09-16T14:49:14.089 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0319210940000001 2024-09-16T14:49:14.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.089 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.089 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:14.089 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980309+0000", 2024-09-16T14:49:14.089 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00016819599999999999 2024-09-16T14:49:14.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.089 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.089 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.089 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.986176+0000", 2024-09-16T14:49:14.089 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0058664870000000001 2024-09-16T14:49:14.089 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.090 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.090 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.090 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.090 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.090 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s5 66/65 ECSubWrite(tid=37, reqid=client.4500.0:595, at_version=66'24, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:14.090 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.007465+0000", 2024-09-16T14:49:14.090 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.99033402599999998, 2024-09-16T14:49:14.090 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0076668559999999997, 2024-09-16T14:49:14.090 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:14.090 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:14.090 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:14.090 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.090 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:14.090 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007465+0000", 2024-09-16T14:49:14.091 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.091 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.091 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.091 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:14.091 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007465+0000", 2024-09-16T14:49:14.091 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:14.091 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.091 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.091 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:14.091 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007464+0000", 2024-09-16T14:49:14.091 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:14.091 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.091 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.091 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:14.092 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007548+0000", 2024-09-16T14:49:14.092 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3887999999999994e-05 2024-09-16T14:49:14.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.092 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.092 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:14.092 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007567+0000", 2024-09-16T14:49:14.092 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9505e-05 2024-09-16T14:49:14.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.092 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.092 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:14.092 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007577+0000", 2024-09-16T14:49:14.092 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4129999999999995e-06 2024-09-16T14:49:14.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007722+0000", 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000145795 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007730+0000", 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.6590000000000007e-06 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.015109+0000", 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0073783829999999996 2024-09-16T14:49:14.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:14.094 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:14.094 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:14.094 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.015132+0000", 2024-09-16T14:49:14.094 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3088999999999999e-05 2024-09-16T14:49:14.094 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.094 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.094 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.094 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:14.094 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:14.094 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:14.095 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:14.096 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:14.096 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:14.096 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:14.097 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:49:14.097 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:14.098 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:14.098 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:14.098 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:14.098 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:14.099 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:14.099 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:14.099 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:14.099 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:14.094+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:14.105 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:14.105 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:14.122 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:14.122 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:14.139 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:14.156 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:14.156 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:14.173 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:14.173 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:14.202 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:14.202 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:14.210 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:49:14.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:14.224+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:14.297 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:14.297+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:14.311 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:14.312 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:14.320 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:49:14.399 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:14.397+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:14.422 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:14.422 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:14.430 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:49:14.500 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:14.499+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:14.544 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:14.545 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:14.553 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:49:14.601 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:14.600+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:14.655 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:14.655 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:14.663 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:49:14.702 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:14.701+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:14.766 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:14.766 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:14.774 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:49:14.818 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:14.817+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:14.853 INFO:tasks.rados.rados.0.smithi040.stdout:256: done (1 left) 2024-09-16T14:49:14.853 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 48 current snap is 20 2024-09-16T14:49:14.853 INFO:tasks.rados.rados.0.smithi040.stdout:257: seq_num 101 ranges {1425408=540672,1966080=507904,2473984=622592,3096576=32768} 2024-09-16T14:49:14.864 INFO:tasks.rados.rados.0.smithi040.stdout:257: writing smithi04040151-48 from 1425408 to 1966080 tid 1 2024-09-16T14:49:14.868 INFO:tasks.rados.rados.0.smithi040.stdout:257: writing smithi04040151-48 from 1966080 to 2473984 tid 2 2024-09-16T14:49:14.872 INFO:tasks.rados.rados.0.smithi040.stdout:257: writing smithi04040151-48 from 2473984 to 3096576 tid 3 2024-09-16T14:49:14.872 INFO:tasks.rados.rados.0.smithi040.stdout:257: writing smithi04040151-48 from 3096576 to 3129344 tid 4 2024-09-16T14:49:14.872 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 32 current snap is 20 2024-09-16T14:49:14.872 INFO:tasks.rados.rados.0.smithi040.stdout:258: seq_num 102 ranges {1458176=557056,2015232=638976,2654208=524288} 2024-09-16T14:49:14.884 INFO:tasks.rados.rados.0.smithi040.stdout:258: writing smithi04040151-32 from 1458176 to 2015232 tid 1 2024-09-16T14:49:14.888 INFO:tasks.rados.rados.0.smithi040.stdout:258: writing smithi04040151-32 from 2015232 to 2654208 tid 2 2024-09-16T14:49:14.888 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:14.888 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:14.892 INFO:tasks.rados.rados.0.smithi040.stdout:258: writing smithi04040151-32 from 2654208 to 3178496 tid 3 2024-09-16T14:49:14.892 INFO:tasks.rados.rados.0.smithi040.stdout:259: copy_from oid 38 from oid 8 current snap is 20 2024-09-16T14:49:14.892 INFO:tasks.rados.rados.0.smithi040.stdout:257: finishing write tid 1 to smithi04040151-48 2024-09-16T14:49:14.892 INFO:tasks.rados.rados.0.smithi040.stdout:257: finishing write tid 2 to smithi04040151-48 2024-09-16T14:49:14.892 INFO:tasks.rados.rados.0.smithi040.stdout:257: finishing write tid 3 to smithi04040151-48 2024-09-16T14:49:14.892 INFO:tasks.rados.rados.0.smithi040.stdout:257: finishing write tid 4 to smithi04040151-48 2024-09-16T14:49:14.892 INFO:tasks.rados.rados.0.smithi040.stdout:257: finishing write tid 5 to smithi04040151-48 2024-09-16T14:49:14.892 INFO:tasks.rados.rados.0.smithi040.stdout:257: finishing write tid 6 to smithi04040151-48 2024-09-16T14:49:14.892 INFO:tasks.rados.rados.0.smithi040.stdout:257: oid 48 updating version 0 to 31 2024-09-16T14:49:14.892 INFO:tasks.rados.rados.0.smithi040.stdout:257: oid 48 version 31 is already newer than 30 2024-09-16T14:49:14.892 INFO:tasks.rados.rados.0.smithi040.stdout:257: oid 48 updating version 31 to 34 2024-09-16T14:49:14.892 INFO:tasks.rados.rados.0.smithi040.stdout:257: oid 48 version 34 is already newer than 33 2024-09-16T14:49:14.893 INFO:tasks.rados.rados.0.smithi040.stdout:257: oid 48 version 34 is already newer than 32 2024-09-16T14:49:14.893 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 48 v 34 (ObjNum 101 snap 20 seq_num 101) dirty exists 2024-09-16T14:49:14.893 INFO:tasks.rados.rados.0.smithi040.stdout:257: left oid 48 (ObjNum 101 snap 20 seq_num 101) 2024-09-16T14:49:14.893 INFO:tasks.rados.rados.0.smithi040.stdout:255: done (3 left) 2024-09-16T14:49:14.893 INFO:tasks.rados.rados.0.smithi040.stdout:257: done (2 left) 2024-09-16T14:49:14.893 INFO:tasks.rados.rados.0.smithi040.stdout:260: rollback oid 37 current snap is 20 2024-09-16T14:49:14.893 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 37 to 18 2024-09-16T14:49:14.893 INFO:tasks.rados.rados.0.smithi040.stdout:261: rmattr oid 16 current snap is 20 2024-09-16T14:49:14.893 INFO:tasks.rados.rados.0.smithi040.stdout:261: done (3 left) 2024-09-16T14:49:14.893 INFO:tasks.rados.rados.0.smithi040.stdout:262: copy_from oid 48 from oid 10 current snap is 20 2024-09-16T14:49:14.893 INFO:tasks.rados.rados.0.smithi040.stdout:263: snap_create 2024-09-16T14:49:14.893 INFO:tasks.rados.rados.0.smithi040.stdout:258: finishing write tid 1 to smithi04040151-32 2024-09-16T14:49:14.895 INFO:tasks.rados.rados.0.smithi040.stdout:258: finishing write tid 2 to smithi04040151-32 2024-09-16T14:49:14.895 INFO:tasks.rados.rados.0.smithi040.stdout:262: finishing copy_from to smithi04040151-48 2024-09-16T14:49:14.896 INFO:tasks.rados.rados.0.smithi040.stdout:262: got expected ENOENT (src dne) 2024-09-16T14:49:14.896 INFO:tasks.rados.rados.0.smithi040.stdout:260: finishing rollback tid 0 to smithi04040151-37 2024-09-16T14:49:14.896 INFO:tasks.rados.rados.0.smithi040.stdout:262: finishing copy_from racing read to smithi04040151-48 2024-09-16T14:49:14.897 INFO:tasks.rados.rados.0.smithi040.stdout:259: finishing copy_from racing read to smithi04040151-38 2024-09-16T14:49:14.898 INFO:tasks.rados.rados.0.smithi040.stdout:260: finishing rollback tid 1 to smithi04040151-37 2024-09-16T14:49:14.898 INFO:tasks.rados.rados.0.smithi040.stdout:258: finishing write tid 3 to smithi04040151-32 2024-09-16T14:49:14.898 INFO:tasks.rados.rados.0.smithi040.stdout:259: finishing copy_from to smithi04040151-38 2024-09-16T14:49:14.898 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 38 v 24 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-16T14:49:14.899 INFO:tasks.rados.rados.0.smithi040.stdout:260: finishing rollback tid 2 to smithi04040151-37 2024-09-16T14:49:14.899 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 37 v 89 (ObjNum 177 snap 0 seq_num 1812966272) dirty exists 2024-09-16T14:49:14.899 INFO:tasks.rados.rados.0.smithi040.stdout:258: finishing write tid 4 to smithi04040151-32 2024-09-16T14:49:14.900 INFO:tasks.rados.rados.0.smithi040.stdout:258: finishing write tid 5 to smithi04040151-32 2024-09-16T14:49:14.900 INFO:tasks.rados.rados.0.smithi040.stdout:258: oid 32 updating version 0 to 49 2024-09-16T14:49:14.900 INFO:tasks.rados.rados.0.smithi040.stdout:258: oid 32 version 49 is already newer than 47 2024-09-16T14:49:14.900 INFO:tasks.rados.rados.0.smithi040.stdout:258: oid 32 version 49 is already newer than 48 2024-09-16T14:49:14.900 INFO:tasks.rados.rados.0.smithi040.stdout:258: oid 32 version 49 is already newer than 46 2024-09-16T14:49:14.901 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 32 v 49 (ObjNum 102 snap 20 seq_num 102) dirty exists 2024-09-16T14:49:14.901 INFO:tasks.rados.rados.0.smithi040.stdout:258: left oid 32 (ObjNum 102 snap 20 seq_num 102) 2024-09-16T14:49:14.902 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:49:14.903 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:14.901+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:15.003 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:15.002+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:15.044 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:15.044 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:15.052 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:49:15.136 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:15.135+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:15.195 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:15.195 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:15.203 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:49:15.205 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:15.204+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:15.305 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:15.304+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:15.346 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:15.347 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:15.355 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:49:15.439 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:15.438+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:15.497 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:15.497 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:15.505 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:49:15.546 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:15.545+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:15.608 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:15.607+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:15.649 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:15.649 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:15.658 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:49:15.709 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:15.708+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:15.758 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:15.758 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:15.767 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:49:15.810 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:15.809+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:15.854 INFO:tasks.rados.rados.0.smithi040.stdout:258: done (4 left) 2024-09-16T14:49:15.854 INFO:tasks.rados.rados.0.smithi040.stdout:259: done (3 left) 2024-09-16T14:49:15.855 INFO:tasks.rados.rados.0.smithi040.stdout:260: done (2 left) 2024-09-16T14:49:15.855 INFO:tasks.rados.rados.0.smithi040.stdout:262: done (1 left) 2024-09-16T14:49:15.855 INFO:tasks.rados.rados.0.smithi040.stdout:263: done (0 left) 2024-09-16T14:49:15.855 INFO:tasks.rados.rados.0.smithi040.stdout:264: snap_create 2024-09-16T14:49:15.869 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:15.869 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:15.878 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:49:15.919 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:15.918+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:15.977 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:49:16.020 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:16.020 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:16.021 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:16.020+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:16.030 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:49:16.113 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:16.112+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:16.143 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:16.143 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:16.151 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:49:16.185 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:49:16.214 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:16.213+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:16.256 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:16.256 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:16.264 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:49:16.308 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:16.309 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:16.315 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:16.313+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:16.325 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:16.325 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:16.341 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:16.341 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:16.356 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:16.357 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:16.367 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:16.368 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:16.373 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:16.374 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:16.378 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:49:16.390 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:16.390 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:16.407 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:16.410 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:16.420 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:16.417+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:16.423 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:16.423 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:16.440 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:16.441 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:16.457 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:16.458 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:16.473 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:16.486 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:16.486 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:16.486 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:16.486 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:16.489 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:16.489 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:16.496 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:49:16.517 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:16.516+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:16.618 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:16.617+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:16.651 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:16.651 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:16.652 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:16.652 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:16.652 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:16.661 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:49:16.719 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:16.718+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:16.809 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:16.810 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:16.810 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:16.810 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:16.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.810 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "pg_update_log_missing(3.5s3 epoch 51/27 rep_tid 54 entries 51'14 (0'0) error 3:a7111aa9:::smithi04040151-45:head by client.4500.0:449 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 37'11)", 2024-09-16T14:49:16.810 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.879285+0000", 2024-09-16T14:49:16.810 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 18.928359158999999, 2024-09-16T14:49:16.810 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0096111870000000002, 2024-09-16T14:49:16.810 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.810 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.810 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.811 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.811 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879285+0000", 2024-09-16T14:49:16.811 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.811 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.811 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.811 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879285+0000", 2024-09-16T14:49:16.811 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.811 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.811 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.811 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879285+0000", 2024-09-16T14:49:16.812 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:16.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.812 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.812 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.812 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879291+0000", 2024-09-16T14:49:16.812 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.3450000000000002e-06 2024-09-16T14:49:16.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.812 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.812 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.813 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879311+0000", 2024-09-16T14:49:16.813 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9602999999999998e-05 2024-09-16T14:49:16.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.813 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.813 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.813 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879324+0000", 2024-09-16T14:49:16.813 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3127e-05 2024-09-16T14:49:16.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.813 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.813 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.813 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879406+0000", 2024-09-16T14:49:16.814 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1942999999999998e-05 2024-09-16T14:49:16.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.814 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.814 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.888896+0000", 2024-09-16T14:49:16.814 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0094904359999999997 2024-09-16T14:49:16.814 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.814 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.814 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.814 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 51/27 ECSubWrite(tid=209, reqid=osd.7.0:209, at_version=51'29, trim_to=0'0, roll_forward_to=47'27))", 2024-09-16T14:49:16.814 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.883322+0000", 2024-09-16T14:49:16.814 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 18.924322567000001, 2024-09-16T14:49:16.815 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0092881140000000001, 2024-09-16T14:49:16.815 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.815 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.815 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.815 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.815 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.883322+0000", 2024-09-16T14:49:16.815 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.815 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.815 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.883322+0000", 2024-09-16T14:49:16.815 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.816 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.816 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.816 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.883321+0000", 2024-09-16T14:49:16.816 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:16.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.816 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.816 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.816 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.883344+0000", 2024-09-16T14:49:16.816 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2317999999999999e-05 2024-09-16T14:49:16.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.816 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.816 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.816 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.883355+0000", 2024-09-16T14:49:16.817 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1759e-05 2024-09-16T14:49:16.817 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.817 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.817 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.817 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.883364+0000", 2024-09-16T14:49:16.817 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0609999999999996e-06 2024-09-16T14:49:16.817 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.817 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.817 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.817 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.883420+0000", 2024-09-16T14:49:16.817 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6218000000000003e-05 2024-09-16T14:49:16.817 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.817 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.817 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.818 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.883430+0000", 2024-09-16T14:49:16.818 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0117e-05 2024-09-16T14:49:16.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.818 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.818 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.818 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.892591+0000", 2024-09-16T14:49:16.818 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0091606770000000007 2024-09-16T14:49:16.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.818 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.818 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.818 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.892610+0000", 2024-09-16T14:49:16.818 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9190000000000001e-05 2024-09-16T14:49:16.818 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.818 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:454 3.6s0 3:608d2d89:::smithi04040151-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 18.878210761999998, 2024-09-16T14:49:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013776338000000001, 2024-09-16T14:49:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 454 2024-09-16T14:49:16.819 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.820 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.820 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.820 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.820 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:16.820 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.820 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.820 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.820 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:16.820 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.820 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.821 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.821 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:16.821 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:16.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.821 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.821 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.821 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929436+0000", 2024-09-16T14:49:16.821 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7939999999999998e-06 2024-09-16T14:49:16.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.821 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.821 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.821 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929444+0000", 2024-09-16T14:49:16.821 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4749999999999993e-06 2024-09-16T14:49:16.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.822 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.822 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.822 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929449+0000", 2024-09-16T14:49:16.822 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2750000000000003e-06 2024-09-16T14:49:16.822 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.822 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.822 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.822 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.933117+0000", 2024-09-16T14:49:16.822 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0036674250000000002 2024-09-16T14:49:16.822 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.822 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.822 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:16.823 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.933207+0000", 2024-09-16T14:49:16.823 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0358000000000005e-05 2024-09-16T14:49:16.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.823 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.823 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.823 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.937436+0000", 2024-09-16T14:49:16.823 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0042287089999999998 2024-09-16T14:49:16.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.823 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.823 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:16.823 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.937535+0000", 2024-09-16T14:49:16.823 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9501999999999997e-05 2024-09-16T14:49:16.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.824 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.824 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.824 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.943210+0000", 2024-09-16T14:49:16.824 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056742429999999998 2024-09-16T14:49:16.824 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.824 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.824 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.824 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.824 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 52/27 ECSubWrite(tid=182, reqid=osd.11.0:182, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:16.824 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:16.824 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 17.920399016000001, 2024-09-16T14:49:16.825 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014202568, 2024-09-16T14:49:16.825 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.825 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.825 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.825 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.825 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.825 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:16.825 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.825 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.825 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.825 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.825 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:16.826 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.826 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.826 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.826 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.826 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:16.826 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:16.826 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.826 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.826 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.826 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887302+0000", 2024-09-16T14:49:16.826 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.7673999999999997e-05 2024-09-16T14:49:16.826 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.826 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.827 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.827 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887313+0000", 2024-09-16T14:49:16.827 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0712e-05 2024-09-16T14:49:16.827 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.827 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.827 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.827 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887321+0000", 2024-09-16T14:49:16.827 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1329999999999999e-06 2024-09-16T14:49:16.827 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.827 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.827 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.827 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887397+0000", 2024-09-16T14:49:16.827 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5564999999999999e-05 2024-09-16T14:49:16.828 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.828 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.828 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.828 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887407+0000", 2024-09-16T14:49:16.828 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9690000000000006e-06 2024-09-16T14:49:16.828 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.828 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.828 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.828 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901422+0000", 2024-09-16T14:49:16.828 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01401468 2024-09-16T14:49:16.828 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.828 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.828 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.828 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901448+0000", 2024-09-16T14:49:16.829 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6191999999999999e-05 2024-09-16T14:49:16.829 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.829 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.829 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.829 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.829 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.829 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 52/27 ECSubWrite(tid=214, reqid=osd.7.0:214, at_version=52'35, trim_to=0'0, roll_forward_to=51'31))", 2024-09-16T14:49:16.829 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:16.829 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 17.920355817000001, 2024-09-16T14:49:16.829 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014116002000000001, 2024-09-16T14:49:16.829 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.829 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.829 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.830 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.830 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.830 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:16.830 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.830 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.830 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.830 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.830 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:16.830 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.830 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.830 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.830 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.830 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:16.831 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:16.831 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.831 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.831 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.831 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887318+0000", 2024-09-16T14:49:16.831 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0477999999999999e-05 2024-09-16T14:49:16.831 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.831 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.831 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.831 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887329+0000", 2024-09-16T14:49:16.831 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0845999999999999e-05 2024-09-16T14:49:16.831 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.831 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.832 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.832 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887337+0000", 2024-09-16T14:49:16.832 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7880000000000007e-06 2024-09-16T14:49:16.832 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.832 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.832 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.832 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887631+0000", 2024-09-16T14:49:16.833 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029412399999999999 2024-09-16T14:49:16.833 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.833 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.833 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.833 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887640+0000", 2024-09-16T14:49:16.833 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0180000000000007e-06 2024-09-16T14:49:16.833 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.833 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.833 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.833 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901368+0000", 2024-09-16T14:49:16.833 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013728346000000001 2024-09-16T14:49:16.833 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.833 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.833 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.834 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901404+0000", 2024-09-16T14:49:16.834 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5964000000000003e-05 2024-09-16T14:49:16.834 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.834 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.834 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.834 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.834 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.834 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 52/27 ECSubWrite(tid=51, reqid=osd.1.0:50, at_version=0'0, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T14:49:16.835 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:16.835 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 17.918482849, 2024-09-16T14:49:16.835 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013036324, 2024-09-16T14:49:16.835 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.835 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.835 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.835 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.835 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.835 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:16.835 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.835 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.836 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.836 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.836 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:16.836 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.836 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.836 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.836 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.836 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889160+0000", 2024-09-16T14:49:16.836 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:16.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.837 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.837 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.837 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889174+0000", 2024-09-16T14:49:16.837 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3416e-05 2024-09-16T14:49:16.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.838 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.838 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.838 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889186+0000", 2024-09-16T14:49:16.838 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2369e-05 2024-09-16T14:49:16.838 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.838 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.838 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.838 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889195+0000", 2024-09-16T14:49:16.839 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6540000000000005e-06 2024-09-16T14:49:16.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.839 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.839 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.839 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889243+0000", 2024-09-16T14:49:16.839 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.8455000000000003e-05 2024-09-16T14:49:16.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.839 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.839 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.840 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889253+0000", 2024-09-16T14:49:16.840 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3700000000000001e-06 2024-09-16T14:49:16.840 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.840 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.840 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.840 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902168+0000", 2024-09-16T14:49:16.840 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012915311 2024-09-16T14:49:16.840 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.840 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.841 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.841 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902198+0000", 2024-09-16T14:49:16.841 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9641e-05 2024-09-16T14:49:16.841 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.841 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.841 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.841 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.841 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s2 52/27 ECSubWrite(tid=29, reqid=osd.4.0:29, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:16.841 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:16.841 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 17.917679673999999, 2024-09-16T14:49:16.841 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012371216000000001, 2024-09-16T14:49:16.842 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.842 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.842 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.842 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.842 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:16.842 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.842 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.842 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:16.842 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.843 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.843 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:16.843 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:16.843 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.843 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.843 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889990+0000", 2024-09-16T14:49:16.843 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5548999999999999e-05 2024-09-16T14:49:16.843 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.843 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.843 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890001+0000", 2024-09-16T14:49:16.843 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.098e-05 2024-09-16T14:49:16.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.844 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.844 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890009+0000", 2024-09-16T14:49:16.844 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5930000000000006e-06 2024-09-16T14:49:16.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.844 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.845 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890069+0000", 2024-09-16T14:49:16.845 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.9855999999999997e-05 2024-09-16T14:49:16.845 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.845 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.845 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.845 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890082+0000", 2024-09-16T14:49:16.845 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3312999999999999e-05 2024-09-16T14:49:16.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.846 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.846 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902310+0000", 2024-09-16T14:49:16.846 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012227780000000001 2024-09-16T14:49:16.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.847 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.847 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902336+0000", 2024-09-16T14:49:16.847 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5652000000000001e-05 2024-09-16T14:49:16.847 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.847 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.847 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.848 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:511 3.6s0 3:6714c9f9:::smithi04040151-4:13 [read 0~65536,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-16T14:49:16.848 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:02.896373+0000", 2024-09-16T14:49:16.848 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 13.911271511000001, 2024-09-16T14:49:16.848 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0094892810000000005, 2024-09-16T14:49:16.848 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.848 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:16.848 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:16.848 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:16.848 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:16.848 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 511 2024-09-16T14:49:16.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.848 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.849 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.849 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.896373+0000", 2024-09-16T14:49:16.849 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.849 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.849 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.896373+0000", 2024-09-16T14:49:16.849 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.849 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.849 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.896372+0000", 2024-09-16T14:49:16.849 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:16.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.850 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.850 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.896384+0000", 2024-09-16T14:49:16.850 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.153e-05 2024-09-16T14:49:16.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.850 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.850 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.896391+0000", 2024-09-16T14:49:16.850 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.6890000000000004e-06 2024-09-16T14:49:16.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.850 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.850 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.896398+0000", 2024-09-16T14:49:16.851 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.2210000000000002e-06 2024-09-16T14:49:16.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.851 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.851 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.896505+0000", 2024-09-16T14:49:16.851 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010744800000000001 2024-09-16T14:49:16.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.851 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:16.851 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.896512+0000", 2024-09-16T14:49:16.851 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.0110000000000003e-06 2024-09-16T14:49:16.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.851 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.852 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.902955+0000", 2024-09-16T14:49:16.852 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0064435129999999997 2024-09-16T14:49:16.852 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.852 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.852 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:16.852 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903054+0000", 2024-09-16T14:49:16.852 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8029000000000006e-05 2024-09-16T14:49:16.852 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.852 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.852 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.852 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.905862+0000", 2024-09-16T14:49:16.852 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0028084080000000001 2024-09-16T14:49:16.853 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.853 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.853 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.853 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s2 61/27 ECSubWrite(tid=35, reqid=osd.4.0:34, at_version=0'0, trim_to=0'0, roll_forward_to=61'22))", 2024-09-16T14:49:16.853 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:16.853 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.8782341719999991, 2024-09-16T14:49:16.854 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012712925, 2024-09-16T14:49:16.854 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.854 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.854 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.854 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.854 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.854 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:16.854 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.854 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.854 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.854 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:16.855 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.855 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.855 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929409+0000", 2024-09-16T14:49:16.855 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:16.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.855 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.855 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929422+0000", 2024-09-16T14:49:16.855 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3497e-05 2024-09-16T14:49:16.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.855 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.856 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929434+0000", 2024-09-16T14:49:16.856 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.205e-05 2024-09-16T14:49:16.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.856 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.856 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929443+0000", 2024-09-16T14:49:16.856 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8419999999999994e-06 2024-09-16T14:49:16.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.856 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.856 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929490+0000", 2024-09-16T14:49:16.856 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6825e-05 2024-09-16T14:49:16.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.857 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.857 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929500+0000", 2024-09-16T14:49:16.860 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8810000000000008e-06 2024-09-16T14:49:16.860 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.860 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.860 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.860 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942092+0000", 2024-09-16T14:49:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012591955 2024-09-16T14:49:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942123+0000", 2024-09-16T14:49:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1015e-05 2024-09-16T14:49:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs6 61/27 ECSubWrite(tid=235, reqid=osd.7.0:235, at_version=61'21, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:16.861 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.8768072609999997, 2024-09-16T14:49:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011696734, 2024-09-16T14:49:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:16.862 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930866+0000", 2024-09-16T14:49:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9138000000000001e-05 2024-09-16T14:49:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930874+0000", 2024-09-16T14:49:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5890000000000003e-06 2024-09-16T14:49:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930882+0000", 2024-09-16T14:49:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.25e-06 2024-09-16T14:49:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930933+0000", 2024-09-16T14:49:16.864 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1536999999999998e-05 2024-09-16T14:49:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930941+0000", 2024-09-16T14:49:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7510000000000008e-06 2024-09-16T14:49:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942507+0000", 2024-09-16T14:49:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011565765 2024-09-16T14:49:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.865 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942534+0000", 2024-09-16T14:49:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6900999999999999e-05 2024-09-16T14:49:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 61/27 ECSubWrite(tid=63, reqid=osd.1.0:63, at_version=61'50, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.8752708249999994, 2024-09-16T14:49:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014509462000000001, 2024-09-16T14:49:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.866 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:16.867 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932395+0000", 2024-09-16T14:49:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2039999999999999e-05 2024-09-16T14:49:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932404+0000", 2024-09-16T14:49:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4879999999999996e-06 2024-09-16T14:49:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.868 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932412+0000", 2024-09-16T14:49:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.61e-06 2024-09-16T14:49:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932577+0000", 2024-09-16T14:49:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00016469799999999999 2024-09-16T14:49:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932605+0000", 2024-09-16T14:49:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7963e-05 2024-09-16T14:49:16.869 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946863+0000", 2024-09-16T14:49:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014258600999999999 2024-09-16T14:49:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946883+0000", 2024-09-16T14:49:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9494e-05 2024-09-16T14:49:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.870 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs6 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.8751874199999996, 2024-09-16T14:49:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014400098, 2024-09-16T14:49:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.871 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932456+0000", 2024-09-16T14:49:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932475+0000", 2024-09-16T14:49:16.872 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8695999999999999e-05 2024-09-16T14:49:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932485+0000", 2024-09-16T14:49:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7799999999999995e-06 2024-09-16T14:49:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932492+0000", 2024-09-16T14:49:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5209999999999997e-06 2024-09-16T14:49:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.873 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932705+0000", 2024-09-16T14:49:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00021277700000000001 2024-09-16T14:49:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932721+0000", 2024-09-16T14:49:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6095000000000001e-05 2024-09-16T14:49:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946825+0000", 2024-09-16T14:49:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014103269999999999 2024-09-16T14:49:16.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946857+0000", 2024-09-16T14:49:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2271999999999999e-05 2024-09-16T14:49:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.8749905709999997, 2024-09-16T14:49:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015300085999999999, 2024-09-16T14:49:16.875 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.876 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932653+0000", 2024-09-16T14:49:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932674+0000", 2024-09-16T14:49:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1047e-05 2024-09-16T14:49:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932685+0000", 2024-09-16T14:49:16.877 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0733999999999999e-05 2024-09-16T14:49:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932693+0000", 2024-09-16T14:49:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0819999999999999e-06 2024-09-16T14:49:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945297+0000", 2024-09-16T14:49:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012604198 2024-09-16T14:49:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.878 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945306+0000", 2024-09-16T14:49:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1549999999999996e-06 2024-09-16T14:49:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947927+0000", 2024-09-16T14:49:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.002621295 2024-09-16T14:49:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947954+0000", 2024-09-16T14:49:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6219000000000001e-05 2024-09-16T14:49:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.879 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s3 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.8749028429999992, 2024-09-16T14:49:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013712046, 2024-09-16T14:49:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.880 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.881 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932760+0000", 2024-09-16T14:49:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9247999999999999e-05 2024-09-16T14:49:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932773+0000", 2024-09-16T14:49:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3081000000000001e-05 2024-09-16T14:49:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932783+0000", 2024-09-16T14:49:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7850000000000003e-06 2024-09-16T14:49:16.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932830+0000", 2024-09-16T14:49:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7608999999999998e-05 2024-09-16T14:49:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932842+0000", 2024-09-16T14:49:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1082e-05 2024-09-16T14:49:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946430+0000", 2024-09-16T14:49:16.883 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013588141999999999 2024-09-16T14:49:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946453+0000", 2024-09-16T14:49:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3714999999999999e-05 2024-09-16T14:49:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:16.884 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.8744879010000002, 2024-09-16T14:49:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013268299000000001, 2024-09-16T14:49:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:16.885 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933155+0000", 2024-09-16T14:49:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933182+0000", 2024-09-16T14:49:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7090999999999999e-05 2024-09-16T14:49:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.886 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933202+0000", 2024-09-16T14:49:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9459000000000001e-05 2024-09-16T14:49:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933211+0000", 2024-09-16T14:49:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.397e-06 2024-09-16T14:49:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933284+0000", 2024-09-16T14:49:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2711000000000002e-05 2024-09-16T14:49:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.887 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933294+0000", 2024-09-16T14:49:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0079e-05 2024-09-16T14:49:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946408+0000", 2024-09-16T14:49:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013114097999999999 2024-09-16T14:49:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.888 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946425+0000", 2024-09-16T14:49:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6568000000000001e-05 2024-09-16T14:49:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs6 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:16.889 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.8742710000000002, 2024-09-16T14:49:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014598939999999999, 2024-09-16T14:49:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.890 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933372+0000", 2024-09-16T14:49:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933398+0000", 2024-09-16T14:49:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6129999999999999e-05 2024-09-16T14:49:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933414+0000", 2024-09-16T14:49:16.891 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5549999999999999e-05 2024-09-16T14:49:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933422+0000", 2024-09-16T14:49:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8620000000000003e-06 2024-09-16T14:49:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945189+0000", 2024-09-16T14:49:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011767552000000001 2024-09-16T14:49:16.892 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.893 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.893 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.893 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945200+0000", 2024-09-16T14:49:16.893 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0766e-05 2024-09-16T14:49:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947957+0000", 2024-09-16T14:49:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0027565319999999999 2024-09-16T14:49:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947972+0000", 2024-09-16T14:49:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5359999999999999e-05 2024-09-16T14:49:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.894 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs6 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.8732647339999993, 2024-09-16T14:49:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010566212, 2024-09-16T14:49:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.895 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.896 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934728+0000", 2024-09-16T14:49:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034932000000000002 2024-09-16T14:49:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934739+0000", 2024-09-16T14:49:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1318000000000001e-05 2024-09-16T14:49:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934752+0000", 2024-09-16T14:49:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2901999999999999e-05 2024-09-16T14:49:16.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934823+0000", 2024-09-16T14:49:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1118000000000006e-05 2024-09-16T14:49:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934834+0000", 2024-09-16T14:49:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0591000000000001e-05 2024-09-16T14:49:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.898 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.944917+0000", 2024-09-16T14:49:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010082906000000001 2024-09-16T14:49:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.944946+0000", 2024-09-16T14:49:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8895999999999999e-05 2024-09-16T14:49:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 61/27 ECSubWrite(tid=237, reqid=osd.7.0:237, at_version=61'50, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:16.899 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.8720646149999993, 2024-09-16T14:49:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011297481, 2024-09-16T14:49:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.900 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935579+0000", 2024-09-16T14:49:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935608+0000", 2024-09-16T14:49:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9578999999999999e-05 2024-09-16T14:49:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.901 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935625+0000", 2024-09-16T14:49:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7075000000000002e-05 2024-09-16T14:49:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935642+0000", 2024-09-16T14:49:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6480000000000001e-05 2024-09-16T14:49:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935874+0000", 2024-09-16T14:49:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00023162399999999999 2024-09-16T14:49:16.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935887+0000", 2024-09-16T14:49:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3811e-05 2024-09-16T14:49:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946860+0000", 2024-09-16T14:49:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010972552999999999 2024-09-16T14:49:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946877+0000", 2024-09-16T14:49:16.903 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7132000000000001e-05 2024-09-16T14:49:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 61/27 ECSubWrite(tid=231, reqid=osd.11.0:231, at_version=61'37, trim_to=0'0, roll_forward_to=60'33))", 2024-09-16T14:49:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.8538160710000007, 2024-09-16T14:49:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012494261, 2024-09-16T14:49:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.905 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.905 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:16.905 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.905 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.906 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953840+0000", 2024-09-16T14:49:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1983e-05 2024-09-16T14:49:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953846+0000", 2024-09-16T14:49:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.319e-06 2024-09-16T14:49:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953852+0000", 2024-09-16T14:49:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.823e-06 2024-09-16T14:49:16.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954193+0000", 2024-09-16T14:49:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034066199999999999 2024-09-16T14:49:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954203+0000", 2024-09-16T14:49:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0421999999999999e-05 2024-09-16T14:49:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966300+0000", 2024-09-16T14:49:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012096519 2024-09-16T14:49:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966322+0000", 2024-09-16T14:49:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2829000000000001e-05 2024-09-16T14:49:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 61/27 ECSubWrite(tid=244, reqid=osd.3.0:243, at_version=0'0, trim_to=0'0, roll_forward_to=61'81))", 2024-09-16T14:49:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.8533658010000007, 2024-09-16T14:49:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012038332000000001, 2024-09-16T14:49:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:16.910 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954277+0000", 2024-09-16T14:49:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:16.911 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954290+0000", 2024-09-16T14:49:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2364000000000001e-05 2024-09-16T14:49:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954302+0000", 2024-09-16T14:49:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2665e-05 2024-09-16T14:49:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:16.912 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954322+0000", 2024-09-16T14:49:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9429e-05 2024-09-16T14:49:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954374+0000", 2024-09-16T14:49:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2182000000000003e-05 2024-09-16T14:49:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954384+0000", 2024-09-16T14:49:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0022e-05 2024-09-16T14:49:16.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966295+0000", 2024-09-16T14:49:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011910399 2024-09-16T14:49:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966317+0000", 2024-09-16T14:49:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2169999999999999e-05 2024-09-16T14:49:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:16.914 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:16.915 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:16.916 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:49:16.917 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is unset 2024-09-16T14:49:16.918 INFO:tasks.rados.rados.0.smithi040.stdout:264: done (0 left) 2024-09-16T14:49:16.918 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 49 current snap is 22 2024-09-16T14:49:16.918 INFO:tasks.rados.rados.0.smithi040.stdout:265: seq_num 103 ranges {819200=573440,1392640=786432,2179072=376832} 2024-09-16T14:49:16.918 INFO:tasks.rados.rados.0.smithi040.stdout:265: writing smithi04040151-49 from 819200 to 1392640 tid 1 2024-09-16T14:49:16.918 INFO:tasks.rados.rados.0.smithi040.stdout:265: writing smithi04040151-49 from 1392640 to 2179072 tid 2 2024-09-16T14:49:16.921 INFO:tasks.rados.rados.0.smithi040.stdout:265: writing smithi04040151-49 from 2179072 to 2555904 tid 3 2024-09-16T14:49:16.921 INFO:tasks.rados.rados.0.smithi040.stdout:266: copy_from oid 35 from oid 21 current snap is 22 2024-09-16T14:49:16.921 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 41 current snap is 22 2024-09-16T14:49:16.921 INFO:tasks.rados.rados.0.smithi040.stdout:267: seq_num 104 ranges {1835008=589824,2424832=540672,2965504=622592} 2024-09-16T14:49:16.921 INFO:tasks.rados.rados.0.smithi040.stdout:267: writing smithi04040151-41 from 1835008 to 2424832 tid 1 2024-09-16T14:49:16.921 INFO:tasks.rados.rados.0.smithi040.stdout:267: writing smithi04040151-41 from 2424832 to 2965504 tid 2 2024-09-16T14:49:16.921 INFO:tasks.rados.rados.0.smithi040.stdout:267: writing smithi04040151-41 from 2965504 to 3588096 tid 3 2024-09-16T14:49:16.921 INFO:tasks.rados.rados.0.smithi040.stdout:268: snap_create 2024-09-16T14:49:16.921 INFO:tasks.rados.rados.0.smithi040.stdout:265: finishing write tid 1 to smithi04040151-49 2024-09-16T14:49:16.922 INFO:tasks.rados.rados.0.smithi040.stdout:266: finishing copy_from to smithi04040151-35 2024-09-16T14:49:16.922 INFO:tasks.rados.rados.0.smithi040.stdout:266: got expected ENOENT (src dne) 2024-09-16T14:49:16.922 INFO:tasks.rados.rados.0.smithi040.stdout:266: finishing copy_from racing read to smithi04040151-35 2024-09-16T14:49:16.922 INFO:tasks.rados.rados.0.smithi040.stdout:265: finishing write tid 2 to smithi04040151-49 2024-09-16T14:49:16.922 INFO:tasks.rados.rados.0.smithi040.stdout:265: finishing write tid 3 to smithi04040151-49 2024-09-16T14:49:16.922 INFO:tasks.rados.rados.0.smithi040.stdout:265: finishing write tid 4 to smithi04040151-49 2024-09-16T14:49:16.922 INFO:tasks.rados.rados.0.smithi040.stdout:265: finishing write tid 5 to smithi04040151-49 2024-09-16T14:49:16.922 INFO:tasks.rados.rados.0.smithi040.stdout:265: oid 49 updating version 0 to 46 2024-09-16T14:49:16.922 INFO:tasks.rados.rados.0.smithi040.stdout:265: oid 49 version 46 is already newer than 44 2024-09-16T14:49:16.922 INFO:tasks.rados.rados.0.smithi040.stdout:265: oid 49 version 46 is already newer than 45 2024-09-16T14:49:16.922 INFO:tasks.rados.rados.0.smithi040.stdout:265: oid 49 updating version 46 to 47 2024-09-16T14:49:16.922 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 49 v 47 (ObjNum 103 snap 22 seq_num 103) dirty exists 2024-09-16T14:49:16.922 INFO:tasks.rados.rados.0.smithi040.stdout:265: left oid 49 (ObjNum 103 snap 22 seq_num 103) 2024-09-16T14:49:16.923 INFO:tasks.rados.rados.0.smithi040.stdout:267: finishing write tid 1 to smithi04040151-41 2024-09-16T14:49:16.923 INFO:tasks.rados.rados.0.smithi040.stdout:267: finishing write tid 2 to smithi04040151-41 2024-09-16T14:49:16.923 INFO:tasks.rados.rados.0.smithi040.stdout:267: finishing write tid 3 to smithi04040151-41 2024-09-16T14:49:16.923 INFO:tasks.rados.rados.0.smithi040.stdout:267: finishing write tid 4 to smithi04040151-41 2024-09-16T14:49:16.923 INFO:tasks.rados.rados.0.smithi040.stdout:267: finishing write tid 5 to smithi04040151-41 2024-09-16T14:49:16.923 INFO:tasks.rados.rados.0.smithi040.stdout:267: oid 41 updating version 0 to 33 2024-09-16T14:49:16.923 INFO:tasks.rados.rados.0.smithi040.stdout:267: oid 41 updating version 33 to 34 2024-09-16T14:49:16.923 INFO:tasks.rados.rados.0.smithi040.stdout:267: oid 41 updating version 34 to 35 2024-09-16T14:49:16.923 INFO:tasks.rados.rados.0.smithi040.stdout:267: oid 41 updating version 35 to 36 2024-09-16T14:49:16.923 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 41 v 36 (ObjNum 104 snap 22 seq_num 104) dirty exists 2024-09-16T14:49:16.923 INFO:tasks.rados.rados.0.smithi040.stdout:267: left oid 41 (ObjNum 104 snap 22 seq_num 104) 2024-09-16T14:49:16.923 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:16.915+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:16.973 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 6] 2024-09-16T14:49:16.973 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T14:49:17.043 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:17.042+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:17.066 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:17.066 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:17.066 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:17.066 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:17.075 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:49:17.118 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:17.116+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:17.218 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:17.217+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:17.222 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:17.222 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:17.222 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:17.222 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:17.222 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:17.231 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:49:17.320 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:17.318+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:17.339 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:17.339 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:17.339 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:17.340 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:17.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.340 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.2s5 37/27 ECSubWrite(tid=123, reqid=osd.3.0:123, at_version=37'38, trim_to=0'0, roll_forward_to=37'34))", 2024-09-16T14:49:17.340 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:17.340 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.401108614000002, 2024-09-16T14:49:17.340 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0094962529999999996, 2024-09-16T14:49:17.340 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.340 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.340 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.341 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.341 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:17.341 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.341 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.341 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:17.341 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.341 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.341 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935902+0000", 2024-09-16T14:49:17.341 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:17.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.342 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.342 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935938+0000", 2024-09-16T14:49:17.342 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5731000000000001e-05 2024-09-16T14:49:17.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.342 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.342 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935949+0000", 2024-09-16T14:49:17.342 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1229999999999999e-05 2024-09-16T14:49:17.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.342 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.343 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935957+0000", 2024-09-16T14:49:17.343 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.473e-06 2024-09-16T14:49:17.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.343 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.343 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.936913+0000", 2024-09-16T14:49:17.343 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00095599900000000002 2024-09-16T14:49:17.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.343 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.343 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.936923+0000", 2024-09-16T14:49:17.343 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4059999999999994e-06 2024-09-16T14:49:17.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.344 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.344 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945371+0000", 2024-09-16T14:49:17.344 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084484279999999991 2024-09-16T14:49:17.344 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.344 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.344 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945399+0000", 2024-09-16T14:49:17.344 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7716e-05 2024-09-16T14:49:17.344 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.344 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.344 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.344 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.345 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s1 37/27 ECSubWrite(tid=108, reqid=osd.11.0:108, at_version=37'33, trim_to=0'0, roll_forward_to=35'31))", 2024-09-16T14:49:17.345 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.940384+0000", 2024-09-16T14:49:17.345 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.396627770000002, 2024-09-16T14:49:17.345 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085853549999999994, 2024-09-16T14:49:17.345 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.345 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.345 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.345 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.345 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.345 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940384+0000", 2024-09-16T14:49:17.345 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.345 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.345 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.346 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.346 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940384+0000", 2024-09-16T14:49:17.346 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.346 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.346 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940383+0000", 2024-09-16T14:49:17.346 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.346 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.346 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940406+0000", 2024-09-16T14:49:17.347 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3003999999999999e-05 2024-09-16T14:49:17.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.347 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.347 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940421+0000", 2024-09-16T14:49:17.347 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5285e-05 2024-09-16T14:49:17.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.347 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.347 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940430+0000", 2024-09-16T14:49:17.347 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.439e-06 2024-09-16T14:49:17.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.348 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.348 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940486+0000", 2024-09-16T14:49:17.348 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6266e-05 2024-09-16T14:49:17.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.348 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.348 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940497+0000", 2024-09-16T14:49:17.348 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1231e-05 2024-09-16T14:49:17.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.348 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.348 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.948934+0000", 2024-09-16T14:49:17.349 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084368450000000001 2024-09-16T14:49:17.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.349 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.349 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.948969+0000", 2024-09-16T14:49:17.349 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.4993999999999997e-05 2024-09-16T14:49:17.349 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.349 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.349 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.349 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.0s3 37/27 ECSubWrite(tid=109, reqid=osd.11.0:109, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:17.349 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:17.350 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.396561994000002, 2024-09-16T14:49:17.350 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090738399999999997, 2024-09-16T14:49:17.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.350 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.350 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.350 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.350 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:17.350 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.350 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.350 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:17.351 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.351 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.351 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:17.351 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.351 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.351 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940467+0000", 2024-09-16T14:49:17.351 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8113000000000001e-05 2024-09-16T14:49:17.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940475+0000", 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8499999999999994e-06 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940481+0000", 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0979999999999999e-06 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940569+0000", 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8395999999999999e-05 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.353 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.353 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940577+0000", 2024-09-16T14:49:17.353 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7500000000000003e-06 2024-09-16T14:49:17.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.353 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.353 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949500+0000", 2024-09-16T14:49:17.353 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089230319999999991 2024-09-16T14:49:17.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.354 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.354 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949523+0000", 2024-09-16T14:49:17.354 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3082e-05 2024-09-16T14:49:17.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.354 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.355 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:302 3.4s0 3:255172f3:::smithi04040151-3:head [create,setxattr qPd\\P_iff[nbYodrbXe[dQ_okmVWqe[RceaqQRNm (630) in=40b,setxattr NWYlZi[dPXnppNo]`fatPqfSaTlsQVetOmrQm^ap (138) in=40b,setxattr iVfcaeRT]TQppeYbq_`lRobpUsaVcWXphplasbgY (1507) in=40b,setxattr YeaeOf[jt`g\\k^]VfXTjOONb`jT[WYUgdiglVjYl (8841) in=40b,setxattr [iqZhSWl_sclXkajVYTXlOX]eeeWkPi\\dOYYt_Wb (1928) in=40b,setxattr \\[gaSPWYO`NQXpTccfbk`SZPaU`lbpTk]a] (1262) in=35b,setxattr ]\\]U^niYqQmjcgO[dsO`YpRZeRV]`X]bU[X^_Qis (1086) in=40b,setxattr VafomYY]aa`a^pbWebSRaithSn_dQjQYq]aTarm\\ (1402) in=40b,setxattr acj^VZhfaaikXrXq]_haYqkl[^_atWUTaloiVj`a (105) in=40b,setxattr VNUlaeZOglfodan^Ycb`dqVeZhSTmO_rabViYVch (70) in=40b,setxattr Qka\\PoUoghRc\\Qcnl`kVbj[rs\\hSdVXmlNqha`ta (594) in=40b,setxattr qsSQhb_apjdTlikdpj\\maaVdS[mVfsh_Vqi[ogWo (771) in=40b,setxattr Tm]oi_Rf_V`pSY[VhYSXSQN^bqjtgmm^fUo^f\\ZV (1087) in=40b,setxattr \\dTq (1262) in=4b,setxattr nettWfThbXj]cUpsYrTjZ^qk^\\[ZoZ[qb]Sf[ZbN (1684) in=40b,setxattr Sa\\hrdN^YombnT\\torZ`N]aRqicS_lP^XgP`e`Oc (1963) in=40b,setxattr dWhoTgYPSTh]jfWhPTs^iVdYbtfhi_WjOdes]NXh (1051) in=40b,setxattr Qbp_QrhXQf`O]blcqeqgRchO]RrhbshoV\\cW[_es (10033) in=40b,setxattr WYbQqgdRjmdbeoaP]U]n]PTWOYkP^lWOanNTnaOr (1998) in=40b,setxattr Rfbdg`[^ZYrN]cl][Ypr^tpXmb_mbh[gf\\idocW] (10804) in=40b] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:17.355 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:17.355 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 32.404596216999998, 2024-09-16T14:49:17.355 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.009468509, 2024-09-16T14:49:17.355 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.355 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:17.355 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:17.355 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:17.355 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:17.355 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 302 2024-09-16T14:49:17.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.356 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.356 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.356 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:17.356 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.356 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.356 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:17.356 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.356 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.357 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932413+0000", 2024-09-16T14:49:17.357 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:17.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.357 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.357 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932459+0000", 2024-09-16T14:49:17.357 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6414e-05 2024-09-16T14:49:17.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.357 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.358 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932491+0000", 2024-09-16T14:49:17.358 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1656000000000003e-05 2024-09-16T14:49:17.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.358 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.358 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932499+0000", 2024-09-16T14:49:17.358 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4940000000000003e-06 2024-09-16T14:49:17.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.358 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.358 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932557+0000", 2024-09-16T14:49:17.359 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8044000000000002e-05 2024-09-16T14:49:17.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.359 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:17.359 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932704+0000", 2024-09-16T14:49:17.359 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00014615 2024-09-16T14:49:17.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.359 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.359 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.934021+0000", 2024-09-16T14:49:17.359 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001317052 2024-09-16T14:49:17.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.360 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.360 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941512+0000", 2024-09-16T14:49:17.360 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074917789999999996 2024-09-16T14:49:17.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.360 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:17.360 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941804+0000", 2024-09-16T14:49:17.361 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029206099999999998 2024-09-16T14:49:17.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.361 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.361 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941884+0000", 2024-09-16T14:49:17.361 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9231000000000001e-05 2024-09-16T14:49:17.361 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.361 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.361 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.361 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es2 38/27 ECSubWrite(tid=113, reqid=client.4500.0:298, at_version=38'12, trim_to=0'0, roll_forward_to=37'11))", 2024-09-16T14:49:17.362 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.933568+0000", 2024-09-16T14:49:17.362 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 32.403443694000003, 2024-09-16T14:49:17.362 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089983270000000004, 2024-09-16T14:49:17.362 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.362 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.362 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.362 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.362 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.362 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933568+0000", 2024-09-16T14:49:17.362 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.363 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.363 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.363 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.363 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933568+0000", 2024-09-16T14:49:17.363 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.363 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.363 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.363 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.363 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933567+0000", 2024-09-16T14:49:17.363 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.363 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.363 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.364 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.364 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933833+0000", 2024-09-16T14:49:17.364 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00026602700000000001 2024-09-16T14:49:17.364 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.364 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.364 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.364 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933846+0000", 2024-09-16T14:49:17.364 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.223e-05 2024-09-16T14:49:17.364 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.364 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.364 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.364 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933854+0000", 2024-09-16T14:49:17.364 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5369999999999997e-06 2024-09-16T14:49:17.365 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.365 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.365 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933912+0000", 2024-09-16T14:49:17.365 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8103999999999998e-05 2024-09-16T14:49:17.365 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.365 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.365 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933922+0000", 2024-09-16T14:49:17.365 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.978e-06 2024-09-16T14:49:17.365 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.365 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.942550+0000", 2024-09-16T14:49:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0086275210000000008 2024-09-16T14:49:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.942566+0000", 2024-09-16T14:49:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6268000000000001e-05 2024-09-16T14:49:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.366 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:365 3.4s0 3:2b1464a3:::smithi04040151-28:head [copy-from ver 15] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:17.367 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:17.367 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.397742820000001, 2024-09-16T14:49:17.367 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.019409261000000001, 2024-09-16T14:49:17.367 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.367 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:17.367 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:17.367 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:17.367 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:17.367 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 365 2024-09-16T14:49:17.367 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.367 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.367 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.367 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.367 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:17.368 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.368 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.368 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.368 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:17.368 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.368 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.368 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.368 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939267+0000", 2024-09-16T14:49:17.368 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:17.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.368 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.369 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.369 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939278+0000", 2024-09-16T14:49:17.369 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.093e-05 2024-09-16T14:49:17.369 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.369 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.369 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.369 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939295+0000", 2024-09-16T14:49:17.369 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7161000000000001e-05 2024-09-16T14:49:17.369 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.369 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.369 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.369 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939304+0000", 2024-09-16T14:49:17.369 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.329e-06 2024-09-16T14:49:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939378+0000", 2024-09-16T14:49:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3456999999999998e-05 2024-09-16T14:49:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939387+0000", 2024-09-16T14:49:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1449999999999997e-06 2024-09-16T14:49:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.370 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.941885+0000", 2024-09-16T14:49:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0024978349999999999 2024-09-16T14:49:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942001+0000", 2024-09-16T14:49:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011674499999999999 2024-09-16T14:49:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.953426+0000", 2024-09-16T14:49:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011424808999999999 2024-09-16T14:49:17.371 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.372 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.372 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.372 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956855+0000", 2024-09-16T14:49:17.372 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0034292160000000001 2024-09-16T14:49:17.372 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.372 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.372 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:17.372 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.958596+0000", 2024-09-16T14:49:17.372 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001740969 2024-09-16T14:49:17.372 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.372 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.372 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.373 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.958678+0000", 2024-09-16T14:49:17.373 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1592000000000001e-05 2024-09-16T14:49:17.373 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.373 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.373 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.373 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.373 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.373 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:366 3.4s0 3:2b1464a3:::smithi04040151-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:17.373 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:17.373 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.39769325, 2024-09-16T14:49:17.373 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.017101175, 2024-09-16T14:49:17.373 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.374 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:17.374 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:17.374 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:17.374 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:17.374 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 366 2024-09-16T14:49:17.374 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.374 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.374 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.374 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.374 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:17.374 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.374 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.375 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.375 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.375 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:17.375 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.375 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.375 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.375 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.375 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:17.375 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.375 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.375 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.375 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.375 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939321+0000", 2024-09-16T14:49:17.375 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.49e-06 2024-09-16T14:49:17.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.376 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.376 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.376 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939331+0000", 2024-09-16T14:49:17.376 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.006e-05 2024-09-16T14:49:17.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.376 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.376 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.376 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939336+0000", 2024-09-16T14:49:17.376 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2959999999999998e-06 2024-09-16T14:49:17.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.376 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.376 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.376 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939454+0000", 2024-09-16T14:49:17.377 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000117726 2024-09-16T14:49:17.377 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.377 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.377 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:17.377 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939461+0000", 2024-09-16T14:49:17.377 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1940000000000001e-06 2024-09-16T14:49:17.377 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.377 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.377 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.377 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942256+0000", 2024-09-16T14:49:17.377 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0027949350000000001 2024-09-16T14:49:17.377 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.377 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942325+0000", 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.9119e-05 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956179+0000", 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013853343000000001 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956295+0000", 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000115899 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.378 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956419+0000", 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012467399999999999 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:391 3.4s0 3:20ea1aa7:::smithi04040151-17:head [append 0~0] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.472481204000001, 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011589693999999999, 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:17.379 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:17.380 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:17.380 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:17.380 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 391 2024-09-16T14:49:17.380 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.380 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.380 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.380 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.380 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:17.380 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.380 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.380 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.380 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.380 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864528+0000", 2024-09-16T14:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864540+0000", 2024-09-16T14:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2514000000000001e-05 2024-09-16T14:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.381 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864569+0000", 2024-09-16T14:49:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8716000000000001e-05 2024-09-16T14:49:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864580+0000", 2024-09-16T14:49:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1564e-05 2024-09-16T14:49:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864681+0000", 2024-09-16T14:49:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010063100000000001 2024-09-16T14:49:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.382 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.383 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:17.383 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864690+0000", 2024-09-16T14:49:17.383 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6300000000000004e-06 2024-09-16T14:49:17.383 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.383 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.383 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.383 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.869283+0000", 2024-09-16T14:49:17.383 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.004593186 2024-09-16T14:49:17.383 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.383 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.383 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:17.383 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.869365+0000", 2024-09-16T14:49:17.383 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.174e-05 2024-09-16T14:49:17.383 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.384 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.384 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.384 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870206+0000", 2024-09-16T14:49:17.384 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00084176100000000001 2024-09-16T14:49:17.384 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.384 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.384 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.384 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.873261+0000", 2024-09-16T14:49:17.384 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0030546549999999999 2024-09-16T14:49:17.384 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.384 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.384 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:17.385 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.876069+0000", 2024-09-16T14:49:17.385 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0028082210000000001 2024-09-16T14:49:17.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.385 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.385 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.385 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.876120+0000", 2024-09-16T14:49:17.385 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.0581999999999998e-05 2024-09-16T14:49:17.385 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.385 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.385 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.385 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.385 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:392 3.4s0 3:20ea1aa7:::smithi04040151-17:head [rollback c] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:17.386 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:17.386 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.472390623999999, 2024-09-16T14:49:17.386 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013845265000000001, 2024-09-16T14:49:17.386 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.386 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:17.386 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:17.386 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:17.386 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:17.386 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 392 2024-09-16T14:49:17.386 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.386 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.386 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.386 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.386 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:17.387 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.387 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.387 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.387 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.387 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:17.387 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.387 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.387 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.387 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.387 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864620+0000", 2024-09-16T14:49:17.387 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:17.387 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.387 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.388 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.388 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864626+0000", 2024-09-16T14:49:17.388 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6699999999999999e-06 2024-09-16T14:49:17.388 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.388 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.388 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.388 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864643+0000", 2024-09-16T14:49:17.388 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7404000000000001e-05 2024-09-16T14:49:17.388 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.388 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.388 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.388 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864651+0000", 2024-09-16T14:49:17.388 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0860000000000002e-06 2024-09-16T14:49:17.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.389 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.389 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864759+0000", 2024-09-16T14:49:17.389 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010751000000000001 2024-09-16T14:49:17.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.389 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:17.389 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864765+0000", 2024-09-16T14:49:17.389 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.737e-06 2024-09-16T14:49:17.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.389 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.390 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870832+0000", 2024-09-16T14:49:17.390 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060664930000000001 2024-09-16T14:49:17.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.390 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.390 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:17.390 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870972+0000", 2024-09-16T14:49:17.390 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00014033599999999999 2024-09-16T14:49:17.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.390 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.390 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.390 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.871637+0000", 2024-09-16T14:49:17.390 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00066502800000000002 2024-09-16T14:49:17.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.391 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.391 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.873263+0000", 2024-09-16T14:49:17.391 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0016256409999999999 2024-09-16T14:49:17.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.391 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:17.391 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.878403+0000", 2024-09-16T14:49:17.391 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0051401600000000004 2024-09-16T14:49:17.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.391 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.391 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.878466+0000", 2024-09-16T14:49:17.392 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.2964000000000002e-05 2024-09-16T14:49:17.392 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.392 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.392 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.392 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.392 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:393 3.4s0 3:20ea1aa7:::smithi04040151-17:head [append 0~0] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:17.392 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:17.392 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.472331029999999, 2024-09-16T14:49:17.392 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.016150277000000001, 2024-09-16T14:49:17.392 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.392 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:17.392 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:17.393 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:17.393 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:17.393 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 393 2024-09-16T14:49:17.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.393 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.393 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.393 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:17.393 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.393 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.393 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:17.394 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.394 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.394 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.394 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:17.394 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.394 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.394 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.394 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864688+0000", 2024-09-16T14:49:17.394 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5259999999999993e-06 2024-09-16T14:49:17.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.394 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864705+0000", 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6368999999999999e-05 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864712+0000", 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4749999999999996e-06 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864791+0000", 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9159000000000006e-05 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.396 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:17.396 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864797+0000", 2024-09-16T14:49:17.396 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5400000000000003e-06 2024-09-16T14:49:17.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.396 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.396 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.396 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872109+0000", 2024-09-16T14:49:17.396 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073124569999999996 2024-09-16T14:49:17.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.396 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.396 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:17.396 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872237+0000", 2024-09-16T14:49:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012735099999999999 2024-09-16T14:49:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872865+0000", 2024-09-16T14:49:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00062808000000000004 2024-09-16T14:49:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.874081+0000", 2024-09-16T14:49:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001216186 2024-09-16T14:49:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.397 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:17.398 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.880752+0000", 2024-09-16T14:49:17.398 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0066711339999999996 2024-09-16T14:49:17.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.398 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.398 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.398 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.880831+0000", 2024-09-16T14:49:17.398 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8681000000000001e-05 2024-09-16T14:49:17.398 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.398 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.398 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.398 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.399 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.9s5 51/27 ECSubWrite(tid=35, reqid=osd.8.0:35, at_version=51'21, trim_to=0'0, roll_forward_to=47'19))", 2024-09-16T14:49:17.399 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:17.399 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.447885777, 2024-09-16T14:49:17.399 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010449557999999999, 2024-09-16T14:49:17.399 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.399 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.399 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.399 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.399 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.399 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:17.399 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.400 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.400 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:17.400 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.400 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.400 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889125+0000", 2024-09-16T14:49:17.400 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.400 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.401 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889142+0000", 2024-09-16T14:49:17.401 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7186000000000001e-05 2024-09-16T14:49:17.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.401 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.401 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.401 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889152+0000", 2024-09-16T14:49:17.401 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7920000000000004e-06 2024-09-16T14:49:17.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.401 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.401 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.401 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889159+0000", 2024-09-16T14:49:17.401 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.0319999999999998e-06 2024-09-16T14:49:17.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.402 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.402 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.402 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889203+0000", 2024-09-16T14:49:17.402 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.4264000000000002e-05 2024-09-16T14:49:17.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.402 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.402 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.402 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889212+0000", 2024-09-16T14:49:17.402 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6940000000000001e-06 2024-09-16T14:49:17.403 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.403 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.403 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.403 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.899537+0000", 2024-09-16T14:49:17.403 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010325328 2024-09-16T14:49:17.403 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.403 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.403 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.403 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.899575+0000", 2024-09-16T14:49:17.404 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.7741999999999999e-05 2024-09-16T14:49:17.404 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.404 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.404 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.404 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.404 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.2s5 51/27 ECSubWrite(tid=175, reqid=client.4500.0:460, at_version=51'53, trim_to=0'0, roll_forward_to=51'51))", 2024-09-16T14:49:17.404 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.994224+0000", 2024-09-16T14:49:17.405 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.342786924999999, 2024-09-16T14:49:17.405 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085171090000000001, 2024-09-16T14:49:17.405 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.405 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.405 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.405 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.405 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.405 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994224+0000", 2024-09-16T14:49:17.405 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.406 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.406 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.406 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994224+0000", 2024-09-16T14:49:17.406 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.406 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.406 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.406 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994223+0000", 2024-09-16T14:49:17.406 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:17.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.407 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.407 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.407 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994266+0000", 2024-09-16T14:49:17.407 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2602000000000003e-05 2024-09-16T14:49:17.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.407 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.407 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.407 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994293+0000", 2024-09-16T14:49:17.407 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7773999999999999e-05 2024-09-16T14:49:17.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.407 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.407 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.407 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994304+0000", 2024-09-16T14:49:17.407 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0932e-05 2024-09-16T14:49:17.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.408 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.408 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.408 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994371+0000", 2024-09-16T14:49:17.408 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6202000000000001e-05 2024-09-16T14:49:17.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.408 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.408 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.408 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994383+0000", 2024-09-16T14:49:17.408 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2685e-05 2024-09-16T14:49:17.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.408 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.408 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002711+0000", 2024-09-16T14:49:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0083273529999999991 2024-09-16T14:49:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002742+0000", 2024-09-16T14:49:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0936000000000003e-05 2024-09-16T14:49:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds4 51/27 ECSubWrite(tid=211, reqid=client.4500.0:459, at_version=51'31, trim_to=0'0, roll_forward_to=51'29))", 2024-09-16T14:49:17.409 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.994919+0000", 2024-09-16T14:49:17.410 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.342092315999999, 2024-09-16T14:49:17.410 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0087600569999999999, 2024-09-16T14:49:17.410 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.410 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.410 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.410 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.410 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.410 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994919+0000", 2024-09-16T14:49:17.410 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.410 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.410 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.410 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.410 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994919+0000", 2024-09-16T14:49:17.411 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.411 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.411 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.411 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994918+0000", 2024-09-16T14:49:17.411 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:17.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.411 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.411 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.411 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995629+0000", 2024-09-16T14:49:17.411 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000711245 2024-09-16T14:49:17.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.411 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.411 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.412 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995643+0000", 2024-09-16T14:49:17.412 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.398e-05 2024-09-16T14:49:17.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.412 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.412 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.412 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995651+0000", 2024-09-16T14:49:17.412 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0479999999999999e-06 2024-09-16T14:49:17.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.412 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.412 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.412 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995722+0000", 2024-09-16T14:49:17.412 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.0137999999999998e-05 2024-09-16T14:49:17.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.412 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.413 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.413 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995731+0000", 2024-09-16T14:49:17.413 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5279999999999992e-06 2024-09-16T14:49:17.413 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.413 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.413 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.413 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003653+0000", 2024-09-16T14:49:17.413 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0079221599999999993 2024-09-16T14:49:17.413 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.413 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.413 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.413 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003679+0000", 2024-09-16T14:49:17.413 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5882999999999999e-05 2024-09-16T14:49:17.413 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.414 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.414 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.414 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.414 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.414 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s5 51/27 ECSubWrite(tid=60, reqid=client.4500.0:467, at_version=51'19, trim_to=0'0, roll_forward_to=51'16))", 2024-09-16T14:49:17.414 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:17.414 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.339086571999999, 2024-09-16T14:49:17.414 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0099497439999999999, 2024-09-16T14:49:17.414 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.414 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.414 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.414 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.414 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.415 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:17.415 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.415 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.415 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.415 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:17.415 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.415 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.415 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.415 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997924+0000", 2024-09-16T14:49:17.415 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:17.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.415 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.416 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.416 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997944+0000", 2024-09-16T14:49:17.416 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0404e-05 2024-09-16T14:49:17.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.416 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.416 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.416 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997957+0000", 2024-09-16T14:49:17.416 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2123000000000001e-05 2024-09-16T14:49:17.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.416 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.416 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.416 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997966+0000", 2024-09-16T14:49:17.416 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6549999999999999e-06 2024-09-16T14:49:17.417 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.417 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.417 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.417 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002158+0000", 2024-09-16T14:49:17.417 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0041915930000000004 2024-09-16T14:49:17.417 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.417 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.417 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.417 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002168+0000", 2024-09-16T14:49:17.417 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0071000000000001e-05 2024-09-16T14:49:17.417 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.417 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.417 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.417 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007845+0000", 2024-09-16T14:49:17.418 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056765690000000002 2024-09-16T14:49:17.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.418 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.418 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.418 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007875+0000", 2024-09-16T14:49:17.418 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0077e-05 2024-09-16T14:49:17.418 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.418 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.418 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.418 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.418 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s5 51/27 ECSubWrite(tid=62, reqid=client.4500.0:468, at_version=51'20, trim_to=0'0, roll_forward_to=51'16))", 2024-09-16T14:49:17.418 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.999395+0000", 2024-09-16T14:49:17.418 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.337616621999999, 2024-09-16T14:49:17.419 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084961840000000004, 2024-09-16T14:49:17.419 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.419 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.419 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.419 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.419 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.419 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999395+0000", 2024-09-16T14:49:17.419 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.419 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.419 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.419 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.419 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999395+0000", 2024-09-16T14:49:17.419 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.419 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.420 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.420 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.420 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999394+0000", 2024-09-16T14:49:17.420 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:17.420 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.420 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.420 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.420 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999413+0000", 2024-09-16T14:49:17.420 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9508999999999999e-05 2024-09-16T14:49:17.420 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.420 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.420 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.420 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999436+0000", 2024-09-16T14:49:17.421 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2512999999999999e-05 2024-09-16T14:49:17.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.421 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.421 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.421 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999445+0000", 2024-09-16T14:49:17.421 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4390000000000006e-06 2024-09-16T14:49:17.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.421 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.421 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.421 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003294+0000", 2024-09-16T14:49:17.421 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0038483250000000001 2024-09-16T14:49:17.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.421 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.421 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.422 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003304+0000", 2024-09-16T14:49:17.422 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:49:17.422 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.422 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.422 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.422 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007879+0000", 2024-09-16T14:49:17.422 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0045754569999999998 2024-09-16T14:49:17.422 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.422 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.422 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.422 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007891+0000", 2024-09-16T14:49:17.422 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1953000000000001e-05 2024-09-16T14:49:17.422 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.422 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.423 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.423 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.423 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.423 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:506 3.4s0 3:255172f3:::smithi04040151-3:head [rollback 13] snapc 1b=[1b,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-16T14:49:17.423 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:17.423 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.442648942, 2024-09-16T14:49:17.423 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0092517710000000006, 2024-09-16T14:49:17.423 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.423 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:17.423 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:17.423 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:17.423 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:17.423 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 506 2024-09-16T14:49:17.423 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.424 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.424 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.424 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.424 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:17.424 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.424 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.424 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.424 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:17.424 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.424 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.424 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.425 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:17.425 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.425 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.425 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.425 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894365+0000", 2024-09-16T14:49:17.425 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.571e-06 2024-09-16T14:49:17.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.425 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.425 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.425 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894370+0000", 2024-09-16T14:49:17.425 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8279999999999999e-06 2024-09-16T14:49:17.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.425 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894374+0000", 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5920000000000001e-06 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894483+0000", 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010895699999999999 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894491+0000", 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9389999999999997e-06 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.426 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.427 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.427 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.897713+0000", 2024-09-16T14:49:17.427 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0032221060000000002 2024-09-16T14:49:17.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.427 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.427 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:17.427 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.897829+0000", 2024-09-16T14:49:17.427 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000116082 2024-09-16T14:49:17.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.427 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.427 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.427 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.898559+0000", 2024-09-16T14:49:17.428 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00072999800000000004 2024-09-16T14:49:17.428 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.428 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.428 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.428 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.901829+0000", 2024-09-16T14:49:17.428 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0032702159999999998 2024-09-16T14:49:17.428 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.428 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.428 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:17.428 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903546+0000", 2024-09-16T14:49:17.429 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001716827 2024-09-16T14:49:17.429 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.429 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.429 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.429 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903614+0000", 2024-09-16T14:49:17.429 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8058999999999993e-05 2024-09-16T14:49:17.429 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.429 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.429 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.429 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.429 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.429 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:507 3.4s0 3:255172f3:::smithi04040151-3:head [append 0~0] snapc 1b=[1b,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-16T14:49:17.429 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:17.429 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.442616966999999, 2024-09-16T14:49:17.430 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0095393700000000001, 2024-09-16T14:49:17.430 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.430 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:17.430 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:17.430 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:17.430 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:17.430 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 507 2024-09-16T14:49:17.430 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.430 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.430 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.430 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.431 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:17.431 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.431 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.431 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.431 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:17.431 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.431 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.431 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.431 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:17.431 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.431 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.431 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.432 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.432 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894397+0000", 2024-09-16T14:49:17.432 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.774e-06 2024-09-16T14:49:17.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.432 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.432 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.432 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894403+0000", 2024-09-16T14:49:17.432 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1079999999999998e-06 2024-09-16T14:49:17.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.432 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.432 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.432 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894406+0000", 2024-09-16T14:49:17.432 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.337e-06 2024-09-16T14:49:17.432 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.433 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.433 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.433 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894536+0000", 2024-09-16T14:49:17.433 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012969199999999999 2024-09-16T14:49:17.433 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.433 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.433 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:17.433 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894545+0000", 2024-09-16T14:49:17.433 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7499999999999992e-06 2024-09-16T14:49:17.433 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.433 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.433 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.433 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899040+0000", 2024-09-16T14:49:17.434 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044951289999999996 2024-09-16T14:49:17.434 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.434 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.434 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:17.434 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899164+0000", 2024-09-16T14:49:17.434 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012377699999999999 2024-09-16T14:49:17.434 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.434 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.434 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.434 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899721+0000", 2024-09-16T14:49:17.434 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00055683800000000004 2024-09-16T14:49:17.434 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.434 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.434 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.435 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.901911+0000", 2024-09-16T14:49:17.435 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0021908650000000002 2024-09-16T14:49:17.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.435 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.435 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:17.435 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903859+0000", 2024-09-16T14:49:17.435 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001947383 2024-09-16T14:49:17.435 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.435 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.435 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.435 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903934+0000", 2024-09-16T14:49:17.435 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5056999999999996e-05 2024-09-16T14:49:17.435 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.435 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.436 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.436 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.436 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.436 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:588 3.4s0 3:255172f3:::smithi04040151-3:head [append 0~0] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:17.436 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:11.975961+0000", 2024-09-16T14:49:17.436 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.361050477, 2024-09-16T14:49:17.436 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0258510810000001, 2024-09-16T14:49:17.436 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.436 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:17.436 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:17.436 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:17.436 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:17.436 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 588 2024-09-16T14:49:17.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.437 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.437 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.437 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.437 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975961+0000", 2024-09-16T14:49:17.437 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.437 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.437 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.437 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975961+0000", 2024-09-16T14:49:17.437 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.437 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.437 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.438 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975959+0000", 2024-09-16T14:49:17.438 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:17.438 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.438 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.438 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.438 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975968+0000", 2024-09-16T14:49:17.438 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1020000000000003e-06 2024-09-16T14:49:17.438 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.438 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.438 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.438 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975982+0000", 2024-09-16T14:49:17.438 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.33e-05 2024-09-16T14:49:17.438 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.439 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.439 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.439 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975990+0000", 2024-09-16T14:49:17.439 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4679999999999992e-06 2024-09-16T14:49:17.439 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.439 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.439 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.439 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976080+0000", 2024-09-16T14:49:17.439 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9642999999999995e-05 2024-09-16T14:49:17.439 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.439 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.439 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for peered", 2024-09-16T14:49:17.439 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976083+0000", 2024-09-16T14:49:17.440 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.92e-06 2024-09-16T14:49:17.440 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.440 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.440 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.440 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.983734+0000", 2024-09-16T14:49:17.440 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.007651056 2024-09-16T14:49:17.440 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.440 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.440 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:17.440 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.984659+0000", 2024-09-16T14:49:17.440 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000925493 2024-09-16T14:49:17.440 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.440 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.441 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.441 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.991407+0000", 2024-09-16T14:49:17.441 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0067480969999999998 2024-09-16T14:49:17.441 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.441 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.441 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:17.441 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.991550+0000", 2024-09-16T14:49:17.441 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00014299700000000001 2024-09-16T14:49:17.441 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.441 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.441 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.441 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.992634+0000", 2024-09-16T14:49:17.441 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001083894 2024-09-16T14:49:17.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.442 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.442 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.998056+0000", 2024-09-16T14:49:17.442 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0054220070000000004 2024-09-16T14:49:17.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.442 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:17.442 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.001429+0000", 2024-09-16T14:49:17.442 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0033727480000000001 2024-09-16T14:49:17.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.442 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.442 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.001812+0000", 2024-09-16T14:49:17.443 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00038303600000000001 2024-09-16T14:49:17.443 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.443 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.443 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.443 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.443 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:589 3.4s0 3:255172f3:::smithi04040151-3:head [rollback 24] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:17.443 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:11.976008+0000", 2024-09-16T14:49:17.443 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.3610035319999998, 2024-09-16T14:49:17.443 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.027673222, 2024-09-16T14:49:17.443 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.443 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:17.443 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:17.444 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:17.444 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:17.444 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 589 2024-09-16T14:49:17.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.444 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.444 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.444 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.444 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976008+0000", 2024-09-16T14:49:17.444 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.444 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.444 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.444 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976008+0000", 2024-09-16T14:49:17.444 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.445 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.445 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976007+0000", 2024-09-16T14:49:17.445 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.445 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.445 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976012+0000", 2024-09-16T14:49:17.445 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9760000000000003e-06 2024-09-16T14:49:17.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.445 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.446 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976025+0000", 2024-09-16T14:49:17.446 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2884000000000001e-05 2024-09-16T14:49:17.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.446 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.446 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976031+0000", 2024-09-16T14:49:17.446 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0079999999999999e-06 2024-09-16T14:49:17.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.446 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.446 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976122+0000", 2024-09-16T14:49:17.446 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0946000000000006e-05 2024-09-16T14:49:17.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.447 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for peered", 2024-09-16T14:49:17.447 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976123+0000", 2024-09-16T14:49:17.447 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1489999999999999e-06 2024-09-16T14:49:17.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.447 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.447 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.985902+0000", 2024-09-16T14:49:17.447 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.009778528 2024-09-16T14:49:17.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.447 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:17.447 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.986100+0000", 2024-09-16T14:49:17.447 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00019825499999999999 2024-09-16T14:49:17.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.448 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.448 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.993202+0000", 2024-09-16T14:49:17.448 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071018089999999997 2024-09-16T14:49:17.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.448 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:17.448 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.993327+0000", 2024-09-16T14:49:17.448 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000125031 2024-09-16T14:49:17.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.448 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.448 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.994109+0000", 2024-09-16T14:49:17.448 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00078209799999999995 2024-09-16T14:49:17.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.449 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.449 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.449 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.999045+0000", 2024-09-16T14:49:17.449 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0049362879999999996 2024-09-16T14:49:17.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.449 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.449 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:17.449 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.003322+0000", 2024-09-16T14:49:17.449 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0042765879999999996 2024-09-16T14:49:17.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.449 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.449 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.449 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.003681+0000", 2024-09-16T14:49:17.450 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00035919199999999999 2024-09-16T14:49:17.450 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.450 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.450 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.450 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:590 3.4s0 3:255172f3:::smithi04040151-3:head [append 0~0] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:17.450 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:11.976073+0000", 2024-09-16T14:49:17.450 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.3609386539999999, 2024-09-16T14:49:17.450 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0282184670000001, 2024-09-16T14:49:17.450 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.450 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:17.450 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:17.450 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:17.451 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:17.451 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 590 2024-09-16T14:49:17.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.451 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.451 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.451 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976073+0000", 2024-09-16T14:49:17.451 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.451 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.451 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976073+0000", 2024-09-16T14:49:17.451 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976072+0000", 2024-09-16T14:49:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976076+0000", 2024-09-16T14:49:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.985e-06 2024-09-16T14:49:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976084+0000", 2024-09-16T14:49:17.452 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0080000000000002e-06 2024-09-16T14:49:17.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.453 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.453 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976095+0000", 2024-09-16T14:49:17.453 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0413e-05 2024-09-16T14:49:17.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.453 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.453 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976150+0000", 2024-09-16T14:49:17.453 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5588999999999999e-05 2024-09-16T14:49:17.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.453 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for peered", 2024-09-16T14:49:17.453 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976151+0000", 2024-09-16T14:49:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8700000000000004e-07 2024-09-16T14:49:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.986165+0000", 2024-09-16T14:49:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.010013982 2024-09-16T14:49:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.986260+0000", 2024-09-16T14:49:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4545999999999999e-05 2024-09-16T14:49:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.454 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.455 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.994573+0000", 2024-09-16T14:49:17.455 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0083134779999999991 2024-09-16T14:49:17.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.455 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:17.455 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.994693+0000", 2024-09-16T14:49:17.455 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000119707 2024-09-16T14:49:17.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.455 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.455 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.995284+0000", 2024-09-16T14:49:17.455 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00059083400000000004 2024-09-16T14:49:17.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.456 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.456 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.999048+0000", 2024-09-16T14:49:17.456 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.003763878 2024-09-16T14:49:17.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.456 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:17.456 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.003931+0000", 2024-09-16T14:49:17.456 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0048833339999999996 2024-09-16T14:49:17.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.456 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.456 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.004291+0000", 2024-09-16T14:49:17.456 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00036028999999999998 2024-09-16T14:49:17.457 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.457 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.457 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.457 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.457 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.457 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:17.458 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:49:17.459 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:17.457+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:17.559 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:17.558+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:17.565 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:17.565 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:17.565 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:17.565 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:17.574 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:49:17.661 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:17.659+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:17.720 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:17.720 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:17.720 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:17.721 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:17.721 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:17.729 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:49:17.762 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:17.760+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:17.837 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:17.837 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:17.837 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:17.837 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:17.837 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.838 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=128, reqid=client.4500.0:369, at_version=42'38, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:17.838 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:17.838 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.875256689, 2024-09-16T14:49:17.838 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060794769999999998, 2024-09-16T14:49:17.838 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.838 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.838 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.838 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.838 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.838 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:17.838 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.839 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.839 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.839 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:17.839 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.839 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.839 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.839 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959976+0000", 2024-09-16T14:49:17.839 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:17.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.840 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.840 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.840 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960436+0000", 2024-09-16T14:49:17.840 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00046022400000000002 2024-09-16T14:49:17.840 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.840 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.840 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.840 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960455+0000", 2024-09-16T14:49:17.840 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8162999999999999e-05 2024-09-16T14:49:17.840 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.840 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.841 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.841 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960468+0000", 2024-09-16T14:49:17.841 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3135999999999999e-05 2024-09-16T14:49:17.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.841 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.841 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.841 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960527+0000", 2024-09-16T14:49:17.841 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8859e-05 2024-09-16T14:49:17.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.841 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.841 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.841 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960533+0000", 2024-09-16T14:49:17.842 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:49:17.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.842 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.842 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.966018+0000", 2024-09-16T14:49:17.842 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0054845069999999996 2024-09-16T14:49:17.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.842 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.842 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.966057+0000", 2024-09-16T14:49:17.842 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.8927999999999997e-05 2024-09-16T14:49:17.843 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.843 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.843 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.843 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.843 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=129, reqid=client.4500.0:370, at_version=42'39, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:17.843 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:17.843 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.874000931000001, 2024-09-16T14:49:17.843 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074279639999999996, 2024-09-16T14:49:17.843 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.843 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.844 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.844 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.844 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:17.844 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.844 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.844 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:17.844 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.845 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.845 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.845 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.845 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961232+0000", 2024-09-16T14:49:17.845 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:17.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.846 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.846 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962118+0000", 2024-09-16T14:49:17.846 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00088592399999999996 2024-09-16T14:49:17.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.846 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.846 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962132+0000", 2024-09-16T14:49:17.846 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4127e-05 2024-09-16T14:49:17.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.847 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.847 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962142+0000", 2024-09-16T14:49:17.847 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0181e-05 2024-09-16T14:49:17.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.847 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.847 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962192+0000", 2024-09-16T14:49:17.847 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9826e-05 2024-09-16T14:49:17.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.847 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.847 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962202+0000", 2024-09-16T14:49:17.848 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8400000000000007e-06 2024-09-16T14:49:17.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.848 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.848 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968622+0000", 2024-09-16T14:49:17.848 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00642021 2024-09-16T14:49:17.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.848 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.848 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968661+0000", 2024-09-16T14:49:17.848 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.8670000000000001e-05 2024-09-16T14:49:17.849 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.849 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.849 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.849 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=130, reqid=client.4500.0:371, at_version=42'40, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:17.849 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:17.849 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.872665676, 2024-09-16T14:49:17.849 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0061144900000000002, 2024-09-16T14:49:17.849 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.849 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.849 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.850 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.850 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:17.850 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.850 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.850 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:17.850 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.850 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.851 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962567+0000", 2024-09-16T14:49:17.851 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.851 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.851 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962595+0000", 2024-09-16T14:49:17.851 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7824999999999999e-05 2024-09-16T14:49:17.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.851 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.851 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962611+0000", 2024-09-16T14:49:17.852 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:49:17.852 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.852 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.852 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.852 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962620+0000", 2024-09-16T14:49:17.852 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8259999999999999e-06 2024-09-16T14:49:17.852 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.852 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.852 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.852 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.963285+0000", 2024-09-16T14:49:17.852 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00066560100000000004 2024-09-16T14:49:17.852 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.853 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.853 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.963295+0000", 2024-09-16T14:49:17.853 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0036e-05 2024-09-16T14:49:17.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.853 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.853 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968667+0000", 2024-09-16T14:49:17.853 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0053715960000000002 2024-09-16T14:49:17.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.853 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.854 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968683+0000", 2024-09-16T14:49:17.854 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5522000000000001e-05 2024-09-16T14:49:17.854 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.854 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.854 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.854 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.854 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 43/27 ECSubWrite(tid=133, reqid=osd.11.0:133, at_version=43'42, trim_to=0'0, roll_forward_to=42'40))", 2024-09-16T14:49:17.854 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:17.854 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 27.981457882000001, 2024-09-16T14:49:17.854 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073093740000000004, 2024-09-16T14:49:17.854 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.855 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.855 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.855 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.855 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:17.855 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.855 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.855 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:17.856 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.856 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.856 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:17.856 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:17.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.856 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.856 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854515+0000", 2024-09-16T14:49:17.856 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00073950999999999997 2024-09-16T14:49:17.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.857 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.857 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854527+0000", 2024-09-16T14:49:17.857 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2012e-05 2024-09-16T14:49:17.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.857 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.857 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854535+0000", 2024-09-16T14:49:17.857 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:49:17.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.858 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.858 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.858 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854952+0000", 2024-09-16T14:49:17.858 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00041779399999999998 2024-09-16T14:49:17.858 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.858 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.858 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.858 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854961+0000", 2024-09-16T14:49:17.858 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7749999999999998e-06 2024-09-16T14:49:17.858 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.858 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.859 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.859 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.861052+0000", 2024-09-16T14:49:17.859 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006090608 2024-09-16T14:49:17.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.859 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.859 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.861085+0000", 2024-09-16T14:49:17.859 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3442999999999998e-05 2024-09-16T14:49:17.859 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.859 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.859 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.860 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.860 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.860 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 51/27 ECSubWrite(tid=169, reqid=osd.11.0:169, at_version=51'52, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:17.860 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:17.860 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.954047961000001, 2024-09-16T14:49:17.860 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006781848, 2024-09-16T14:49:17.860 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.860 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.860 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.860 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.860 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.861 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:17.861 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.861 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.861 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:17.861 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.861 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.861 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881185+0000", 2024-09-16T14:49:17.861 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.862 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.862 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.862 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881208+0000", 2024-09-16T14:49:17.862 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2823000000000002e-05 2024-09-16T14:49:17.862 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.862 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.862 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881218+0000", 2024-09-16T14:49:17.862 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3929999999999997e-06 2024-09-16T14:49:17.862 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.863 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.863 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881225+0000", 2024-09-16T14:49:17.863 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1629999999999999e-06 2024-09-16T14:49:17.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.863 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.863 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881279+0000", 2024-09-16T14:49:17.863 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4147000000000002e-05 2024-09-16T14:49:17.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.863 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.864 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881288+0000", 2024-09-16T14:49:17.864 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4630000000000007e-06 2024-09-16T14:49:17.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.864 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.864 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.887941+0000", 2024-09-16T14:49:17.864 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0066530210000000003 2024-09-16T14:49:17.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.864 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.864 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.887968+0000", 2024-09-16T14:49:17.864 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6228999999999999e-05 2024-09-16T14:49:17.865 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.865 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.865 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.865 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.865 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 51/27 ECSubWrite(tid=173, reqid=osd.11.0:173, at_version=51'54, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:17.865 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:17.865 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.953896734000001, 2024-09-16T14:49:17.865 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0080949799999999999, 2024-09-16T14:49:17.865 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.865 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.865 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.866 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.866 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:17.866 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.866 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.866 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.866 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:17.866 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.866 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.867 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.867 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881336+0000", 2024-09-16T14:49:17.867 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.867 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.867 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.867 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.867 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881356+0000", 2024-09-16T14:49:17.867 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9334000000000002e-05 2024-09-16T14:49:17.867 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.867 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.868 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.868 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881366+0000", 2024-09-16T14:49:17.868 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0742e-05 2024-09-16T14:49:17.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.868 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.868 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.868 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881374+0000", 2024-09-16T14:49:17.868 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5020000000000003e-06 2024-09-16T14:49:17.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.868 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.868 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.869 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.882087+0000", 2024-09-16T14:49:17.869 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00071271300000000002 2024-09-16T14:49:17.869 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.869 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.869 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.869 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.882096+0000", 2024-09-16T14:49:17.869 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4550000000000001e-06 2024-09-16T14:49:17.869 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.869 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.869 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.869 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889403+0000", 2024-09-16T14:49:17.870 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073067749999999997 2024-09-16T14:49:17.870 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.870 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.870 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.870 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889432+0000", 2024-09-16T14:49:17.870 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9065999999999999e-05 2024-09-16T14:49:17.870 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.870 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.870 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.870 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.870 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.871 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 52/27 ECSubWrite(tid=50, reqid=osd.1.0:50, at_version=52'40, trim_to=0'0, roll_forward_to=51'36))", 2024-09-16T14:49:17.871 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:17.871 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 18.952693865000001, 2024-09-16T14:49:17.871 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060730380000000002, 2024-09-16T14:49:17.871 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.871 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.871 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.871 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.871 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.871 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:17.871 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.871 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.872 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.872 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.872 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:17.872 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.872 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.872 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.872 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882539+0000", 2024-09-16T14:49:17.872 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:17.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.872 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.872 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.873 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882566+0000", 2024-09-16T14:49:17.873 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7101999999999999e-05 2024-09-16T14:49:17.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.873 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.873 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.873 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882580+0000", 2024-09-16T14:49:17.873 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4605e-05 2024-09-16T14:49:17.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.873 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.873 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.873 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882589+0000", 2024-09-16T14:49:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1020000000000003e-06 2024-09-16T14:49:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.884101+0000", 2024-09-16T14:49:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001511601 2024-09-16T14:49:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.884112+0000", 2024-09-16T14:49:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1055e-05 2024-09-16T14:49:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.875 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.875 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.875 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888581+0000", 2024-09-16T14:49:17.875 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044684240000000004 2024-09-16T14:49:17.875 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.875 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.875 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.875 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888613+0000", 2024-09-16T14:49:17.875 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2354000000000002e-05 2024-09-16T14:49:17.875 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.875 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.876 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.876 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.876 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.876 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=235, reqid=osd.7.0:235, at_version=61'21, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:17.876 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:17.876 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.9037078580000006, 2024-09-16T14:49:17.876 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0080251469999999998, 2024-09-16T14:49:17.876 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.876 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.876 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.876 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.877 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.877 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:17.877 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.877 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.877 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.877 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:17.877 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.877 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.880 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.880 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931525+0000", 2024-09-16T14:49:17.880 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.880 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.880 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931556+0000", 2024-09-16T14:49:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0713000000000002e-05 2024-09-16T14:49:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931577+0000", 2024-09-16T14:49:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0879000000000001e-05 2024-09-16T14:49:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931599+0000", 2024-09-16T14:49:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2107999999999999e-05 2024-09-16T14:49:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932715+0000", 2024-09-16T14:49:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0011159119999999999 2024-09-16T14:49:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932725+0000", 2024-09-16T14:49:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8239999999999995e-06 2024-09-16T14:49:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939520+0000", 2024-09-16T14:49:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00679556 2024-09-16T14:49:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939551+0000", 2024-09-16T14:49:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0582e-05 2024-09-16T14:49:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.884 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.884 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.884 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.884 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 61/27 ECSubWrite(tid=63, reqid=osd.1.0:63, at_version=61'50, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:17.884 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:17.884 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.9035870680000002, 2024-09-16T14:49:17.884 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090916260000000002, 2024-09-16T14:49:17.884 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.884 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.884 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.884 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.885 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.885 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:17.885 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.885 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.885 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.885 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:17.885 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.885 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.886 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.886 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931646+0000", 2024-09-16T14:49:17.886 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:17.886 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.886 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.886 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.886 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931665+0000", 2024-09-16T14:49:17.886 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8459999999999999e-05 2024-09-16T14:49:17.886 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.886 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.887 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.887 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931677+0000", 2024-09-16T14:49:17.887 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2150000000000001e-05 2024-09-16T14:49:17.887 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.887 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.887 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.887 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931684+0000", 2024-09-16T14:49:17.888 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3599999999999998e-06 2024-09-16T14:49:17.888 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.888 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.888 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.888 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931754+0000", 2024-09-16T14:49:17.888 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.9733999999999993e-05 2024-09-16T14:49:17.888 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.888 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.888 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.889 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931766+0000", 2024-09-16T14:49:17.889 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1873e-05 2024-09-16T14:49:17.889 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.889 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.889 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.889 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940712+0000", 2024-09-16T14:49:17.889 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089461669999999997 2024-09-16T14:49:17.889 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.889 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.889 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.889 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940738+0000", 2024-09-16T14:49:17.890 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6092999999999999e-05 2024-09-16T14:49:17.890 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.890 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.890 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.890 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.890 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.890 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:17.890 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:17.890 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.9031870550000001, 2024-09-16T14:49:17.890 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010994122, 2024-09-16T14:49:17.890 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.891 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.891 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.891 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.891 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.891 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:17.891 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.891 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.891 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.891 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.891 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:17.891 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.891 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.891 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.892 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.892 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:17.892 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:17.892 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.892 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.892 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.892 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932066+0000", 2024-09-16T14:49:17.892 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9802000000000001e-05 2024-09-16T14:49:17.892 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.892 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.892 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932076+0000", 2024-09-16T14:49:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8470000000000008e-06 2024-09-16T14:49:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932086+0000", 2024-09-16T14:49:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7010000000000008e-06 2024-09-16T14:49:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933307+0000", 2024-09-16T14:49:17.894 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0012207450000000001 2024-09-16T14:49:17.894 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.894 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.894 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.894 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933319+0000", 2024-09-16T14:49:17.894 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2473000000000001e-05 2024-09-16T14:49:17.894 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.894 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.894 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.894 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943013+0000", 2024-09-16T14:49:17.894 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0096943700000000008 2024-09-16T14:49:17.895 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.895 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.895 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.895 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943041+0000", 2024-09-16T14:49:17.895 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7376999999999999e-05 2024-09-16T14:49:17.895 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.895 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.895 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.895 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.895 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.895 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:17.895 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:17.896 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.902062227, 2024-09-16T14:49:17.896 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0099360779999999992, 2024-09-16T14:49:17.896 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.896 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.896 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.896 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.896 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.896 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:17.896 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.896 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.896 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.897 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.897 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:17.897 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.897 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.897 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.897 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933171+0000", 2024-09-16T14:49:17.897 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.897 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.897 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.897 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933205+0000", 2024-09-16T14:49:17.898 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3630999999999997e-05 2024-09-16T14:49:17.898 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.898 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.898 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.898 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933214+0000", 2024-09-16T14:49:17.898 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0620000000000007e-06 2024-09-16T14:49:17.898 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.898 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.898 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.898 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933222+0000", 2024-09-16T14:49:17.898 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:17.898 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.899 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.899 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933271+0000", 2024-09-16T14:49:17.899 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9251e-05 2024-09-16T14:49:17.899 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.899 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.899 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933280+0000", 2024-09-16T14:49:17.899 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5419999999999994e-06 2024-09-16T14:49:17.899 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.899 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.900 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943089+0000", 2024-09-16T14:49:17.900 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0098082529999999994 2024-09-16T14:49:17.900 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.900 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.900 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.900 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943108+0000", 2024-09-16T14:49:17.900 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8916e-05 2024-09-16T14:49:17.900 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.900 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.900 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.900 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.901 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.901 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s2 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:17.901 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:17.901 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.9019236980000009, 2024-09-16T14:49:17.901 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01234993, 2024-09-16T14:49:17.901 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.901 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.901 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.901 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.901 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.901 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:17.901 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.902 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.902 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:17.902 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.902 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.902 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:17.902 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.903 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.903 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933339+0000", 2024-09-16T14:49:17.903 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9107000000000001e-05 2024-09-16T14:49:17.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.903 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.903 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.903 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933347+0000", 2024-09-16T14:49:17.903 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6219999999999998e-06 2024-09-16T14:49:17.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.903 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.903 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.903 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933354+0000", 2024-09-16T14:49:17.903 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.72e-06 2024-09-16T14:49:17.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.904 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.904 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933438+0000", 2024-09-16T14:49:17.904 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.3666999999999997e-05 2024-09-16T14:49:17.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.904 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.904 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933453+0000", 2024-09-16T14:49:17.904 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4814000000000001e-05 2024-09-16T14:49:17.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.905 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.905 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.905 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945638+0000", 2024-09-16T14:49:17.905 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012185367000000001 2024-09-16T14:49:17.905 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.905 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.905 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.905 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945660+0000", 2024-09-16T14:49:17.905 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2033999999999999e-05 2024-09-16T14:49:17.905 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.905 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.905 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.905 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.906 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:17.906 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:17.906 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.9018785769999997, 2024-09-16T14:49:17.906 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012249365, 2024-09-16T14:49:17.906 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.906 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.906 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.906 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.906 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:17.906 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.907 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.907 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.907 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:17.907 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.907 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.907 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.907 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933354+0000", 2024-09-16T14:49:17.907 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:17.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.907 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933377+0000", 2024-09-16T14:49:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3087000000000001e-05 2024-09-16T14:49:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933393+0000", 2024-09-16T14:49:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5231e-05 2024-09-16T14:49:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933400+0000", 2024-09-16T14:49:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.6190000000000002e-06 2024-09-16T14:49:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933470+0000", 2024-09-16T14:49:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.936e-05 2024-09-16T14:49:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933479+0000", 2024-09-16T14:49:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2609999999999999e-06 2024-09-16T14:49:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945575+0000", 2024-09-16T14:49:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012095896 2024-09-16T14:49:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945605+0000", 2024-09-16T14:49:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9765e-05 2024-09-16T14:49:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.9018362829999997, 2024-09-16T14:49:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013525466, 2024-09-16T14:49:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933451+0000", 2024-09-16T14:49:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4162000000000001e-05 2024-09-16T14:49:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933459+0000", 2024-09-16T14:49:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1110000000000008e-06 2024-09-16T14:49:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5050000000000004e-06 2024-09-16T14:49:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.936478+0000", 2024-09-16T14:49:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0030122790000000001 2024-09-16T14:49:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.936488+0000", 2024-09-16T14:49:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0197000000000001e-05 2024-09-16T14:49:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946886+0000", 2024-09-16T14:49:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010397912 2024-09-16T14:49:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946923+0000", 2024-09-16T14:49:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6699999999999998e-05 2024-09-16T14:49:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.9006788219999997, 2024-09-16T14:49:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0083397039999999999, 2024-09-16T14:49:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934554+0000", 2024-09-16T14:49:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935547+0000", 2024-09-16T14:49:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00099309300000000001 2024-09-16T14:49:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935565+0000", 2024-09-16T14:49:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8335999999999999e-05 2024-09-16T14:49:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935578+0000", 2024-09-16T14:49:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2831e-05 2024-09-16T14:49:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935653+0000", 2024-09-16T14:49:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4411000000000003e-05 2024-09-16T14:49:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935663+0000", 2024-09-16T14:49:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0869999999999999e-05 2024-09-16T14:49:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942652+0000", 2024-09-16T14:49:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006988129 2024-09-16T14:49:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942895+0000", 2024-09-16T14:49:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00024308899999999999 2024-09-16T14:49:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s2 61/53 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=61'38, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.8973399420000003, 2024-09-16T14:49:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090892060000000007, 2024-09-16T14:49:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937893+0000", 2024-09-16T14:49:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937915+0000", 2024-09-16T14:49:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2566e-05 2024-09-16T14:49:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937929+0000", 2024-09-16T14:49:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3314e-05 2024-09-16T14:49:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937938+0000", 2024-09-16T14:49:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2050000000000008e-06 2024-09-16T14:49:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940150+0000", 2024-09-16T14:49:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0022119040000000002 2024-09-16T14:49:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940158+0000", 2024-09-16T14:49:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2930000000000001e-06 2024-09-16T14:49:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946972+0000", 2024-09-16T14:49:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0068139150000000002 2024-09-16T14:49:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946983+0000", 2024-09-16T14:49:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0907000000000001e-05 2024-09-16T14:49:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 61/27 ECSubWrite(tid=226, reqid=osd.11.0:226, at_version=61'82, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.8968173270000008, 2024-09-16T14:49:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085452879999999998, 2024-09-16T14:49:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938432+0000", 2024-09-16T14:49:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5931e-05 2024-09-16T14:49:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938444+0000", 2024-09-16T14:49:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1933000000000001e-05 2024-09-16T14:49:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938449+0000", 2024-09-16T14:49:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2070000000000003e-06 2024-09-16T14:49:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938492+0000", 2024-09-16T14:49:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2827999999999997e-05 2024-09-16T14:49:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938500+0000", 2024-09-16T14:49:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7600000000000008e-06 2024-09-16T14:49:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946932+0000", 2024-09-16T14:49:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084318099999999997 2024-09-16T14:49:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946962+0000", 2024-09-16T14:49:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9603000000000001e-05 2024-09-16T14:49:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=238, reqid=client.4500.0:536, at_version=61'23, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.8959943549999991, 2024-09-16T14:49:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0064046049999999998, 2024-09-16T14:49:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939238+0000", 2024-09-16T14:49:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939537+0000", 2024-09-16T14:49:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029911299999999999 2024-09-16T14:49:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939551+0000", 2024-09-16T14:49:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4003e-05 2024-09-16T14:49:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939561+0000", 2024-09-16T14:49:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4269999999999997e-06 2024-09-16T14:49:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939777+0000", 2024-09-16T14:49:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00021644599999999999 2024-09-16T14:49:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939790+0000", 2024-09-16T14:49:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2381999999999999e-05 2024-09-16T14:49:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.936 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945601+0000", 2024-09-16T14:49:17.936 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0058117139999999999 2024-09-16T14:49:17.936 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.936 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.936 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.936 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945644+0000", 2024-09-16T14:49:17.936 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.265e-05 2024-09-16T14:49:17.936 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.936 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.936 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.936 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.937 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.937 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=239, reqid=client.4500.0:537, at_version=61'24, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:17.937 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.940949+0000", 2024-09-16T14:49:17.937 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.8942850559999993, 2024-09-16T14:49:17.937 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0059680189999999998, 2024-09-16T14:49:17.937 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.937 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.937 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.937 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.937 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.937 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940949+0000", 2024-09-16T14:49:17.937 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.937 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.938 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.938 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.938 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940949+0000", 2024-09-16T14:49:17.938 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.938 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.938 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.938 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.938 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940948+0000", 2024-09-16T14:49:17.938 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:17.938 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.938 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.938 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.939 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.941054+0000", 2024-09-16T14:49:17.939 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000106405 2024-09-16T14:49:17.939 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.939 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.939 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.939 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.941074+0000", 2024-09-16T14:49:17.939 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9813000000000002e-05 2024-09-16T14:49:17.939 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.939 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.939 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.939 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.941083+0000", 2024-09-16T14:49:17.939 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6850000000000007e-06 2024-09-16T14:49:17.940 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.940 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.940 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.940 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.941156+0000", 2024-09-16T14:49:17.940 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3003999999999994e-05 2024-09-16T14:49:17.940 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.940 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.940 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.940 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.941165+0000", 2024-09-16T14:49:17.940 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6299999999999993e-06 2024-09-16T14:49:17.940 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.940 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.940 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.941 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946887+0000", 2024-09-16T14:49:17.941 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.005722024 2024-09-16T14:49:17.941 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.941 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.941 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.941 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946917+0000", 2024-09-16T14:49:17.941 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9252e-05 2024-09-16T14:49:17.941 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.941 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.941 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.941 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.941 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.942 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 66/65 ECSubWrite(tid=69, reqid=client.4500.0:588, at_version=66'56, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:17.942 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:12.992576+0000", 2024-09-16T14:49:17.942 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.8426574139999996, 2024-09-16T14:49:17.942 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0058763239999999996, 2024-09-16T14:49:17.942 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:17.942 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:17.942 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:17.942 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.942 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:17.942 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.992576+0000", 2024-09-16T14:49:17.942 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.942 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.943 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:17.943 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.992576+0000", 2024-09-16T14:49:17.943 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:17.943 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.943 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:17.943 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.992575+0000", 2024-09-16T14:49:17.943 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:17.943 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.943 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:17.944 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.992607+0000", 2024-09-16T14:49:17.944 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1649999999999997e-05 2024-09-16T14:49:17.944 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.944 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.944 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:17.944 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.992625+0000", 2024-09-16T14:49:17.944 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.827e-05 2024-09-16T14:49:17.944 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.944 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.944 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:17.944 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.992634+0000", 2024-09-16T14:49:17.944 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1300000000000007e-06 2024-09-16T14:49:17.944 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.945 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.945 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:17.945 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.992685+0000", 2024-09-16T14:49:17.945 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1180999999999997e-05 2024-09-16T14:49:17.945 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.945 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.945 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:17.945 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.992701+0000", 2024-09-16T14:49:17.945 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5716999999999999e-05 2024-09-16T14:49:17.945 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.945 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.945 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:17.946 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.998427+0000", 2024-09-16T14:49:17.946 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0057257489999999996 2024-09-16T14:49:17.946 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:17.946 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:17.946 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:17.946 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.998453+0000", 2024-09-16T14:49:17.946 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5938000000000001e-05 2024-09-16T14:49:17.946 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.946 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.946 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.946 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:17.946 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:17.947 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:17.948 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:49:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T14:49:17.949 INFO:tasks.rados.rados.0.smithi040.stdout:265: done (3 left) 2024-09-16T14:49:17.949 INFO:tasks.rados.rados.0.smithi040.stdout:266: done (2 left) 2024-09-16T14:49:17.949 INFO:tasks.rados.rados.0.smithi040.stdout:267: done (1 left) 2024-09-16T14:49:17.949 INFO:tasks.rados.rados.0.smithi040.stdout:268: done (0 left) 2024-09-16T14:49:17.950 INFO:tasks.rados.rados.0.smithi040.stdout:269: snap_create 2024-09-16T14:49:17.950 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T14:49:17.951 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:17.947+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:18.049 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:18.047+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:18.098 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:18.098 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:18.098 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:18.098 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:18.108 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:49:18.149 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:18.148+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:18.215 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:18.215 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:18.215 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:18.215 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:18.215 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:18.223 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:49:18.251 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:18.249+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:18.331 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:18.332 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:18.332 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:18.332 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:18.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.332 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:303 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:18.332 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:18.332 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.397146908000003, 2024-09-16T14:49:18.332 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014763235, 2024-09-16T14:49:18.332 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.332 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:18.333 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:18.333 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:18.333 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:18.333 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 303 2024-09-16T14:49:18.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.333 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.333 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.333 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:18.333 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.333 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.334 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:18.334 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.334 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.334 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932467+0000", 2024-09-16T14:49:18.334 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:18.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.334 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.334 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932474+0000", 2024-09-16T14:49:18.335 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3189999999999997e-06 2024-09-16T14:49:18.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.335 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.335 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932490+0000", 2024-09-16T14:49:18.335 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:49:18.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.335 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.335 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932498+0000", 2024-09-16T14:49:18.335 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7789999999999996e-06 2024-09-16T14:49:18.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.336 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.336 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932544+0000", 2024-09-16T14:49:18.336 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6486000000000002e-05 2024-09-16T14:49:18.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.336 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:18.336 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932550+0000", 2024-09-16T14:49:18.336 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.3149999999999999e-06 2024-09-16T14:49:18.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.336 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.336 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.939655+0000", 2024-09-16T14:49:18.337 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071050039999999998 2024-09-16T14:49:18.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.337 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:18.337 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.939762+0000", 2024-09-16T14:49:18.337 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000106885 2024-09-16T14:49:18.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.337 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.337 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.947232+0000", 2024-09-16T14:49:18.338 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074701619999999998 2024-09-16T14:49:18.338 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.338 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.338 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.338 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:304 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:18.338 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:18.338 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.397109166999996, 2024-09-16T14:49:18.338 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015391726, 2024-09-16T14:49:18.338 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.338 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:18.338 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:18.338 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:18.339 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:18.339 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 304 2024-09-16T14:49:18.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.339 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.339 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.339 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:18.339 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.339 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.339 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:18.339 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.340 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.340 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:18.340 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:18.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.340 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.340 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932510+0000", 2024-09-16T14:49:18.340 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.1069999999999998e-06 2024-09-16T14:49:18.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.340 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.340 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932516+0000", 2024-09-16T14:49:18.341 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4699999999999999e-06 2024-09-16T14:49:18.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.341 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.341 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932522+0000", 2024-09-16T14:49:18.341 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4750000000000001e-06 2024-09-16T14:49:18.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.341 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.341 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932610+0000", 2024-09-16T14:49:18.341 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8623999999999997e-05 2024-09-16T14:49:18.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.341 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:18.342 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932616+0000", 2024-09-16T14:49:18.342 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.9649999999999996e-06 2024-09-16T14:49:18.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.342 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.342 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940205+0000", 2024-09-16T14:49:18.342 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0075882570000000002 2024-09-16T14:49:18.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.343 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:18.344 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940285+0000", 2024-09-16T14:49:18.344 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0783000000000002e-05 2024-09-16T14:49:18.344 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.344 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.344 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.947898+0000", 2024-09-16T14:49:18.344 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0076124820000000003 2024-09-16T14:49:18.344 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.344 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.344 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.344 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.345 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:305 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:18.345 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:18.345 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.397028376999998, 2024-09-16T14:49:18.345 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015660833999999998, 2024-09-16T14:49:18.345 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.345 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:18.345 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:18.345 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:18.345 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:18.345 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 305 2024-09-16T14:49:18.345 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.345 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.345 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.345 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.346 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:18.346 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.346 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.346 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:18.346 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.346 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.346 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932586+0000", 2024-09-16T14:49:18.346 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:18.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932591+0000", 2024-09-16T14:49:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2470000000000002e-06 2024-09-16T14:49:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932602+0000", 2024-09-16T14:49:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1396000000000001e-05 2024-09-16T14:49:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932608+0000", 2024-09-16T14:49:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.7169999999999996e-06 2024-09-16T14:49:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932646+0000", 2024-09-16T14:49:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.7978999999999997e-05 2024-09-16T14:49:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932652+0000", 2024-09-16T14:49:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0310000000000004e-06 2024-09-16T14:49:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941247+0000", 2024-09-16T14:49:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085954180000000005 2024-09-16T14:49:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941327+0000", 2024-09-16T14:49:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9354e-05 2024-09-16T14:49:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.948248+0000", 2024-09-16T14:49:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0069212400000000004 2024-09-16T14:49:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:403 3.2s0 3:421e4b92:::smithi04040151-11:head [copy-from ver 2] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.44605048, 2024-09-16T14:49:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015904270000000002, 2024-09-16T14:49:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 403 2024-09-16T14:49:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883563+0000", 2024-09-16T14:49:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883573+0000", 2024-09-16T14:49:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0583e-05 2024-09-16T14:49:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883593+0000", 2024-09-16T14:49:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0052e-05 2024-09-16T14:49:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883603+0000", 2024-09-16T14:49:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4409999999999999e-06 2024-09-16T14:49:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883667+0000", 2024-09-16T14:49:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4671000000000005e-05 2024-09-16T14:49:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883775+0000", 2024-09-16T14:49:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000108068 2024-09-16T14:49:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.896809+0000", 2024-09-16T14:49:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01303355 2024-09-16T14:49:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897717+0000", 2024-09-16T14:49:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00090801599999999999 2024-09-16T14:49:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.899408+0000", 2024-09-16T14:49:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0016911459999999999 2024-09-16T14:49:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.899469+0000", 2024-09-16T14:49:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0968999999999999e-05 2024-09-16T14:49:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:404 3.2s0 3:421e4b92:::smithi04040151-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.445997182999999, 2024-09-16T14:49:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0139831, 2024-09-16T14:49:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 404 2024-09-16T14:49:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883617+0000", 2024-09-16T14:49:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883622+0000", 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7489999999999998e-06 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883631+0000", 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6489999999999997e-06 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883639+0000", 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2120000000000003e-06 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883930+0000", 2024-09-16T14:49:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029138200000000002 2024-09-16T14:49:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883961+0000", 2024-09-16T14:49:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0190999999999999e-05 2024-09-16T14:49:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897411+0000", 2024-09-16T14:49:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013450037999999999 2024-09-16T14:49:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.360 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:18.360 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897506+0000", 2024-09-16T14:49:18.360 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5785000000000005e-05 2024-09-16T14:49:18.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.360 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.360 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897601+0000", 2024-09-16T14:49:18.360 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4868e-05 2024-09-16T14:49:18.360 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.360 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.360 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.361 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:461 3.2s0 3:421e4b92:::smithi04040151-11:head [rollback 10] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:18.361 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:18.361 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.33600714, 2024-09-16T14:49:18.361 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011419016000000001, 2024-09-16T14:49:18.361 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.361 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:18.361 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:18.361 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:18.361 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:18.361 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 461 2024-09-16T14:49:18.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.361 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.361 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.362 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:18.362 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.362 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.362 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.362 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:18.362 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.362 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.362 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.362 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:18.362 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:18.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.362 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993611+0000", 2024-09-16T14:49:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6440000000000002e-06 2024-09-16T14:49:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993619+0000", 2024-09-16T14:49:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.199e-06 2024-09-16T14:49:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993624+0000", 2024-09-16T14:49:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.8790000000000002e-06 2024-09-16T14:49:18.363 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.364 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.364 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.364 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994734+0000", 2024-09-16T14:49:18.364 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0011103339999999999 2024-09-16T14:49:18.364 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.364 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.364 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:18.364 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994830+0000", 2024-09-16T14:49:18.364 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6125000000000005e-05 2024-09-16T14:49:18.364 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.364 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.364 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:18.364 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995335+0000", 2024-09-16T14:49:18.365 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00050461799999999997 2024-09-16T14:49:18.365 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.365 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:18.365 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.001584+0000", 2024-09-16T14:49:18.365 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0062489609999999999 2024-09-16T14:49:18.365 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.365 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:18.365 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004966+0000", 2024-09-16T14:49:18.365 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.003382674 2024-09-16T14:49:18.365 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.005027+0000", 2024-09-16T14:49:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0974000000000003e-05 2024-09-16T14:49:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:470 3.2s0 3:4ee78709:::smithi04040151-40:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.333752676, 2024-09-16T14:49:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011056359999999999, 2024-09-16T14:49:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.366 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:18.367 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:18.367 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:18.367 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:18.367 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 470 2024-09-16T14:49:18.367 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.367 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.367 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.367 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.367 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:18.367 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.367 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.367 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.367 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.367 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:18.368 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.368 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.368 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.368 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995862+0000", 2024-09-16T14:49:18.368 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:18.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.368 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.368 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.368 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995865+0000", 2024-09-16T14:49:18.368 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2440000000000002e-06 2024-09-16T14:49:18.368 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.368 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.368 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.369 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995872+0000", 2024-09-16T14:49:18.369 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5640000000000002e-06 2024-09-16T14:49:18.369 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.369 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.369 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.369 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995877+0000", 2024-09-16T14:49:18.369 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1680000000000003e-06 2024-09-16T14:49:18.369 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.369 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.369 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.369 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.996768+0000", 2024-09-16T14:49:18.369 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00089129699999999999 2024-09-16T14:49:18.369 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.369 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.370 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:18.370 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.996867+0000", 2024-09-16T14:49:18.370 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8259999999999998e-05 2024-09-16T14:49:18.370 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.370 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.370 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.370 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.006919+0000", 2024-09-16T14:49:18.370 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010052284 2024-09-16T14:49:18.370 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.370 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.370 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.370 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.370 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.370 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:471 3.2s0 3:4ee78709:::smithi04040151-40:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:18.371 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:18.371 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.333733949999999, 2024-09-16T14:49:18.371 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012013398999999999, 2024-09-16T14:49:18.371 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.371 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:18.371 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:18.371 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:18.371 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:18.371 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 471 2024-09-16T14:49:18.371 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.371 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.371 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.371 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.371 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:18.372 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.372 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.372 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.372 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.372 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:18.372 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.372 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.372 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.372 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.372 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:18.372 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:18.372 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.372 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.372 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.373 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995884+0000", 2024-09-16T14:49:18.373 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:49:18.373 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.373 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.373 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.373 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995891+0000", 2024-09-16T14:49:18.373 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.72e-06 2024-09-16T14:49:18.373 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.373 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.373 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.373 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995896+0000", 2024-09-16T14:49:18.373 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1889999999999998e-06 2024-09-16T14:49:18.373 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.373 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997239+0000", 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0013430460000000001 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997352+0000", 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011287500000000001 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007895+0000", 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010543113999999999 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.374 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s4 52/27 ECSubWrite(tid=183, reqid=osd.11.0:183, at_version=52'60, trim_to=0'0, roll_forward_to=51'56))", 2024-09-16T14:49:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.442345078999999, 2024-09-16T14:49:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011901715, 2024-09-16T14:49:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:18.375 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.376 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.376 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.376 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:18.376 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.376 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.376 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.376 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:18.376 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:18.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.376 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.376 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.376 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887292+0000", 2024-09-16T14:49:18.377 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1920999999999999e-05 2024-09-16T14:49:18.377 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.377 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.377 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.377 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887303+0000", 2024-09-16T14:49:18.377 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1472e-05 2024-09-16T14:49:18.377 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.377 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.377 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.377 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887311+0000", 2024-09-16T14:49:18.377 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:18.377 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.377 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.377 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.378 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887360+0000", 2024-09-16T14:49:18.378 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9002999999999998e-05 2024-09-16T14:49:18.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.378 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.378 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:18.378 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887369+0000", 2024-09-16T14:49:18.378 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9299999999999992e-06 2024-09-16T14:49:18.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.378 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.378 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:18.378 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.899136+0000", 2024-09-16T14:49:18.378 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011767123000000001 2024-09-16T14:49:18.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.379 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.379 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.379 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.899172+0000", 2024-09-16T14:49:18.379 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6161000000000002e-05 2024-09-16T14:49:18.379 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.379 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.379 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.379 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.379 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.379 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds5 52/27 ECSubWrite(tid=214, reqid=osd.7.0:214, at_version=52'35, trim_to=0'0, roll_forward_to=51'31))", 2024-09-16T14:49:18.379 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:18.379 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.441565914999998, 2024-09-16T14:49:18.379 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010846751, 2024-09-16T14:49:18.379 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.380 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:18.380 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.380 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.380 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.380 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:18.380 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.380 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.380 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.380 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.380 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:18.380 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.380 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.380 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.380 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.381 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888049+0000", 2024-09-16T14:49:18.381 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:18.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.381 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.381 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.381 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888072+0000", 2024-09-16T14:49:18.381 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3008000000000001e-05 2024-09-16T14:49:18.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.381 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.381 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.381 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888082+0000", 2024-09-16T14:49:18.381 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0129000000000001e-05 2024-09-16T14:49:18.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.381 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.382 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.382 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888090+0000", 2024-09-16T14:49:18.382 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7510000000000008e-06 2024-09-16T14:49:18.382 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.382 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.382 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.382 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889906+0000", 2024-09-16T14:49:18.382 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001816603 2024-09-16T14:49:18.382 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.382 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.382 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:18.382 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889931+0000", 2024-09-16T14:49:18.382 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4165e-05 2024-09-16T14:49:18.382 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.898870+0000", 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089394299999999999 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.898896+0000", 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6188999999999999e-05 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.383 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s4 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:18.384 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.931930+0000", 2024-09-16T14:49:18.384 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.397685281999999, 2024-09-16T14:49:18.384 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014515299000000001, 2024-09-16T14:49:18.384 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.384 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:18.384 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.384 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.384 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.384 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931930+0000", 2024-09-16T14:49:18.384 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.384 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.384 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.384 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931930+0000", 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931929+0000", 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931950+0000", 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0661000000000001e-05 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.385 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931962+0000", 2024-09-16T14:49:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1949e-05 2024-09-16T14:49:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931970+0000", 2024-09-16T14:49:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9510000000000006e-06 2024-09-16T14:49:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932022+0000", 2024-09-16T14:49:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1990000000000002e-05 2024-09-16T14:49:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.386 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932032+0000", 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0474e-05 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946420+0000", 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0143874 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946445+0000", 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5627e-05 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.387 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds5 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932528+0000", 2024-09-16T14:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.397087015, 2024-09-16T14:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011203032999999999, 2024-09-16T14:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932528+0000", 2024-09-16T14:49:18.388 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.389 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.389 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932528+0000", 2024-09-16T14:49:18.389 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.389 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.389 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932527+0000", 2024-09-16T14:49:18.389 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:18.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.389 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.389 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932559+0000", 2024-09-16T14:49:18.390 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1553000000000001e-05 2024-09-16T14:49:18.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.390 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.390 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.390 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932575+0000", 2024-09-16T14:49:18.390 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6188000000000002e-05 2024-09-16T14:49:18.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.390 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.390 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.390 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932600+0000", 2024-09-16T14:49:18.390 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4505e-05 2024-09-16T14:49:18.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.390 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.391 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.391 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932684+0000", 2024-09-16T14:49:18.391 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4512999999999995e-05 2024-09-16T14:49:18.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.391 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:18.391 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932694+0000", 2024-09-16T14:49:18.391 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5510000000000005e-06 2024-09-16T14:49:18.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.391 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:18.391 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943713+0000", 2024-09-16T14:49:18.391 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011019340000000001 2024-09-16T14:49:18.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.392 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.392 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.392 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943731+0000", 2024-09-16T14:49:18.392 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8260000000000001e-05 2024-09-16T14:49:18.392 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.392 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.392 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.392 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.392 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.as6 61/53 ECSubWrite(tid=225, reqid=osd.11.0:225, at_version=61'26, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:18.392 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932886+0000", 2024-09-16T14:49:18.392 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.396729789, 2024-09-16T14:49:18.392 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014177248, 2024-09-16T14:49:18.392 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.393 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:18.393 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.393 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.393 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932886+0000", 2024-09-16T14:49:18.393 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.393 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.393 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932886+0000", 2024-09-16T14:49:18.393 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932885+0000", 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932916+0000", 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1257999999999998e-05 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932934+0000", 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7958999999999999e-05 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.394 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.395 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.395 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932942+0000", 2024-09-16T14:49:18.395 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9189999999999999e-06 2024-09-16T14:49:18.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.395 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.395 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933003+0000", 2024-09-16T14:49:18.395 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1301000000000001e-05 2024-09-16T14:49:18.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.395 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:18.395 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933023+0000", 2024-09-16T14:49:18.395 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0258000000000002e-05 2024-09-16T14:49:18.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.396 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.396 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:18.396 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946638+0000", 2024-09-16T14:49:18.396 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013614924 2024-09-16T14:49:18.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.396 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.396 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.396 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947063+0000", 2024-09-16T14:49:18.396 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00042440400000000002 2024-09-16T14:49:18.396 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.396 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.396 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.397 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs1 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:18.397 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933121+0000", 2024-09-16T14:49:18.397 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.396494768, 2024-09-16T14:49:18.397 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013854730000000001, 2024-09-16T14:49:18.397 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.397 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:18.397 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.397 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.397 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933121+0000", 2024-09-16T14:49:18.397 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933121+0000", 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933120+0000", 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933143+0000", 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3048999999999999e-05 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.398 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.399 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.399 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933158+0000", 2024-09-16T14:49:18.399 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.543e-05 2024-09-16T14:49:18.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.399 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.399 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.399 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933166+0000", 2024-09-16T14:49:18.399 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0839999999999992e-06 2024-09-16T14:49:18.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.399 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.399 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.399 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933239+0000", 2024-09-16T14:49:18.399 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2447999999999995e-05 2024-09-16T14:49:18.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.400 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:18.400 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933250+0000", 2024-09-16T14:49:18.400 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1143e-05 2024-09-16T14:49:18.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.400 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:18.400 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946560+0000", 2024-09-16T14:49:18.400 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013309642999999999 2024-09-16T14:49:18.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.400 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.400 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946975+0000", 2024-09-16T14:49:18.400 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00041567600000000002 2024-09-16T14:49:18.401 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.401 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.401 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.401 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.401 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s1 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:18.401 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933175+0000", 2024-09-16T14:49:18.401 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.396440460000001, 2024-09-16T14:49:18.401 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014250087999999999, 2024-09-16T14:49:18.401 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.401 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:18.401 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.401 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.401 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.402 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933175+0000", 2024-09-16T14:49:18.402 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.402 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.402 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.402 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933175+0000", 2024-09-16T14:49:18.402 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.402 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.402 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933174+0000", 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933196+0000", 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1577999999999999e-05 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933210+0000", 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3864e-05 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.403 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.404 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933217+0000", 2024-09-16T14:49:18.404 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1999999999999997e-06 2024-09-16T14:49:18.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.404 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.404 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.404 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933306+0000", 2024-09-16T14:49:18.404 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8985000000000003e-05 2024-09-16T14:49:18.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.404 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.404 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:18.404 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933315+0000", 2024-09-16T14:49:18.404 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5730000000000008e-06 2024-09-16T14:49:18.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.405 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.405 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:18.405 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947397+0000", 2024-09-16T14:49:18.405 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014082643000000001 2024-09-16T14:49:18.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.405 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.405 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.405 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947425+0000", 2024-09-16T14:49:18.405 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7793e-05 2024-09-16T14:49:18.405 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.405 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.405 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.405 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.406 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs1 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:18.406 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933463+0000", 2024-09-16T14:49:18.406 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.396152183, 2024-09-16T14:49:18.406 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014016617, 2024-09-16T14:49:18.406 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.406 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:18.406 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.406 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.406 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.406 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933463+0000", 2024-09-16T14:49:18.406 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.406 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.406 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.407 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933463+0000", 2024-09-16T14:49:18.407 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.407 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.407 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.407 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933462+0000", 2024-09-16T14:49:18.407 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:18.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.407 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.407 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.407 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933487+0000", 2024-09-16T14:49:18.407 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4287e-05 2024-09-16T14:49:18.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.407 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.408 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.408 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933501+0000", 2024-09-16T14:49:18.408 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4202999999999999e-05 2024-09-16T14:49:18.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.408 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.408 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.408 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933509+0000", 2024-09-16T14:49:18.408 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5979999999999999e-06 2024-09-16T14:49:18.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.408 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.408 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.408 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940824+0000", 2024-09-16T14:49:18.408 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073150840000000003 2024-09-16T14:49:18.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940836+0000", 2024-09-16T14:49:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2002e-05 2024-09-16T14:49:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947393+0000", 2024-09-16T14:49:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006557223 2024-09-16T14:49:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947480+0000", 2024-09-16T14:49:18.409 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6948999999999997e-05 2024-09-16T14:49:18.410 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.410 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.410 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.410 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.410 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.410 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s1 61/53 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=61'38, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:18.410 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.937911+0000", 2024-09-16T14:49:18.410 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.391704302000001, 2024-09-16T14:49:18.410 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012160381, 2024-09-16T14:49:18.410 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.410 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:18.410 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.410 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.410 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.411 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937911+0000", 2024-09-16T14:49:18.411 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.411 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.411 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.411 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937911+0000", 2024-09-16T14:49:18.411 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.411 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.411 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.411 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937910+0000", 2024-09-16T14:49:18.411 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:18.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.412 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.412 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.412 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937945+0000", 2024-09-16T14:49:18.412 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.4999999999999997e-05 2024-09-16T14:49:18.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.412 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.412 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.412 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937979+0000", 2024-09-16T14:49:18.412 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3309999999999998e-05 2024-09-16T14:49:18.412 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.412 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.412 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.412 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937988+0000", 2024-09-16T14:49:18.412 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8509999999999994e-06 2024-09-16T14:49:18.413 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.413 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.413 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.413 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943764+0000", 2024-09-16T14:49:18.413 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0057755419999999998 2024-09-16T14:49:18.413 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.413 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.413 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:18.413 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943771+0000", 2024-09-16T14:49:18.413 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8399999999999997e-06 2024-09-16T14:49:18.413 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.413 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.413 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:18.413 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.950049+0000", 2024-09-16T14:49:18.414 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0062787390000000002 2024-09-16T14:49:18.414 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.414 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.414 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.414 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.950071+0000", 2024-09-16T14:49:18.414 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1940999999999999e-05 2024-09-16T14:49:18.414 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.414 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.414 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.414 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.414 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.414 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s4 61/27 ECSubWrite(tid=229, reqid=osd.11.0:229, at_version=61'84, trim_to=0'0, roll_forward_to=61'82))", 2024-09-16T14:49:18.414 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.953930+0000", 2024-09-16T14:49:18.414 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.375684917999999, 2024-09-16T14:49:18.415 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012298072, 2024-09-16T14:49:18.415 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.415 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:18.415 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.415 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.415 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.415 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953930+0000", 2024-09-16T14:49:18.415 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.415 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.415 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.415 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953930+0000", 2024-09-16T14:49:18.415 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.415 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.416 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.416 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.416 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953930+0000", 2024-09-16T14:49:18.416 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:18.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.416 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.416 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.416 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953948+0000", 2024-09-16T14:49:18.416 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8253999999999998e-05 2024-09-16T14:49:18.416 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.416 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.416 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.416 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953964+0000", 2024-09-16T14:49:18.416 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5101e-05 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953971+0000", 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8250000000000005e-06 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954019+0000", 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7250000000000003e-05 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954029+0000", 2024-09-16T14:49:18.417 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0154e-05 2024-09-16T14:49:18.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.418 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.418 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:18.418 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966203+0000", 2024-09-16T14:49:18.418 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01217391 2024-09-16T14:49:18.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.418 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.418 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.418 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966228+0000", 2024-09-16T14:49:18.418 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5799000000000002e-05 2024-09-16T14:49:18.418 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.418 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.418 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.418 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.419 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.419 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es6 61/27 ECSubWrite(tid=230, reqid=osd.11.0:230, at_version=61'35, trim_to=0'0, roll_forward_to=60'33))", 2024-09-16T14:49:18.419 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.953982+0000", 2024-09-16T14:49:18.419 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.375633366000001, 2024-09-16T14:49:18.419 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012264608, 2024-09-16T14:49:18.419 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.419 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:18.419 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.419 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.419 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.419 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953982+0000", 2024-09-16T14:49:18.419 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.419 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.419 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.420 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.420 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953982+0000", 2024-09-16T14:49:18.420 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.420 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.420 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.420 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.420 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953982+0000", 2024-09-16T14:49:18.420 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:18.420 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.420 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.420 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.420 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954000+0000", 2024-09-16T14:49:18.420 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8175000000000001e-05 2024-09-16T14:49:18.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.421 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.421 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.421 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954009+0000", 2024-09-16T14:49:18.421 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2280000000000004e-06 2024-09-16T14:49:18.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.421 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.421 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.421 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954016+0000", 2024-09-16T14:49:18.421 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6839999999999999e-06 2024-09-16T14:49:18.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.421 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.421 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.421 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954084+0000", 2024-09-16T14:49:18.422 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8365e-05 2024-09-16T14:49:18.422 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.422 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.422 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:18.422 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954094+0000", 2024-09-16T14:49:18.422 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0147e-05 2024-09-16T14:49:18.422 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.422 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.422 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:18.422 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966228+0000", 2024-09-16T14:49:18.422 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012133961 2024-09-16T14:49:18.422 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.422 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.422 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966247+0000", 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8386e-05 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es6 61/27 ECSubWrite(tid=231, reqid=osd.11.0:231, at_version=61'37, trim_to=0'0, roll_forward_to=60'33))", 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.954093+0000", 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.375522467, 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012660772000000001, 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:18.423 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.424 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:18.424 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954093+0000", 2024-09-16T14:49:18.424 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.424 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.424 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:18.424 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954093+0000", 2024-09-16T14:49:18.424 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:18.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.424 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.424 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:18.424 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954092+0000", 2024-09-16T14:49:18.424 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:18.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.425 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.425 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:18.425 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954118+0000", 2024-09-16T14:49:18.425 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5442000000000001e-05 2024-09-16T14:49:18.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.425 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.425 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:18.425 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954127+0000", 2024-09-16T14:49:18.425 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9560000000000003e-06 2024-09-16T14:49:18.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.425 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.425 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:18.425 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954133+0000", 2024-09-16T14:49:18.425 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5810000000000002e-06 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954527+0000", 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000393161 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954536+0000", 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5699999999999999e-06 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966735+0000", 2024-09-16T14:49:18.426 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012198413999999999 2024-09-16T14:49:18.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:18.427 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:18.427 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:18.427 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966754+0000", 2024-09-16T14:49:18.427 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9082000000000001e-05 2024-09-16T14:49:18.427 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.427 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.427 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.427 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:18.427 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:18.427 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:18.429 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:49:18.429 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:18.427+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:18.500 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:49:18.529 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:18.528+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:18.545 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:18.545 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:18.545 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:18.545 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:18.554 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:49:18.623 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:18.623 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:18.631 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:18.629+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:18.639 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:18.639 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:18.654 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:18.655 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:18.659 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:18.659 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:18.659 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:18.659 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:18.659 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:18.667 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:49:18.670 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:18.671 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:18.686 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:18.687 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:18.703 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:18.703 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:18.732 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:18.730+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:18.770 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:18.771 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:18.779 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:49:18.833 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:18.831+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:18.863 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T14:49:18.864 INFO:tasks.rados.rados.0.smithi040.stdout:269: done (0 left) 2024-09-16T14:49:18.864 INFO:tasks.rados.rados.0.smithi040.stdout:270: snap_remove snap 19 2024-09-16T14:49:18.878 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T14:49:18.885 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:18.885 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:18.894 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:49:18.920 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:49:18.933 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:18.932+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:18.996 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:18.996 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:19.005 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:49:19.049 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:19.047+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:19.109 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:19.109 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:19.116 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:49:19.158 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:19.156+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:19.236 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:19.234+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:19.260 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:19.260 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:19.260 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:19.260 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:19.269 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:49:19.322 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:19.322 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:19.337 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:19.336+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:19.415 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:19.415 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:19.415 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:19.415 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:19.415 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:19.423 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:49:19.438 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:19.437+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:19.527 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:19.527 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:19.527 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:19.527 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:19.527 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.528 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:49:19.528 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:19.528 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 57.385291021, 2024-09-16T14:49:19.528 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.223181782, 2024-09-16T14:49:19.528 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.528 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:19.528 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.528 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:19.528 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:19.528 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 1 2024-09-16T14:49:19.528 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.528 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.528 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.529 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.529 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:19.529 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.529 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.529 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.529 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.529 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:19.529 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.529 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.529 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.529 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.529 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139495+0000", 2024-09-16T14:49:19.529 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:19.529 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.530 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.530 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.530 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139504+0000", 2024-09-16T14:49:19.530 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6119999999999999e-06 2024-09-16T14:49:19.530 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.530 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.530 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.530 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139514+0000", 2024-09-16T14:49:19.530 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:49:19.530 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.530 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.530 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.530 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139533+0000", 2024-09-16T14:49:19.530 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9573e-05 2024-09-16T14:49:19.531 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.531 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.531 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.531 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139585+0000", 2024-09-16T14:49:19.531 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2170999999999999e-05 2024-09-16T14:49:19.531 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.531 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.531 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:19.531 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139594+0000", 2024-09-16T14:49:19.531 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.884e-06 2024-09-16T14:49:19.531 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.531 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.531 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.532 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359233+0000", 2024-09-16T14:49:19.532 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.21963890899999999 2024-09-16T14:49:19.532 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.532 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.532 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.532 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359437+0000", 2024-09-16T14:49:19.532 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00020374999999999999 2024-09-16T14:49:19.532 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.532 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.532 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:19.532 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359611+0000", 2024-09-16T14:49:19.532 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017445899999999999 2024-09-16T14:49:19.532 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.533 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.533 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:19.533 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.360506+0000", 2024-09-16T14:49:19.533 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00089459299999999995 2024-09-16T14:49:19.533 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.533 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.533 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:19.533 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362594+0000", 2024-09-16T14:49:19.533 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002087776 2024-09-16T14:49:19.533 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.533 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.533 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:19.533 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362636+0000", 2024-09-16T14:49:19.533 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1933e-05 2024-09-16T14:49:19.534 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.534 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.534 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.534 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362679+0000", 2024-09-16T14:49:19.534 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2877999999999998e-05 2024-09-16T14:49:19.534 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.534 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.534 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.534 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.534 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.534 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:19.534 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:49:19.534 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 56.132620703000001, 2024-09-16T14:49:19.534 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083881519999999994, 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 164 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.535 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.536 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:49:19.536 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.536 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.536 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.536 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.536 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392167+0000", 2024-09-16T14:49:19.536 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:19.536 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.536 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.536 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.536 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392212+0000", 2024-09-16T14:49:19.536 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5108999999999998e-05 2024-09-16T14:49:19.536 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.537 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.537 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.537 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392216+0000", 2024-09-16T14:49:19.537 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2549999999999999e-06 2024-09-16T14:49:19.537 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.537 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.537 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.537 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392220+0000", 2024-09-16T14:49:19.537 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3850000000000002e-06 2024-09-16T14:49:19.537 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.537 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.537 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.537 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.396961+0000", 2024-09-16T14:49:19.537 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.004740976 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397029+0000", 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7889e-05 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397131+0000", 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000102315 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400259+0000", 2024-09-16T14:49:19.538 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031285229999999998 2024-09-16T14:49:19.539 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.539 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.539 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:19.539 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400501+0000", 2024-09-16T14:49:19.539 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00024211999999999999 2024-09-16T14:49:19.539 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.539 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.539 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:19.539 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400530+0000", 2024-09-16T14:49:19.539 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8935999999999999e-05 2024-09-16T14:49:19.539 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.539 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.539 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.539 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.400555+0000", 2024-09-16T14:49:19.540 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4861000000000001e-05 2024-09-16T14:49:19.540 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.540 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.540 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.540 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.540 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.540 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-16T14:49:19.540 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:49:19.540 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 56.132564938000002, 2024-09-16T14:49:19.540 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010593673, 2024-09-16T14:49:19.540 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.540 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:19.540 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.541 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:19.541 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:19.541 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 165 2024-09-16T14:49:19.541 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.541 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.541 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.541 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.541 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:49:19.541 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.542 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.542 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.542 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.542 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:49:19.542 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.542 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.542 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.542 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.542 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392223+0000", 2024-09-16T14:49:19.542 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:19.542 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.542 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.542 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.542 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392226+0000", 2024-09-16T14:49:19.543 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2789999999999999e-06 2024-09-16T14:49:19.543 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.543 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.543 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.543 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392234+0000", 2024-09-16T14:49:19.543 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2369999999999992e-06 2024-09-16T14:49:19.543 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.543 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.543 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.543 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.392237+0000", 2024-09-16T14:49:19.543 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8619999999999999e-06 2024-09-16T14:49:19.543 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.543 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.543 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.544 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397503+0000", 2024-09-16T14:49:19.544 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0052656220000000002 2024-09-16T14:49:19.544 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.544 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.544 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.544 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397558+0000", 2024-09-16T14:49:19.544 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5386999999999997e-05 2024-09-16T14:49:19.544 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.544 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.544 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:19.544 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.397628+0000", 2024-09-16T14:49:19.544 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0119999999999999e-05 2024-09-16T14:49:19.544 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.545 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.545 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:19.545 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402533+0000", 2024-09-16T14:49:19.545 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049046200000000002 2024-09-16T14:49:19.545 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.545 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.545 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:19.545 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402778+0000", 2024-09-16T14:49:19.545 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000245348 2024-09-16T14:49:19.545 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.545 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.545 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:19.545 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402800+0000", 2024-09-16T14:49:19.545 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2277e-05 2024-09-16T14:49:19.546 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.546 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.546 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.546 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:23.402817+0000", 2024-09-16T14:49:19.546 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6265000000000001e-05 2024-09-16T14:49:19.546 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.546 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.546 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.546 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.546 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.546 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:309 3.ds0 3:b9c0d236:::smithi04040151-19:head [append 0~737280] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:19.546 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:49:19.546 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 34.585373801999999, 2024-09-16T14:49:19.546 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01003656, 2024-09-16T14:49:19.547 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.547 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:19.547 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.547 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.547 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.547 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 309 2024-09-16T14:49:19.547 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.547 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.547 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.547 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.547 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:49:19.547 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.547 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.548 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.548 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.548 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939414+0000", 2024-09-16T14:49:19.548 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.548 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.548 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.548 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.548 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.939413+0000", 2024-09-16T14:49:19.548 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:19.548 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.548 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.548 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.548 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942226+0000", 2024-09-16T14:49:19.549 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0028130450000000001 2024-09-16T14:49:19.549 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.549 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.549 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.549 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942238+0000", 2024-09-16T14:49:19.549 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2639e-05 2024-09-16T14:49:19.549 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.549 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.549 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.549 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942245+0000", 2024-09-16T14:49:19.549 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3840000000000002e-06 2024-09-16T14:49:19.549 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.549 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.549 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942288+0000", 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3186000000000003e-05 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.942368+0000", 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0162999999999998e-05 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943660+0000", 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001291698 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.550 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:19.551 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.947440+0000", 2024-09-16T14:49:19.551 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003780079 2024-09-16T14:49:19.551 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.551 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.551 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:19.551 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.949391+0000", 2024-09-16T14:49:19.551 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0019512990000000001 2024-09-16T14:49:19.551 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.551 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.551 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.551 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.949451+0000", 2024-09-16T14:49:19.551 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9404000000000002e-05 2024-09-16T14:49:19.551 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.552 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.552 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.552 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.552 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.552 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:312 3.ds0 3:b9c0d236:::smithi04040151-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:19.552 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:49:19.552 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 34.581394287000002, 2024-09-16T14:49:19.552 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084967020000000001, 2024-09-16T14:49:19.552 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.552 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:19.552 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.552 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.552 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.553 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 312 2024-09-16T14:49:19.553 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.553 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.553 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.553 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.553 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:49:19.553 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.553 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.553 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.553 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.553 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943394+0000", 2024-09-16T14:49:19.553 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.553 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.553 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.554 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.554 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943393+0000", 2024-09-16T14:49:19.554 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:19.554 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.554 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.554 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.554 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943396+0000", 2024-09-16T14:49:19.554 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6079999999999998e-06 2024-09-16T14:49:19.554 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.554 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.554 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.554 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943402+0000", 2024-09-16T14:49:19.554 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6100000000000002e-06 2024-09-16T14:49:19.554 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.555 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.555 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.555 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.943413+0000", 2024-09-16T14:49:19.555 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1178e-05 2024-09-16T14:49:19.555 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.555 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.555 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.555 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.946818+0000", 2024-09-16T14:49:19.555 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034043049999999998 2024-09-16T14:49:19.555 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.555 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.555 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:19.555 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.946922+0000", 2024-09-16T14:49:19.555 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000104076 2024-09-16T14:49:19.556 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.556 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.556 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.556 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.950275+0000", 2024-09-16T14:49:19.556 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0033530220000000002 2024-09-16T14:49:19.556 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.556 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.556 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.556 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.950357+0000", 2024-09-16T14:49:19.556 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2087000000000002e-05 2024-09-16T14:49:19.556 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.556 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.556 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:44.951890+0000", 2024-09-16T14:49:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001533333 2024-09-16T14:49:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:390 3.ds0 3:b855c7ac:::smithi04040151-47:head [append 0~0] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:53.864520+0000", 2024-09-16T14:49:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 25.660268090999999, 2024-09-16T14:49:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089883559999999994, 2024-09-16T14:49:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.558 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.558 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.558 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 390 2024-09-16T14:49:19.558 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.558 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.558 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.558 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.558 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864520+0000", 2024-09-16T14:49:19.558 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.558 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.558 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.558 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.558 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864520+0000", 2024-09-16T14:49:19.558 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.559 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.559 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.559 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.559 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864519+0000", 2024-09-16T14:49:19.559 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:19.559 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.559 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.559 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.559 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:19.559 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0545e-05 2024-09-16T14:49:19.559 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.559 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.559 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.559 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864536+0000", 2024-09-16T14:49:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8069999999999996e-06 2024-09-16T14:49:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.864541+0000", 2024-09-16T14:49:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0900000000000004e-06 2024-09-16T14:49:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.866852+0000", 2024-09-16T14:49:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0023115399999999999 2024-09-16T14:49:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.561 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.866946+0000", 2024-09-16T14:49:19.561 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3311000000000002e-05 2024-09-16T14:49:19.561 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.561 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.561 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:19.561 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.867413+0000", 2024-09-16T14:49:19.561 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00046786499999999998 2024-09-16T14:49:19.561 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.561 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.561 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:19.561 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.868091+0000", 2024-09-16T14:49:19.561 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00067759500000000002 2024-09-16T14:49:19.561 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.561 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.873468+0000", 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053766930000000001 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:53.873508+0000", 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0314000000000001e-05 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:459 3.ds0 3:b9c0d236:::smithi04040151-19:head [create,setxattr bk]W^gTaniZ\\TV]dqj_UT\\QgY_\\pYYUOcYdSe`m_ (1859) in=40b,setxattr XSPcap`qmQ\\ZXYl^_bQYNltYZfOtfbRjqmVk_mOa (419) in=40b,setxattr l_Qd^aXhnapg`O][bWecsVdjlWkdYc^tOnjZh]^m (1577) in=40b,setxattr facegtqpaQroppsVbsrZelUak]RbamP`srOTdms[ (350) in=40b,setxattr QllprTts[liem]moe]Pi`nNtcQko (1402) in=28b,setxattr sjm^Un[ahPriW\\TNkaYS\\NnPZfZ\\Zmq[rQ^TWYZm (1892) in=40b,setxattr kRdb__OpRdYV^sYmYmYdhes^Vbf]QaV`SVZkhRcY (209) in=40b,setxattr b[QTnoQc[_Yc^RmdY[ (1402) in=18b,setxattr ^iZkhfc`oO (1402) in=10b,setxattr d\\QUOkoScOhonabPS_YeNlheOOeOal]aQ^ZW]lc` (2805) in=40b,setxattr SioYcliigTgheX\\jnVqipcelfYhbZRjQRet]mQQU (1365) in=40b,setxattr jWcgaRWbr\\ZQ_XXr_eenPa]sXtbaPlmeshkgjOsm (1192) in=40b,setxattr WmVtOY (1402) in=6b,setxattr ]tk_t\\gnSfccZfYfnWnaqP (1402) in=22b,setxattr dS_dYomnsYPtQarmqoZQkO]Ofk_SfgjgitphTZPg (18524) in=40b,setxattr g_a^Zpt`YVahZiOW]V[jQ\\QccqSTo_ma[\\c\\[aft (454) in=40b,setxattr qmmdmgOR__hN\\UTjpR_gbY^s_pOVSdbTob`k (1402) in=36b,setxattr cW`PPOYrbfXV^hmaWhVNb`qVSXqjcdej`pY][]l^ (665) in=40b,setxattr jjo^VgcaQffrnVNPlnq_iTmcrlnTOenhaZV[_SZf (453) in=40b,setxattr n^nqj]bnVcl\\njtUkprb\\laSh^Rbmam_Z_NkeXqj (595) in=40b] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:19.562 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.993572+0000", 2024-09-16T14:49:19.563 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 21.531216085000001, 2024-09-16T14:49:19.563 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010337796999999999, 2024-09-16T14:49:19.563 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.563 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:19.563 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.563 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.563 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.563 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 459 2024-09-16T14:49:19.563 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.563 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.563 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.563 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.563 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993572+0000", 2024-09-16T14:49:19.563 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.564 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.564 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.564 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.564 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993572+0000", 2024-09-16T14:49:19.564 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.564 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.564 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.564 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.564 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993569+0000", 2024-09-16T14:49:19.564 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:19.564 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.564 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.564 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.564 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993625+0000", 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5105000000000002e-05 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993647+0000", 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1962000000000001e-05 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993656+0000", 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3179999999999996e-06 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.565 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993717+0000", 2024-09-16T14:49:19.566 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.0738e-05 2024-09-16T14:49:19.566 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.566 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.566 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.566 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.993841+0000", 2024-09-16T14:49:19.566 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000124537 2024-09-16T14:49:19.566 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.566 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.566 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:19.566 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.994891+0000", 2024-09-16T14:49:19.566 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0010499719999999999 2024-09-16T14:49:19.566 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.566 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.566 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:19.567 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.999326+0000", 2024-09-16T14:49:19.567 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044351240000000004 2024-09-16T14:49:19.567 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.567 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.567 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:19.567 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003847+0000", 2024-09-16T14:49:19.567 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045207629999999997 2024-09-16T14:49:19.567 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.567 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.567 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.567 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003910+0000", 2024-09-16T14:49:19.567 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2527999999999995e-05 2024-09-16T14:49:19.567 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.567 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.568 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.568 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.568 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.568 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:535 3.fs0 3:ff6b916c:::smithi04040151-25:head [append 0~786432] snapc 1e=[1e,1c,1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:19.568 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:19.568 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.593425945, 2024-09-16T14:49:19.568 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013876462000000001, 2024-09-16T14:49:19.568 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.568 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:19.568 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.568 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.568 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.568 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 535 2024-09-16T14:49:19.568 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.569 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.569 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.569 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.569 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:19.569 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.569 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.569 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.569 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.569 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:19.569 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.569 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.569 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.569 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.569 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931360+0000", 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933014+0000", 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001653715 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933028+0000", 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4239999999999999e-05 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.570 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933036+0000", 2024-09-16T14:49:19.571 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8019999999999992e-06 2024-09-16T14:49:19.571 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.571 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.571 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.571 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933084+0000", 2024-09-16T14:49:19.571 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7663000000000001e-05 2024-09-16T14:49:19.571 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.571 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.571 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.571 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933184+0000", 2024-09-16T14:49:19.571 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000100549 2024-09-16T14:49:19.571 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.571 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.571 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.934324+0000", 2024-09-16T14:49:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011399229999999999 2024-09-16T14:49:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941715+0000", 2024-09-16T14:49:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.007391023 2024-09-16T14:49:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.945187+0000", 2024-09-16T14:49:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034717709999999998 2024-09-16T14:49:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.572 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.573 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.573 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.945238+0000", 2024-09-16T14:49:19.573 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.1459999999999999e-05 2024-09-16T14:49:19.573 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.573 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.573 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.573 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.573 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.573 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:535 3.fs0 3:ff6b916c:::smithi04040151-25:head [append 0~786432] snapc 1e=[1e,1c,1b] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:19.573 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.936346+0000", 2024-09-16T14:49:19.573 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.588441611, 2024-09-16T14:49:19.573 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0088507770000000006, 2024-09-16T14:49:19.573 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.573 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "waiting for ondisk", 2024-09-16T14:49:19.574 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.574 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.574 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.574 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 535 2024-09-16T14:49:19.574 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.574 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.574 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.574 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.574 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.936346+0000", 2024-09-16T14:49:19.575 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.575 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.575 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.575 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.575 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.936346+0000", 2024-09-16T14:49:19.575 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.575 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.575 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.575 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.575 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.936345+0000", 2024-09-16T14:49:19.575 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:19.575 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.575 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.575 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.576 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937664+0000", 2024-09-16T14:49:19.576 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0013194000000000001 2024-09-16T14:49:19.576 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.576 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.576 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.576 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937679+0000", 2024-09-16T14:49:19.576 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4588e-05 2024-09-16T14:49:19.576 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.576 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.576 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.576 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937685+0000", 2024-09-16T14:49:19.576 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2389999999999999e-06 2024-09-16T14:49:19.576 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.576 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937772+0000", 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6396000000000004e-05 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for ondisk", 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937865+0000", 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3029999999999995e-05 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.945197+0000", 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0073322810000000004 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.577 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.578 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.578 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.578 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.578 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:536 3.fs0 3:ff6b916c:::smithi04040151-25:head [append 0~491520] snapc 1e=[1e,1c,1b] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:19.578 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.937695+0000", 2024-09-16T14:49:19.578 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.587093302, 2024-09-16T14:49:19.578 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0087034420000000005, 2024-09-16T14:49:19.578 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.578 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:19.578 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.578 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.578 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.578 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 536 2024-09-16T14:49:19.578 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937695+0000", 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937695+0000", 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.937693+0000", 2024-09-16T14:49:19.579 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:19.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.580 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.580 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.580 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938195+0000", 2024-09-16T14:49:19.580 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00050157700000000001 2024-09-16T14:49:19.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.580 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.580 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.580 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938204+0000", 2024-09-16T14:49:19.580 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7870000000000007e-06 2024-09-16T14:49:19.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.580 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.580 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.580 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938208+0000", 2024-09-16T14:49:19.581 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.318e-06 2024-09-16T14:49:19.581 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.581 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.581 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.581 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938247+0000", 2024-09-16T14:49:19.581 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9214000000000001e-05 2024-09-16T14:49:19.581 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.581 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.581 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.581 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938351+0000", 2024-09-16T14:49:19.581 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000103931 2024-09-16T14:49:19.581 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.581 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.582 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:19.582 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939194+0000", 2024-09-16T14:49:19.582 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00084270199999999997 2024-09-16T14:49:19.582 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.582 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.582 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:19.582 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941717+0000", 2024-09-16T14:49:19.582 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025227100000000001 2024-09-16T14:49:19.582 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.582 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.582 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:19.582 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.946316+0000", 2024-09-16T14:49:19.582 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045990639999999999 2024-09-16T14:49:19.582 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.583 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.583 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.583 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.946398+0000", 2024-09-16T14:49:19.583 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2218999999999993e-05 2024-09-16T14:49:19.583 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.583 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.583 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.583 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.583 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.583 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:537 3.fs0 3:ff6b916c:::smithi04040151-25:head [append 0~278528] snapc 1e=[1e,1c,1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:19.583 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.938214+0000", 2024-09-16T14:49:19.583 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.586574152000001, 2024-09-16T14:49:19.583 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089476360000000001, 2024-09-16T14:49:19.583 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.584 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:19.584 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.584 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.584 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.584 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 537 2024-09-16T14:49:19.584 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.584 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.584 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.584 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.584 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938214+0000", 2024-09-16T14:49:19.584 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.584 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.584 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.584 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.585 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938214+0000", 2024-09-16T14:49:19.585 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.585 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.585 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.585 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.585 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938213+0000", 2024-09-16T14:49:19.585 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:19.585 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.585 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.585 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.585 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938431+0000", 2024-09-16T14:49:19.585 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00021819100000000001 2024-09-16T14:49:19.585 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.585 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.586 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.586 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938440+0000", 2024-09-16T14:49:19.586 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0210000000000005e-06 2024-09-16T14:49:19.586 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.586 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.586 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.586 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938447+0000", 2024-09-16T14:49:19.586 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.5200000000000003e-06 2024-09-16T14:49:19.586 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.586 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.586 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.586 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939886+0000", 2024-09-16T14:49:19.586 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0014394950000000001 2024-09-16T14:49:19.586 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.587 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.587 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.587 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.939979+0000", 2024-09-16T14:49:19.587 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2519e-05 2024-09-16T14:49:19.587 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.587 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.587 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:19.587 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.940972+0000", 2024-09-16T14:49:19.587 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00099328099999999994 2024-09-16T14:49:19.587 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.587 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.587 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:19.587 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.944530+0000", 2024-09-16T14:49:19.587 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003557639 2024-09-16T14:49:19.588 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.588 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.588 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:19.588 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947108+0000", 2024-09-16T14:49:19.588 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025781100000000002 2024-09-16T14:49:19.588 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.588 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.588 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.588 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.947161+0000", 2024-09-16T14:49:19.588 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3510000000000001e-05 2024-09-16T14:49:19.588 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.588 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.588 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.588 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:538 3.fs0 3:ff6b916c:::smithi04040151-25:head [setxattr _header (57) in=7b] snapc 1e=[1e,1c,1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.938463+0000", 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.586325091999999, 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010050811999999999, 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 538 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.589 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.590 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938463+0000", 2024-09-16T14:49:19.590 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.590 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.590 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.590 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.590 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938463+0000", 2024-09-16T14:49:19.590 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.590 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.590 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.590 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.590 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938462+0000", 2024-09-16T14:49:19.590 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:19.590 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.590 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938466+0000", 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.7979999999999999e-06 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938473+0000", 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2910000000000002e-06 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938478+0000", 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3079999999999998e-06 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.591 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941806+0000", 2024-09-16T14:49:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0033274899999999998 2024-09-16T14:49:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941906+0000", 2024-09-16T14:49:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010026600000000001 2024-09-16T14:49:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942348+0000", 2024-09-16T14:49:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00044172800000000001 2024-09-16T14:49:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.593 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.593 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:19.593 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.944534+0000", 2024-09-16T14:49:19.593 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021862439999999999 2024-09-16T14:49:19.593 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.593 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.593 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:19.593 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.948427+0000", 2024-09-16T14:49:19.593 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0038925370000000002 2024-09-16T14:49:19.593 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.593 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.593 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.593 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.948514+0000", 2024-09-16T14:49:19.593 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6868999999999998e-05 2024-09-16T14:49:19.594 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.594 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.594 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.594 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.594 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.594 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:539 3.fs0 3:ff6b916c:::smithi04040151-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:19.594 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:19.594 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.586304413000001, 2024-09-16T14:49:19.594 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01551078, 2024-09-16T14:49:19.594 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.594 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:19.594 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.594 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.594 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 539 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.595 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.596 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:19.596 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:19.596 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.596 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.596 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.596 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938486+0000", 2024-09-16T14:49:19.596 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7779999999999999e-06 2024-09-16T14:49:19.596 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.596 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.596 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.596 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938492+0000", 2024-09-16T14:49:19.596 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8120000000000002e-06 2024-09-16T14:49:19.596 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.596 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.597 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.597 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938495+0000", 2024-09-16T14:49:19.597 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.948e-06 2024-09-16T14:49:19.597 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.597 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.597 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.597 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942566+0000", 2024-09-16T14:49:19.597 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0040700839999999999 2024-09-16T14:49:19.597 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.597 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.597 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:19.597 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942659+0000", 2024-09-16T14:49:19.597 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3887000000000004e-05 2024-09-16T14:49:19.597 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.598 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.598 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.598 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.948875+0000", 2024-09-16T14:49:19.598 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0062154630000000001 2024-09-16T14:49:19.598 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.598 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.598 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.598 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.948968+0000", 2024-09-16T14:49:19.598 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2758999999999997e-05 2024-09-16T14:49:19.598 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.598 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.598 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.598 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.953994+0000", 2024-09-16T14:49:19.598 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050265580000000004 2024-09-16T14:49:19.599 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.599 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.599 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.599 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.599 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.599 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:600 3.ds0 3:bcba132a:::smithi04040151-1:head [append 0~507904] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:19.599 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.993737+0000", 2024-09-16T14:49:19.599 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 7.5310510510000004, 2024-09-16T14:49:19.599 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.008095819, 2024-09-16T14:49:19.599 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.599 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:19.599 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.599 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.599 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 600 2024-09-16T14:49:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.993737+0000", 2024-09-16T14:49:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.993737+0000", 2024-09-16T14:49:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.600 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.601 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.601 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.993735+0000", 2024-09-16T14:49:19.601 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:19.601 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.601 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.601 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.601 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994263+0000", 2024-09-16T14:49:19.601 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00052718199999999995 2024-09-16T14:49:19.601 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.601 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.601 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.601 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994274+0000", 2024-09-16T14:49:19.601 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1535999999999999e-05 2024-09-16T14:49:19.601 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994279+0000", 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0839999999999999e-06 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994330+0000", 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0467999999999999e-05 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994332+0000", 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:49:19.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.603 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.603 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.603 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980708+0000", 2024-09-16T14:49:19.603 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98637637199999995 2024-09-16T14:49:19.603 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.603 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.603 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:19.603 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.981437+0000", 2024-09-16T14:49:19.603 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00072860299999999998 2024-09-16T14:49:19.603 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.603 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.603 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.603 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.989211+0000", 2024-09-16T14:49:19.603 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077741490000000002 2024-09-16T14:49:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.989340+0000", 2024-09-16T14:49:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00012908900000000001 2024-09-16T14:49:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.990969+0000", 2024-09-16T14:49:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001628612 2024-09-16T14:49:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:19.604 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996293+0000", 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053247440000000002 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.001434+0000", 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0051404349999999996 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.001833+0000", 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00039878500000000002 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.605 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.606 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.606 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:601 3.ds0 3:bcba132a:::smithi04040151-1:head [append 0~606208] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:19.606 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.998248+0000", 2024-09-16T14:49:19.606 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 7.5265397329999999, 2024-09-16T14:49:19.606 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0062125260000001, 2024-09-16T14:49:19.606 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.606 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:19.606 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.606 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.606 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.606 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 601 2024-09-16T14:49:19.606 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.606 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.606 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.607 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.607 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998248+0000", 2024-09-16T14:49:19.607 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.607 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.607 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.607 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.607 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998248+0000", 2024-09-16T14:49:19.607 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.607 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.607 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.607 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.607 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998246+0000", 2024-09-16T14:49:19.607 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:19.607 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.608 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.608 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.608 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998747+0000", 2024-09-16T14:49:19.608 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00050040399999999997 2024-09-16T14:49:19.608 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.608 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.608 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.608 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998755+0000", 2024-09-16T14:49:19.608 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3329999999999997e-06 2024-09-16T14:49:19.608 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.608 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.608 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.608 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998759+0000", 2024-09-16T14:49:19.608 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8380000000000004e-06 2024-09-16T14:49:19.609 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.609 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.609 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.609 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998846+0000", 2024-09-16T14:49:19.609 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6456e-05 2024-09-16T14:49:19.609 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.609 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.609 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:19.609 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998847+0000", 2024-09-16T14:49:19.609 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.635e-06 2024-09-16T14:49:19.609 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.609 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.609 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.609 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984175+0000", 2024-09-16T14:49:19.610 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98532833399999997 2024-09-16T14:49:19.610 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.610 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.610 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:19.610 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984375+0000", 2024-09-16T14:49:19.610 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00019923299999999999 2024-09-16T14:49:19.610 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.610 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.610 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.610 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992431+0000", 2024-09-16T14:49:19.610 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0080562129999999996 2024-09-16T14:49:19.610 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.610 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.610 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992552+0000", 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000120986 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.993913+0000", 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001361206 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996295+0000", 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0023814629999999999 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.611 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004101+0000", 2024-09-16T14:49:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0078063500000000001 2024-09-16T14:49:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004461+0000", 2024-09-16T14:49:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00035967799999999999 2024-09-16T14:49:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:602 3.ds0 3:bcba132a:::smithi04040151-1:head [append 0~557056] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:19.612 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:19.613 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 7.5227275139999996, 2024-09-16T14:49:19.613 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0032171249999999, 2024-09-16T14:49:19.613 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.613 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:19.613 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.613 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.613 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.613 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 602 2024-09-16T14:49:19.613 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.613 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.613 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.613 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.613 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:19.613 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.614 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.614 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.614 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.614 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:19.614 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.614 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.614 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.614 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.614 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:19.614 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:19.614 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.614 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.614 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.614 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002547+0000", 2024-09-16T14:49:19.615 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00048700700000000002 2024-09-16T14:49:19.615 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.615 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.615 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.615 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002554+0000", 2024-09-16T14:49:19.615 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.435e-06 2024-09-16T14:49:19.615 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.615 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.615 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.615 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002558+0000", 2024-09-16T14:49:19.615 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.501e-06 2024-09-16T14:49:19.615 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.615 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.615 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.616 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002641+0000", 2024-09-16T14:49:19.616 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2874999999999995e-05 2024-09-16T14:49:19.616 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.616 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.616 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:19.616 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002642+0000", 2024-09-16T14:49:19.616 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6670000000000001e-06 2024-09-16T14:49:19.616 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.616 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.616 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.616 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984450+0000", 2024-09-16T14:49:19.616 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98180768500000004 2024-09-16T14:49:19.616 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.616 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984591+0000", 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014090700000000001 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996586+0000", 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011994898 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996695+0000", 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000108962 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.617 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.618 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:19.618 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.997804+0000", 2024-09-16T14:49:19.618 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011095090000000001 2024-09-16T14:49:19.618 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.618 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.618 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:19.618 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000509+0000", 2024-09-16T14:49:19.618 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0027042440000000002 2024-09-16T14:49:19.618 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.618 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.618 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:19.618 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004755+0000", 2024-09-16T14:49:19.618 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0042461269999999997 2024-09-16T14:49:19.618 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.619 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.619 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.619 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005277+0000", 2024-09-16T14:49:19.619 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00052277599999999997 2024-09-16T14:49:19.619 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.619 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.619 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.619 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.619 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.619 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:603 3.ds0 3:bcba132a:::smithi04040151-1:head [setxattr _header (56) in=7b] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:19.619 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:19.619 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 7.522207925, 2024-09-16T14:49:19.619 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.003299656, 2024-09-16T14:49:19.619 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.620 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:19.620 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.620 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.620 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.620 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 603 2024-09-16T14:49:19.620 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.620 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.620 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.620 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.620 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:19.620 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.620 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.621 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.621 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.621 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:19.621 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.621 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.621 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.621 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.621 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:19.621 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:19.621 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.621 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.621 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.622 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002582+0000", 2024-09-16T14:49:19.622 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.345e-06 2024-09-16T14:49:19.622 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.622 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.622 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.622 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002586+0000", 2024-09-16T14:49:19.622 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4710000000000001e-06 2024-09-16T14:49:19.622 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.622 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.622 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.622 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002589+0000", 2024-09-16T14:49:19.622 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.694e-06 2024-09-16T14:49:19.622 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.622 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.623 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.623 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002673+0000", 2024-09-16T14:49:19.623 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4349000000000002e-05 2024-09-16T14:49:19.623 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.623 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.623 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:19.623 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002674+0000", 2024-09-16T14:49:19.623 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9300000000000006e-07 2024-09-16T14:49:19.623 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.623 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.623 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.623 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984634+0000", 2024-09-16T14:49:19.623 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98195955099999999 2024-09-16T14:49:19.623 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.624 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.624 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:19.624 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984742+0000", 2024-09-16T14:49:19.624 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000108491 2024-09-16T14:49:19.624 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.624 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.624 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.624 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.998850+0000", 2024-09-16T14:49:19.624 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014108005 2024-09-16T14:49:19.624 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.624 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.624 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.624 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.999006+0000", 2024-09-16T14:49:19.625 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015594799999999999 2024-09-16T14:49:19.625 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.625 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.625 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:19.625 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.999686+0000", 2024-09-16T14:49:19.625 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00067984999999999996 2024-09-16T14:49:19.625 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.625 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.625 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:19.625 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000622+0000", 2024-09-16T14:49:19.625 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00093585200000000004 2024-09-16T14:49:19.625 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.625 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.625 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:19.626 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005500+0000", 2024-09-16T14:49:19.626 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048775049999999999 2024-09-16T14:49:19.626 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.626 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.626 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.626 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005880+0000", 2024-09-16T14:49:19.626 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00037999399999999998 2024-09-16T14:49:19.626 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.626 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.626 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.626 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.626 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.626 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:604 3.ds0 3:bcba132a:::smithi04040151-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:19.626 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 7.5221963829999998, 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.00580336, 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 604 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.627 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.628 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.628 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.628 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:19.628 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.628 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.628 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.628 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.628 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:19.628 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:19.628 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.628 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.628 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.628 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002593+0000", 2024-09-16T14:49:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0559999999999999e-06 2024-09-16T14:49:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002597+0000", 2024-09-16T14:49:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.7349999999999998e-06 2024-09-16T14:49:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002601+0000", 2024-09-16T14:49:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6519999999999999e-06 2024-09-16T14:49:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.629 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.630 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002698+0000", 2024-09-16T14:49:19.630 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7378000000000002e-05 2024-09-16T14:49:19.630 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.630 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.630 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:19.630 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002699+0000", 2024-09-16T14:49:19.630 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.5099999999999999e-07 2024-09-16T14:49:19.630 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.630 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.630 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.630 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984866+0000", 2024-09-16T14:49:19.630 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98216700499999998 2024-09-16T14:49:19.630 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.630 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.631 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:19.631 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984953+0000", 2024-09-16T14:49:19.631 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6847000000000004e-05 2024-09-16T14:49:19.631 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.631 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.631 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.631 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000753+0000", 2024-09-16T14:49:19.631 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015800455000000001 2024-09-16T14:49:19.631 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.631 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.631 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:19.631 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000883+0000", 2024-09-16T14:49:19.631 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00013054000000000001 2024-09-16T14:49:19.631 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.632 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.632 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.632 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006482+0000", 2024-09-16T14:49:19.632 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0055987839999999999 2024-09-16T14:49:19.632 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.632 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.632 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.632 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006630+0000", 2024-09-16T14:49:19.632 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014820600000000001 2024-09-16T14:49:19.632 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.632 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.632 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.632 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.008395+0000", 2024-09-16T14:49:19.632 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0017643139999999999 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(osd.11.18:2 3.ds0 3:b855c7ac:::smithi04040151-47:head [assert-version v27,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.984427+0000", 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 6.5403605789999997, 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011694438999999999, 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.633 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984427+0000", 2024-09-16T14:49:19.634 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.634 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.634 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.634 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.634 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984427+0000", 2024-09-16T14:49:19.634 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.634 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.634 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.634 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.634 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984426+0000", 2024-09-16T14:49:19.634 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:19.634 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.634 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.634 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.635 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984456+0000", 2024-09-16T14:49:19.635 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0670000000000003e-05 2024-09-16T14:49:19.635 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.635 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.635 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.635 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984470+0000", 2024-09-16T14:49:19.635 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3492000000000001e-05 2024-09-16T14:49:19.635 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.635 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.635 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.635 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984478+0000", 2024-09-16T14:49:19.635 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9680000000000006e-06 2024-09-16T14:49:19.635 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.636 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.636 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.636 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.985078+0000", 2024-09-16T14:49:19.636 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00059978199999999998 2024-09-16T14:49:19.636 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.636 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.636 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.636 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.985345+0000", 2024-09-16T14:49:19.636 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00026778700000000002 2024-09-16T14:49:19.636 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.636 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.636 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.636 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996122+0000", 2024-09-16T14:49:19.636 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010776365 2024-09-16T14:49:19.637 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.637 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.637 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.637 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.637 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.637 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:643 3.fs0 3:f17be48c:::smithi04040151-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e70)", 2024-09-16T14:49:19.637 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:16.894120+0000", 2024-09-16T14:49:19.637 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 2.6306677600000001, 2024-09-16T14:49:19.637 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083996120000000007, 2024-09-16T14:49:19.637 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:19.637 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:19.637 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:19.637 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:19.637 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 643 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.894120+0000", 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.894120+0000", 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.638 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:19.639 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.894120+0000", 2024-09-16T14:49:19.639 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:19.639 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.639 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.639 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:19.639 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.894123+0000", 2024-09-16T14:49:19.639 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1650000000000002e-06 2024-09-16T14:49:19.639 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.639 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.639 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:19.639 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.894127+0000", 2024-09-16T14:49:19.639 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.873e-06 2024-09-16T14:49:19.639 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.639 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.640 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:19.640 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.894130+0000", 2024-09-16T14:49:19.640 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7520000000000001e-06 2024-09-16T14:49:19.640 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.640 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.640 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.640 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.896491+0000", 2024-09-16T14:49:19.640 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002361681 2024-09-16T14:49:19.640 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.640 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.640 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:19.640 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.896583+0000", 2024-09-16T14:49:19.640 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1623e-05 2024-09-16T14:49:19.640 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.641 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.641 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:19.641 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.901638+0000", 2024-09-16T14:49:19.641 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050549030000000003 2024-09-16T14:49:19.641 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.641 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.641 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:19.641 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.901725+0000", 2024-09-16T14:49:19.641 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7423000000000006e-05 2024-09-16T14:49:19.641 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:19.641 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:19.641 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:19.641 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.902520+0000", 2024-09-16T14:49:19.641 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00079451200000000004 2024-09-16T14:49:19.642 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.642 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.642 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.642 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:19.642 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:19.642 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:19.643 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:19.643 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:19.644 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:19.644 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:19.644 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:49:19.645 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:19.645 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:19.645 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:19.646 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:19.646 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:19.647 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:19.647 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:19.642+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:19.744 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:19.742+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:19.762 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:19.762 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:19.771 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:49:19.845 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:19.844+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:19.853 INFO:tasks.rados.rados.0.smithi040.stdout:270: done (0 left) 2024-09-16T14:49:19.853 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 39 current snap is 24 2024-09-16T14:49:19.853 INFO:tasks.rados.rados.0.smithi040.stdout:271: seq_num 105 ranges {671744=606208,1277952=688128,1966080=475136} 2024-09-16T14:49:19.854 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is set 2024-09-16T14:49:19.854 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T14:49:19.861 INFO:tasks.rados.rados.0.smithi040.stdout:271: writing smithi04040151-39 from 671744 to 1277952 tid 1 2024-09-16T14:49:19.866 INFO:tasks.rados.rados.0.smithi040.stdout:271: writing smithi04040151-39 from 1277952 to 1966080 tid 2 2024-09-16T14:49:19.870 INFO:tasks.rados.rados.0.smithi040.stdout:271: writing smithi04040151-39 from 1966080 to 2441216 tid 3 2024-09-16T14:49:19.870 INFO:tasks.rados.rados.0.smithi040.stdout:272: read oid 50 snap -1 2024-09-16T14:49:19.870 INFO:tasks.rados.rados.0.smithi040.stdout:272: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-16T14:49:19.870 INFO:tasks.rados.rados.0.smithi040.stdout:273: delete oid 21 current snap is 24 2024-09-16T14:49:19.876 INFO:tasks.rados.rados.0.smithi040.stdout:272: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-16T14:49:19.886 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T14:49:19.886 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:19.886 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:19.895 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:49:19.904 INFO:tasks.rados.rados.0.smithi040.stdout:271: finishing write tid 1 to smithi04040151-39 2024-09-16T14:49:19.904 INFO:tasks.rados.rados.0.smithi040.stdout:271: finishing write tid 2 to smithi04040151-39 2024-09-16T14:49:19.904 INFO:tasks.rados.rados.0.smithi040.stdout:271: finishing write tid 3 to smithi04040151-39 2024-09-16T14:49:19.904 INFO:tasks.rados.rados.0.smithi040.stdout:271: finishing write tid 4 to smithi04040151-39 2024-09-16T14:49:19.904 INFO:tasks.rados.rados.0.smithi040.stdout:271: finishing write tid 5 to smithi04040151-39 2024-09-16T14:49:19.904 INFO:tasks.rados.rados.0.smithi040.stdout:271: oid 39 updating version 0 to 40 2024-09-16T14:49:19.904 INFO:tasks.rados.rados.0.smithi040.stdout:271: oid 39 updating version 40 to 41 2024-09-16T14:49:19.904 INFO:tasks.rados.rados.0.smithi040.stdout:271: oid 39 updating version 41 to 43 2024-09-16T14:49:19.904 INFO:tasks.rados.rados.0.smithi040.stdout:271: oid 39 version 43 is already newer than 42 2024-09-16T14:49:19.905 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 39 v 43 (ObjNum 105 snap 24 seq_num 105) dirty exists 2024-09-16T14:49:19.905 INFO:tasks.rados.rados.0.smithi040.stdout:271: left oid 39 (ObjNum 105 snap 24 seq_num 105) 2024-09-16T14:49:19.905 INFO:tasks.rados.rados.0.smithi040.stdout:271: done (2 left) 2024-09-16T14:49:19.905 INFO:tasks.rados.rados.0.smithi040.stdout:272: done (1 left) 2024-09-16T14:49:19.905 INFO:tasks.rados.rados.0.smithi040.stdout:273: done (0 left) 2024-09-16T14:49:19.905 INFO:tasks.rados.rados.0.smithi040.stdout:274: delete oid 20 current snap is 24 2024-09-16T14:49:19.909 INFO:tasks.rados.rados.0.smithi040.stdout:274: done (0 left) 2024-09-16T14:49:19.909 INFO:tasks.rados.rados.0.smithi040.stdout:275: read oid 10 snap -1 2024-09-16T14:49:19.909 INFO:tasks.rados.rados.0.smithi040.stdout:275: expect deleted 2024-09-16T14:49:19.909 INFO:tasks.rados.rados.0.smithi040.stdout:276: snap_remove snap 22 2024-09-16T14:49:19.980 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:19.979+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:20.037 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:20.038 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:20.046 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:49:20.047 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:20.045+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:20.147 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:20.145+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:20.190 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:20.190 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:20.198 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:49:20.248 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:20.246+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:20.300 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:20.300 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:20.308 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:49:20.349 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:20.348+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:20.449 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:20.448+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:20.451 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:20.451 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:20.460 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:49:20.551 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:20.549+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:20.602 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:20.602 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:20.611 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:49:20.651 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:20.650+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:20.752 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:20.752 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:20.753 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:20.752+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:20.761 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:49:20.853 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:20.852+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:20.862 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:20.862 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:20.871 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:49:20.886 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T14:49:20.886 INFO:tasks.rados.rados.0.smithi040.stdout:276: done (1 left) 2024-09-16T14:49:20.886 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 27 current snap is 24 2024-09-16T14:49:20.887 INFO:tasks.rados.rados.0.smithi040.stdout:277: seq_num 108 ranges {475136=655360,1130496=606208,1736704=475136,2211840=81920} 2024-09-16T14:49:20.892 INFO:tasks.rados.rados.0.smithi040.stdout:277: writing smithi04040151-27 from 475136 to 1130496 tid 1 2024-09-16T14:49:20.896 INFO:tasks.rados.rados.0.smithi040.stdout:277: writing smithi04040151-27 from 1130496 to 1736704 tid 2 2024-09-16T14:49:20.898 INFO:tasks.rados.rados.0.smithi040.stdout:277: writing smithi04040151-27 from 1736704 to 2211840 tid 3 2024-09-16T14:49:20.899 INFO:tasks.rados.rados.0.smithi040.stdout:277: writing smithi04040151-27 from 2211840 to 2293760 tid 4 2024-09-16T14:49:20.899 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 7 current snap is 24 2024-09-16T14:49:20.899 INFO:tasks.rados.rados.0.smithi040.stdout:278: seq_num 109 ranges {1294336=671744,1966080=753664,2719744=409600} 2024-09-16T14:49:20.906 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T14:49:20.911 INFO:tasks.rados.rados.0.smithi040.stdout:278: writing smithi04040151-7 from 1294336 to 1966080 tid 1 2024-09-16T14:49:20.915 INFO:tasks.rados.rados.0.smithi040.stdout:278: writing smithi04040151-7 from 1966080 to 2719744 tid 2 2024-09-16T14:49:20.919 INFO:tasks.rados.rados.0.smithi040.stdout:278: writing smithi04040151-7 from 2719744 to 3129344 tid 3 2024-09-16T14:49:20.919 INFO:tasks.rados.rados.0.smithi040.stdout:279: rmattr oid 50 current snap is 24 2024-09-16T14:49:20.919 INFO:tasks.rados.rados.0.smithi040.stdout:279: done (3 left) 2024-09-16T14:49:20.919 INFO:tasks.rados.rados.0.smithi040.stdout:280: setattr oid 50 current snap is 24 2024-09-16T14:49:20.919 INFO:tasks.rados.rados.0.smithi040.stdout:277: finishing write tid 1 to smithi04040151-27 2024-09-16T14:49:20.919 INFO:tasks.rados.rados.0.smithi040.stdout:277: finishing write tid 2 to smithi04040151-27 2024-09-16T14:49:20.919 INFO:tasks.rados.rados.0.smithi040.stdout:277: finishing write tid 3 to smithi04040151-27 2024-09-16T14:49:20.919 INFO:tasks.rados.rados.0.smithi040.stdout:277: finishing write tid 4 to smithi04040151-27 2024-09-16T14:49:20.919 INFO:tasks.rados.rados.0.smithi040.stdout:277: finishing write tid 5 to smithi04040151-27 2024-09-16T14:49:20.919 INFO:tasks.rados.rados.0.smithi040.stdout:277: finishing write tid 6 to smithi04040151-27 2024-09-16T14:49:20.919 INFO:tasks.rados.rados.0.smithi040.stdout:277: oid 27 updating version 0 to 54 2024-09-16T14:49:20.919 INFO:tasks.rados.rados.0.smithi040.stdout:277: oid 27 updating version 54 to 56 2024-09-16T14:49:20.920 INFO:tasks.rados.rados.0.smithi040.stdout:277: oid 27 version 56 is already newer than 55 2024-09-16T14:49:20.920 INFO:tasks.rados.rados.0.smithi040.stdout:277: oid 27 updating version 56 to 57 2024-09-16T14:49:20.920 INFO:tasks.rados.rados.0.smithi040.stdout:277: oid 27 version 57 is already newer than 53 2024-09-16T14:49:20.920 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 27 v 57 (ObjNum 108 snap 24 seq_num 108) dirty exists 2024-09-16T14:49:20.920 INFO:tasks.rados.rados.0.smithi040.stdout:277: left oid 27 (ObjNum 108 snap 24 seq_num 108) 2024-09-16T14:49:20.921 INFO:tasks.rados.rados.0.smithi040.stdout:278: finishing write tid 1 to smithi04040151-7 2024-09-16T14:49:20.921 INFO:tasks.rados.rados.0.smithi040.stdout:275: done (3 left) 2024-09-16T14:49:20.921 INFO:tasks.rados.rados.0.smithi040.stdout:277: done (2 left) 2024-09-16T14:49:20.921 INFO:tasks.rados.rados.0.smithi040.stdout:281: read oid 1 snap -1 2024-09-16T14:49:20.921 INFO:tasks.rados.rados.0.smithi040.stdout:281: expect (ObjNum 99 snap 18 seq_num 99) 2024-09-16T14:49:20.921 INFO:tasks.rados.rados.0.smithi040.stdout:282: rollback oid 21 current snap is 24 2024-09-16T14:49:20.921 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 21 to 20 2024-09-16T14:49:20.921 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 16 current snap is 24 2024-09-16T14:49:20.921 INFO:tasks.rados.rados.0.smithi040.stdout:283: seq_num 110 ranges {786432=688128,1474560=507904,1982464=655360} 2024-09-16T14:49:20.930 INFO:tasks.rados.rados.0.smithi040.stdout:283: writing smithi04040151-16 from 786432 to 1474560 tid 1 2024-09-16T14:49:20.934 INFO:tasks.rados.rados.0.smithi040.stdout:283: writing smithi04040151-16 from 1474560 to 1982464 tid 2 2024-09-16T14:49:20.938 INFO:tasks.rados.rados.0.smithi040.stdout:283: writing smithi04040151-16 from 1982464 to 2637824 tid 3 2024-09-16T14:49:20.939 INFO:tasks.rados.rados.0.smithi040.stdout:284: snap_create 2024-09-16T14:49:20.939 INFO:tasks.rados.rados.0.smithi040.stdout:278: finishing write tid 2 to smithi04040151-7 2024-09-16T14:49:20.939 INFO:tasks.rados.rados.0.smithi040.stdout:278: finishing write tid 3 to smithi04040151-7 2024-09-16T14:49:20.939 INFO:tasks.rados.rados.0.smithi040.stdout:282: finishing rollback tid 1 to smithi04040151-21 2024-09-16T14:49:20.939 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 21 v 0 (ObjNum 1297 snap 0 seq_num 0) dirty dne 2024-09-16T14:49:20.939 INFO:tasks.rados.rados.0.smithi040.stdout:278: finishing write tid 4 to smithi04040151-7 2024-09-16T14:49:20.939 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 50 v 51 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-16T14:49:20.939 INFO:tasks.rados.rados.0.smithi040.stdout:278: finishing write tid 5 to smithi04040151-7 2024-09-16T14:49:20.939 INFO:tasks.rados.rados.0.smithi040.stdout:278: oid 7 updating version 0 to 28 2024-09-16T14:49:20.940 INFO:tasks.rados.rados.0.smithi040.stdout:278: oid 7 updating version 28 to 29 2024-09-16T14:49:20.940 INFO:tasks.rados.rados.0.smithi040.stdout:278: oid 7 updating version 29 to 31 2024-09-16T14:49:20.940 INFO:tasks.rados.rados.0.smithi040.stdout:278: oid 7 version 31 is already newer than 30 2024-09-16T14:49:20.940 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 7 v 31 (ObjNum 109 snap 24 seq_num 109) dirty exists 2024-09-16T14:49:20.940 INFO:tasks.rados.rados.0.smithi040.stdout:278: left oid 7 (ObjNum 109 snap 24 seq_num 109) 2024-09-16T14:49:20.940 INFO:tasks.rados.rados.0.smithi040.stdout:281: expect (ObjNum 99 snap 18 seq_num 99) 2024-09-16T14:49:20.956 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:20.955+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:20.986 INFO:tasks.rados.rados.0.smithi040.stdout:283: finishing write tid 1 to smithi04040151-16 2024-09-16T14:49:20.986 INFO:tasks.rados.rados.0.smithi040.stdout:283: finishing write tid 2 to smithi04040151-16 2024-09-16T14:49:20.986 INFO:tasks.rados.rados.0.smithi040.stdout:283: finishing write tid 3 to smithi04040151-16 2024-09-16T14:49:20.986 INFO:tasks.rados.rados.0.smithi040.stdout:283: finishing write tid 4 to smithi04040151-16 2024-09-16T14:49:20.987 INFO:tasks.rados.rados.0.smithi040.stdout:283: finishing write tid 5 to smithi04040151-16 2024-09-16T14:49:20.987 INFO:tasks.rados.rados.0.smithi040.stdout:283: oid 16 updating version 0 to 35 2024-09-16T14:49:20.987 INFO:tasks.rados.rados.0.smithi040.stdout:283: oid 16 version 35 is already newer than 34 2024-09-16T14:49:20.987 INFO:tasks.rados.rados.0.smithi040.stdout:283: oid 16 version 35 is already newer than 33 2024-09-16T14:49:20.987 INFO:tasks.rados.rados.0.smithi040.stdout:283: oid 16 version 35 is already newer than 32 2024-09-16T14:49:20.987 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 16 v 35 (ObjNum 110 snap 24 seq_num 110) dirty exists 2024-09-16T14:49:20.987 INFO:tasks.rados.rados.0.smithi040.stdout:283: left oid 16 (ObjNum 110 snap 24 seq_num 110) 2024-09-16T14:49:21.021 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:21.022 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:21.030 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:49:21.055 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:21.053+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:21.158 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:21.156+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:21.172 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:21.172 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:21.181 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:49:21.265 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:21.264+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:21.324 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:21.324 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:21.333 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:49:21.374 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:21.373+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:21.446 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:21.446 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:21.454 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:49:21.462 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:21.461+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:21.557 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:21.558 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:21.563 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:21.562+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:21.569 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:49:21.647 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:49:21.671 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:21.671 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:21.680 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:49:21.692 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:21.691+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:21.765 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:21.763+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:21.776 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:21.777 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:21.793 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:21.793 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:21.801 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:21.801 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:21.801 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:21.801 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:21.809 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:21.809 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:21.811 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:49:21.825 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:21.826 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:21.843 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:21.843 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:21.859 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:21.865 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:21.864+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T14:49:21.884 INFO:tasks.rados.rados.0.smithi040.stdout:278: done (5 left) 2024-09-16T14:49:21.885 INFO:tasks.rados.rados.0.smithi040.stdout:280: done (4 left) 2024-09-16T14:49:21.885 INFO:tasks.rados.rados.0.smithi040.stdout:281: done (3 left) 2024-09-16T14:49:21.885 INFO:tasks.rados.rados.0.smithi040.stdout:282: done (2 left) 2024-09-16T14:49:21.885 INFO:tasks.rados.rados.0.smithi040.stdout:283: done (1 left) 2024-09-16T14:49:21.885 INFO:tasks.rados.rados.0.smithi040.stdout:284: done (0 left) 2024-09-16T14:49:21.885 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 35 current snap is 25 2024-09-16T14:49:21.885 INFO:tasks.rados.rados.0.smithi040.stdout:285: seq_num 111 ranges {2998272=704512,3702784=655360,4358144=507904} 2024-09-16T14:49:21.885 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:49:21.891 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:21.891 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:21.900 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T14:49:21.904 INFO:tasks.rados.rados.0.smithi040.stdout:285: writing smithi04040151-35 from 2998272 to 3702784 tid 1 2024-09-16T14:49:21.908 INFO:tasks.rados.rados.0.smithi040.stdout:285: writing smithi04040151-35 from 3702784 to 4358144 tid 2 2024-09-16T14:49:21.913 INFO:tasks.rados.rados.0.smithi040.stdout:285: writing smithi04040151-35 from 4358144 to 4866048 tid 3 2024-09-16T14:49:21.913 INFO:tasks.rados.rados.0.smithi040.stdout:286: copy_from oid 48 from oid 9 current snap is 25 2024-09-16T14:49:21.913 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 38 current snap is 25 2024-09-16T14:49:21.913 INFO:tasks.rados.rados.0.smithi040.stdout:287: seq_num 112 ranges {131072=720896,851968=802816,1654784=360448} 2024-09-16T14:49:21.915 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:21.915 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:21.918 INFO:tasks.rados.rados.0.smithi040.stdout:287: writing smithi04040151-38 from 131072 to 851968 tid 1 2024-09-16T14:49:21.924 INFO:tasks.rados.rados.0.smithi040.stdout:287: writing smithi04040151-38 from 851968 to 1654784 tid 2 2024-09-16T14:49:21.927 INFO:tasks.rados.rados.0.smithi040.stdout:287: writing smithi04040151-38 from 1654784 to 2015232 tid 3 2024-09-16T14:49:21.927 INFO:tasks.rados.rados.0.smithi040.stdout:285: finishing write tid 1 to smithi04040151-35 2024-09-16T14:49:21.927 INFO:tasks.rados.rados.0.smithi040.stdout:285: finishing write tid 2 to smithi04040151-35 2024-09-16T14:49:21.927 INFO:tasks.rados.rados.0.smithi040.stdout:285: finishing write tid 3 to smithi04040151-35 2024-09-16T14:49:21.927 INFO:tasks.rados.rados.0.smithi040.stdout:285: finishing write tid 4 to smithi04040151-35 2024-09-16T14:49:21.928 INFO:tasks.rados.rados.0.smithi040.stdout:285: finishing write tid 5 to smithi04040151-35 2024-09-16T14:49:21.928 INFO:tasks.rados.rados.0.smithi040.stdout:285: oid 35 updating version 0 to 50 2024-09-16T14:49:21.928 INFO:tasks.rados.rados.0.smithi040.stdout:285: oid 35 updating version 50 to 52 2024-09-16T14:49:21.928 INFO:tasks.rados.rados.0.smithi040.stdout:285: oid 35 version 52 is already newer than 51 2024-09-16T14:49:21.928 INFO:tasks.rados.rados.0.smithi040.stdout:285: oid 35 version 52 is already newer than 49 2024-09-16T14:49:21.928 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 35 v 52 (ObjNum 111 snap 25 seq_num 111) dirty exists 2024-09-16T14:49:21.928 INFO:tasks.rados.rados.0.smithi040.stdout:285: left oid 35 (ObjNum 111 snap 25 seq_num 111) 2024-09-16T14:49:21.928 INFO:tasks.rados.rados.0.smithi040.stdout:285: done (2 left) 2024-09-16T14:49:21.928 INFO:tasks.rados.rados.0.smithi040.stdout:288: snap_create 2024-09-16T14:49:21.931 INFO:tasks.rados.rados.0.smithi040.stdout:286: finishing copy_from racing read to smithi04040151-48 2024-09-16T14:49:21.932 INFO:tasks.rados.rados.0.smithi040.stdout:287: finishing write tid 1 to smithi04040151-38 2024-09-16T14:49:21.932 INFO:tasks.rados.rados.0.smithi040.stdout:287: finishing write tid 2 to smithi04040151-38 2024-09-16T14:49:21.933 INFO:tasks.rados.rados.0.smithi040.stdout:287: finishing write tid 3 to smithi04040151-38 2024-09-16T14:49:21.933 INFO:tasks.rados.rados.0.smithi040.stdout:287: finishing write tid 4 to smithi04040151-38 2024-09-16T14:49:21.936 INFO:tasks.rados.rados.0.smithi040.stdout:287: finishing write tid 5 to smithi04040151-38 2024-09-16T14:49:21.937 INFO:tasks.rados.rados.0.smithi040.stdout:287: oid 38 updating version 0 to 49 2024-09-16T14:49:21.937 INFO:tasks.rados.rados.0.smithi040.stdout:287: oid 38 updating version 49 to 51 2024-09-16T14:49:21.937 INFO:tasks.rados.rados.0.smithi040.stdout:287: oid 38 version 51 is already newer than 50 2024-09-16T14:49:21.937 INFO:tasks.rados.rados.0.smithi040.stdout:287: oid 38 updating version 51 to 52 2024-09-16T14:49:21.937 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 38 v 52 (ObjNum 112 snap 25 seq_num 112) dirty exists 2024-09-16T14:49:21.937 INFO:tasks.rados.rados.0.smithi040.stdout:287: left oid 38 (ObjNum 112 snap 25 seq_num 112) 2024-09-16T14:49:21.937 INFO:tasks.rados.rados.0.smithi040.stdout:286: finishing copy_from to smithi04040151-48 2024-09-16T14:49:21.937 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 48 v 4 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-16T14:49:21.943 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:21.944 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:21.945 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:21.945 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:21.945 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:21.945 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:21.945 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:21.954 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:49:21.960 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:21.961 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:21.967 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:21.965+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:21.979 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:21.979 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:21.996 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:21.996 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:22.067 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:22.066+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:22.091 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:22.091 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:22.091 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:22.091 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:22.091 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.091 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "pg_update_log_missing(3.5s3 epoch 51/27 rep_tid 54 entries 51'14 (0'0) error 3:a7111aa9:::smithi04040151-45:head by client.4500.0:449 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 37'11)", 2024-09-16T14:49:22.091 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.879285+0000", 2024-09-16T14:49:22.091 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.209384606, 2024-09-16T14:49:22.091 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0096111870000000002, 2024-09-16T14:49:22.091 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.091 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.092 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.092 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.092 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879285+0000", 2024-09-16T14:49:22.092 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.092 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.092 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.092 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879285+0000", 2024-09-16T14:49:22.092 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.092 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.092 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.093 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879285+0000", 2024-09-16T14:49:22.093 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.093 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.093 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879291+0000", 2024-09-16T14:49:22.093 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.3450000000000002e-06 2024-09-16T14:49:22.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.093 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.093 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879311+0000", 2024-09-16T14:49:22.093 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9602999999999998e-05 2024-09-16T14:49:22.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.094 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.094 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879324+0000", 2024-09-16T14:49:22.094 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3127e-05 2024-09-16T14:49:22.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.094 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.094 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.879406+0000", 2024-09-16T14:49:22.094 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1942999999999998e-05 2024-09-16T14:49:22.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.094 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.094 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.888896+0000", 2024-09-16T14:49:22.094 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0094904359999999997 2024-09-16T14:49:22.094 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.095 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.095 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.095 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.095 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:454 3.6s0 3:608d2d89:::smithi04040151-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:22.095 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:22.095 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.159236208999999, 2024-09-16T14:49:22.095 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013776338000000001, 2024-09-16T14:49:22.095 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.095 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:22.095 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:22.095 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:22.095 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:22.096 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 454 2024-09-16T14:49:22.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.096 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.096 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.096 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:22.096 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.096 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.096 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:22.096 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.097 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.097 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.097 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:22.097 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.097 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.097 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.097 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.097 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929436+0000", 2024-09-16T14:49:22.097 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7939999999999998e-06 2024-09-16T14:49:22.097 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.097 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.097 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.097 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929444+0000", 2024-09-16T14:49:22.098 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4749999999999993e-06 2024-09-16T14:49:22.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.098 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.098 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.098 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929449+0000", 2024-09-16T14:49:22.098 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2750000000000003e-06 2024-09-16T14:49:22.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.098 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.098 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.098 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.933117+0000", 2024-09-16T14:49:22.098 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0036674250000000002 2024-09-16T14:49:22.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.098 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.098 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:22.099 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.933207+0000", 2024-09-16T14:49:22.099 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0358000000000005e-05 2024-09-16T14:49:22.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.099 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.099 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.937436+0000", 2024-09-16T14:49:22.099 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0042287089999999998 2024-09-16T14:49:22.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.099 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:22.099 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.937535+0000", 2024-09-16T14:49:22.099 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9501999999999997e-05 2024-09-16T14:49:22.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.100 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.100 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.943210+0000", 2024-09-16T14:49:22.100 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056742429999999998 2024-09-16T14:49:22.100 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.100 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.100 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.100 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 52/27 ECSubWrite(tid=182, reqid=osd.11.0:182, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:22.100 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:22.100 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.201424462999999, 2024-09-16T14:49:22.100 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014202568, 2024-09-16T14:49:22.100 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.100 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.101 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.101 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.101 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:22.101 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.101 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.101 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:22.101 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.101 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.101 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887302+0000", 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.7673999999999997e-05 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887313+0000", 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0712e-05 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.102 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887321+0000", 2024-09-16T14:49:22.103 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1329999999999999e-06 2024-09-16T14:49:22.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.103 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.103 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887397+0000", 2024-09-16T14:49:22.103 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5564999999999999e-05 2024-09-16T14:49:22.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.103 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.103 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887407+0000", 2024-09-16T14:49:22.103 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9690000000000006e-06 2024-09-16T14:49:22.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.103 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.104 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901422+0000", 2024-09-16T14:49:22.104 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01401468 2024-09-16T14:49:22.104 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.104 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.104 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.104 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901448+0000", 2024-09-16T14:49:22.104 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6191999999999999e-05 2024-09-16T14:49:22.104 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.104 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.104 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.104 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.104 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.104 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 52/27 ECSubWrite(tid=214, reqid=osd.7.0:214, at_version=52'35, trim_to=0'0, roll_forward_to=51'31))", 2024-09-16T14:49:22.104 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:22.105 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.201381263999998, 2024-09-16T14:49:22.105 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014116002000000001, 2024-09-16T14:49:22.105 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.105 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.105 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.105 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.105 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:22.105 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.105 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.105 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:22.105 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.106 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.106 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:22.106 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.106 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.106 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887318+0000", 2024-09-16T14:49:22.106 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0477999999999999e-05 2024-09-16T14:49:22.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.106 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.106 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887329+0000", 2024-09-16T14:49:22.107 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0845999999999999e-05 2024-09-16T14:49:22.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.107 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.107 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887337+0000", 2024-09-16T14:49:22.107 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7880000000000007e-06 2024-09-16T14:49:22.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.107 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.107 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887631+0000", 2024-09-16T14:49:22.107 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029412399999999999 2024-09-16T14:49:22.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.107 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.108 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887640+0000", 2024-09-16T14:49:22.108 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0180000000000007e-06 2024-09-16T14:49:22.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.108 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.108 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901368+0000", 2024-09-16T14:49:22.108 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013728346000000001 2024-09-16T14:49:22.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.108 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.108 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901404+0000", 2024-09-16T14:49:22.108 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5964000000000003e-05 2024-09-16T14:49:22.108 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.108 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 52/27 ECSubWrite(tid=51, reqid=osd.1.0:50, at_version=0'0, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T14:49:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.199508296000001, 2024-09-16T14:49:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013036324, 2024-09-16T14:49:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:22.109 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.110 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.110 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:22.110 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.110 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.110 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889160+0000", 2024-09-16T14:49:22.110 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:22.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.110 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.110 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889174+0000", 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3416e-05 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889186+0000", 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2369e-05 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889195+0000", 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6540000000000005e-06 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.111 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889243+0000", 2024-09-16T14:49:22.112 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.8455000000000003e-05 2024-09-16T14:49:22.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.112 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.112 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889253+0000", 2024-09-16T14:49:22.112 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3700000000000001e-06 2024-09-16T14:49:22.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.112 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.112 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902168+0000", 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012915311 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902198+0000", 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9641e-05 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s2 52/27 ECSubWrite(tid=29, reqid=osd.4.0:29, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 23.198705121, 2024-09-16T14:49:22.113 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012371216000000001, 2024-09-16T14:49:22.114 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.114 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.114 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.114 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.114 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:22.114 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.114 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.114 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:22.114 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.115 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.115 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:22.115 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.115 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.115 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889990+0000", 2024-09-16T14:49:22.115 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5548999999999999e-05 2024-09-16T14:49:22.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.115 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.115 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890001+0000", 2024-09-16T14:49:22.115 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.098e-05 2024-09-16T14:49:22.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.116 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.116 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890009+0000", 2024-09-16T14:49:22.116 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5930000000000006e-06 2024-09-16T14:49:22.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.116 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.116 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890069+0000", 2024-09-16T14:49:22.116 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.9855999999999997e-05 2024-09-16T14:49:22.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.116 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.116 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890082+0000", 2024-09-16T14:49:22.116 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3312999999999999e-05 2024-09-16T14:49:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902310+0000", 2024-09-16T14:49:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012227780000000001 2024-09-16T14:49:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902336+0000", 2024-09-16T14:49:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5652000000000001e-05 2024-09-16T14:49:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s2 61/27 ECSubWrite(tid=35, reqid=osd.4.0:34, at_version=0'0, trim_to=0'0, roll_forward_to=61'22))", 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.159259619, 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012712925, 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.118 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.119 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:22.119 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.119 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.119 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929409+0000", 2024-09-16T14:49:22.119 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:22.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.119 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.119 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929422+0000", 2024-09-16T14:49:22.119 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3497e-05 2024-09-16T14:49:22.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.120 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.120 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929434+0000", 2024-09-16T14:49:22.120 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.205e-05 2024-09-16T14:49:22.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.120 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.120 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929443+0000", 2024-09-16T14:49:22.120 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8419999999999994e-06 2024-09-16T14:49:22.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.120 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.120 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929490+0000", 2024-09-16T14:49:22.120 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6825e-05 2024-09-16T14:49:22.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.121 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.121 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929500+0000", 2024-09-16T14:49:22.121 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8810000000000008e-06 2024-09-16T14:49:22.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.121 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.121 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942092+0000", 2024-09-16T14:49:22.121 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012591955 2024-09-16T14:49:22.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.121 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.121 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942123+0000", 2024-09-16T14:49:22.122 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1015e-05 2024-09-16T14:49:22.122 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.122 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.122 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.122 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs6 61/27 ECSubWrite(tid=235, reqid=osd.7.0:235, at_version=61'21, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:22.123 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:22.123 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.157832708000001, 2024-09-16T14:49:22.123 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011696734, 2024-09-16T14:49:22.123 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.123 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.123 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.124 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.124 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:22.124 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.124 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.124 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:22.124 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.124 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.124 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:22.124 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:22.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.125 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.125 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930866+0000", 2024-09-16T14:49:22.125 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9138000000000001e-05 2024-09-16T14:49:22.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.125 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.125 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930874+0000", 2024-09-16T14:49:22.125 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5890000000000003e-06 2024-09-16T14:49:22.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.125 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.125 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930882+0000", 2024-09-16T14:49:22.126 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.25e-06 2024-09-16T14:49:22.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.126 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.126 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930933+0000", 2024-09-16T14:49:22.126 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1536999999999998e-05 2024-09-16T14:49:22.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.126 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.126 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930941+0000", 2024-09-16T14:49:22.126 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7510000000000008e-06 2024-09-16T14:49:22.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.127 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.127 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942507+0000", 2024-09-16T14:49:22.127 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011565765 2024-09-16T14:49:22.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.127 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.127 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.127 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942534+0000", 2024-09-16T14:49:22.127 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6900999999999999e-05 2024-09-16T14:49:22.127 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.128 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.128 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.128 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 61/27 ECSubWrite(tid=63, reqid=osd.1.0:63, at_version=61'50, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:22.128 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:22.128 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.156296272000001, 2024-09-16T14:49:22.128 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014509462000000001, 2024-09-16T14:49:22.128 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.128 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.128 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.128 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.129 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:22.129 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.129 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.129 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:22.129 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.129 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.129 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:22.129 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.130 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.130 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932395+0000", 2024-09-16T14:49:22.130 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2039999999999999e-05 2024-09-16T14:49:22.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.130 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.130 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932404+0000", 2024-09-16T14:49:22.130 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4879999999999996e-06 2024-09-16T14:49:22.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.131 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.131 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932412+0000", 2024-09-16T14:49:22.131 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.61e-06 2024-09-16T14:49:22.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.131 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.131 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932577+0000", 2024-09-16T14:49:22.132 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00016469799999999999 2024-09-16T14:49:22.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.132 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.132 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932605+0000", 2024-09-16T14:49:22.132 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7963e-05 2024-09-16T14:49:22.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.132 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.132 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946863+0000", 2024-09-16T14:49:22.132 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014258600999999999 2024-09-16T14:49:22.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.133 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.133 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946883+0000", 2024-09-16T14:49:22.133 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9494e-05 2024-09-16T14:49:22.133 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.133 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.133 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.133 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs6 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:22.133 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:22.133 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.156212867000001, 2024-09-16T14:49:22.133 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014400098, 2024-09-16T14:49:22.133 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.134 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.134 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.134 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.134 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:22.134 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.134 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.134 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:22.134 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.134 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.135 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932456+0000", 2024-09-16T14:49:22.135 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.135 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.135 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932475+0000", 2024-09-16T14:49:22.135 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8695999999999999e-05 2024-09-16T14:49:22.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.135 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.135 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932485+0000", 2024-09-16T14:49:22.135 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7799999999999995e-06 2024-09-16T14:49:22.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.136 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.136 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932492+0000", 2024-09-16T14:49:22.136 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5209999999999997e-06 2024-09-16T14:49:22.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.136 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.136 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932705+0000", 2024-09-16T14:49:22.136 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00021277700000000001 2024-09-16T14:49:22.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.136 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.136 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932721+0000", 2024-09-16T14:49:22.136 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6095000000000001e-05 2024-09-16T14:49:22.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.137 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.137 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946825+0000", 2024-09-16T14:49:22.137 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014103269999999999 2024-09-16T14:49:22.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.137 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.137 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946857+0000", 2024-09-16T14:49:22.137 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2271999999999999e-05 2024-09-16T14:49:22.137 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.138 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.138 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.139 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:22.139 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:22.139 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.156016018000001, 2024-09-16T14:49:22.139 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015300085999999999, 2024-09-16T14:49:22.139 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.140 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.140 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.140 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.140 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:22.140 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.140 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.141 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:22.141 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.141 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.141 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932653+0000", 2024-09-16T14:49:22.141 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.141 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.142 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932674+0000", 2024-09-16T14:49:22.142 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1047e-05 2024-09-16T14:49:22.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.142 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.142 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932685+0000", 2024-09-16T14:49:22.142 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0733999999999999e-05 2024-09-16T14:49:22.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.142 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.142 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932693+0000", 2024-09-16T14:49:22.142 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0819999999999999e-06 2024-09-16T14:49:22.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.143 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.143 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945297+0000", 2024-09-16T14:49:22.143 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012604198 2024-09-16T14:49:22.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.143 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.143 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945306+0000", 2024-09-16T14:49:22.143 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1549999999999996e-06 2024-09-16T14:49:22.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.144 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.144 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947927+0000", 2024-09-16T14:49:22.144 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.002621295 2024-09-16T14:49:22.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.144 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.144 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947954+0000", 2024-09-16T14:49:22.145 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6219000000000001e-05 2024-09-16T14:49:22.145 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.145 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.145 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.145 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s3 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:22.146 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:22.146 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.15592829, 2024-09-16T14:49:22.146 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013712046, 2024-09-16T14:49:22.146 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.146 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.146 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.146 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.146 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:22.146 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.147 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.147 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:22.147 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.147 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.147 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:22.147 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.147 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.148 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932760+0000", 2024-09-16T14:49:22.148 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9247999999999999e-05 2024-09-16T14:49:22.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.148 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.148 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932773+0000", 2024-09-16T14:49:22.148 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3081000000000001e-05 2024-09-16T14:49:22.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.148 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932783+0000", 2024-09-16T14:49:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7850000000000003e-06 2024-09-16T14:49:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932830+0000", 2024-09-16T14:49:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7608999999999998e-05 2024-09-16T14:49:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932842+0000", 2024-09-16T14:49:22.149 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1082e-05 2024-09-16T14:49:22.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.150 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.150 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946430+0000", 2024-09-16T14:49:22.150 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013588141999999999 2024-09-16T14:49:22.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.150 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.150 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946453+0000", 2024-09-16T14:49:22.151 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3714999999999999e-05 2024-09-16T14:49:22.151 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.151 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.151 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.151 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:22.151 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:22.151 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.155513347999999, 2024-09-16T14:49:22.151 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013268299000000001, 2024-09-16T14:49:22.151 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.152 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.152 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.152 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.152 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:22.152 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.152 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.152 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:22.153 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.153 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.153 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933155+0000", 2024-09-16T14:49:22.153 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:22.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.153 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.153 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933182+0000", 2024-09-16T14:49:22.153 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7090999999999999e-05 2024-09-16T14:49:22.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.154 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.154 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933202+0000", 2024-09-16T14:49:22.154 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9459000000000001e-05 2024-09-16T14:49:22.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.154 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.154 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933211+0000", 2024-09-16T14:49:22.154 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.397e-06 2024-09-16T14:49:22.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.155 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.155 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933284+0000", 2024-09-16T14:49:22.155 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2711000000000002e-05 2024-09-16T14:49:22.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.155 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.155 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933294+0000", 2024-09-16T14:49:22.155 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0079e-05 2024-09-16T14:49:22.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.155 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.156 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946408+0000", 2024-09-16T14:49:22.156 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013114097999999999 2024-09-16T14:49:22.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.156 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.156 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.156 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946425+0000", 2024-09-16T14:49:22.156 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6568000000000001e-05 2024-09-16T14:49:22.156 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.156 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.156 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.157 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs6 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:22.157 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:22.157 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.155296447, 2024-09-16T14:49:22.157 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014598939999999999, 2024-09-16T14:49:22.157 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.157 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.157 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.158 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.158 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:22.158 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.158 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.158 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:22.158 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.158 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.159 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933372+0000", 2024-09-16T14:49:22.159 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:22.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.159 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.159 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.159 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933398+0000", 2024-09-16T14:49:22.159 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6129999999999999e-05 2024-09-16T14:49:22.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.159 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.159 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.159 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933414+0000", 2024-09-16T14:49:22.159 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5549999999999999e-05 2024-09-16T14:49:22.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.160 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.160 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.160 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933422+0000", 2024-09-16T14:49:22.160 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8620000000000003e-06 2024-09-16T14:49:22.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.160 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.160 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.160 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945189+0000", 2024-09-16T14:49:22.160 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011767552000000001 2024-09-16T14:49:22.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.161 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.161 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945200+0000", 2024-09-16T14:49:22.161 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0766e-05 2024-09-16T14:49:22.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.161 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.161 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947957+0000", 2024-09-16T14:49:22.161 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0027565319999999999 2024-09-16T14:49:22.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.162 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.162 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.162 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947972+0000", 2024-09-16T14:49:22.162 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5359999999999999e-05 2024-09-16T14:49:22.162 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.162 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.162 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.162 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.162 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs6 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:22.162 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:22.162 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.154290181, 2024-09-16T14:49:22.163 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010566212, 2024-09-16T14:49:22.163 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.163 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.163 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.163 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.163 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:22.163 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.164 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.164 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:22.164 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.164 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.164 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:22.164 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:22.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.165 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.165 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.165 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934728+0000", 2024-09-16T14:49:22.165 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034932000000000002 2024-09-16T14:49:22.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.165 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.165 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.165 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934739+0000", 2024-09-16T14:49:22.165 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1318000000000001e-05 2024-09-16T14:49:22.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.165 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.166 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.166 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934752+0000", 2024-09-16T14:49:22.166 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2901999999999999e-05 2024-09-16T14:49:22.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.166 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.166 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.166 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934823+0000", 2024-09-16T14:49:22.167 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1118000000000006e-05 2024-09-16T14:49:22.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.167 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.167 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934834+0000", 2024-09-16T14:49:22.167 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0591000000000001e-05 2024-09-16T14:49:22.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.168 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.168 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.944917+0000", 2024-09-16T14:49:22.168 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010082906000000001 2024-09-16T14:49:22.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.169 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.169 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.169 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.944946+0000", 2024-09-16T14:49:22.169 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8895999999999999e-05 2024-09-16T14:49:22.169 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.169 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.169 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.170 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 61/27 ECSubWrite(tid=237, reqid=osd.7.0:237, at_version=61'50, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:22.170 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:22.170 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.153090062, 2024-09-16T14:49:22.170 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011297481, 2024-09-16T14:49:22.171 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.171 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.171 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.171 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.171 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.171 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:22.171 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.172 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:22.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.173 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.173 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935579+0000", 2024-09-16T14:49:22.173 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:22.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.173 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.174 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935608+0000", 2024-09-16T14:49:22.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9578999999999999e-05 2024-09-16T14:49:22.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.174 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.174 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.174 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935625+0000", 2024-09-16T14:49:22.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7075000000000002e-05 2024-09-16T14:49:22.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.175 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.175 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935642+0000", 2024-09-16T14:49:22.175 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6480000000000001e-05 2024-09-16T14:49:22.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.175 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.176 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935874+0000", 2024-09-16T14:49:22.176 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00023162399999999999 2024-09-16T14:49:22.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.176 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.176 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.176 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935887+0000", 2024-09-16T14:49:22.176 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3811e-05 2024-09-16T14:49:22.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.177 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.177 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946860+0000", 2024-09-16T14:49:22.177 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010972552999999999 2024-09-16T14:49:22.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.178 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.178 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.178 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946877+0000", 2024-09-16T14:49:22.178 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7132000000000001e-05 2024-09-16T14:49:22.178 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.178 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.178 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.179 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 61/27 ECSubWrite(tid=231, reqid=osd.11.0:231, at_version=61'37, trim_to=0'0, roll_forward_to=60'33))", 2024-09-16T14:49:22.179 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:22.179 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.134841518, 2024-09-16T14:49:22.179 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012494261, 2024-09-16T14:49:22.179 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.180 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.180 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.180 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.180 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.180 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:22.180 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.181 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.181 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:22.181 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.182 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.182 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:22.182 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.182 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.182 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953840+0000", 2024-09-16T14:49:22.182 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1983e-05 2024-09-16T14:49:22.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.182 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.183 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953846+0000", 2024-09-16T14:49:22.183 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.319e-06 2024-09-16T14:49:22.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.183 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.183 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.183 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953852+0000", 2024-09-16T14:49:22.183 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.823e-06 2024-09-16T14:49:22.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.184 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.184 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954193+0000", 2024-09-16T14:49:22.184 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034066199999999999 2024-09-16T14:49:22.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.185 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.185 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954203+0000", 2024-09-16T14:49:22.185 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0421999999999999e-05 2024-09-16T14:49:22.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.185 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.185 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966300+0000", 2024-09-16T14:49:22.185 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012096519 2024-09-16T14:49:22.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.186 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.186 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966322+0000", 2024-09-16T14:49:22.186 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2829000000000001e-05 2024-09-16T14:49:22.186 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.186 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 61/27 ECSubWrite(tid=244, reqid=osd.3.0:243, at_version=0'0, trim_to=0'0, roll_forward_to=61'81))", 2024-09-16T14:49:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 14.134391248, 2024-09-16T14:49:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012038332000000001, 2024-09-16T14:49:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.188 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.188 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:22.188 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.188 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.188 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:22.188 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.189 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.189 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954277+0000", 2024-09-16T14:49:22.189 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:22.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.189 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.190 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954290+0000", 2024-09-16T14:49:22.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2364000000000001e-05 2024-09-16T14:49:22.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.190 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.190 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954302+0000", 2024-09-16T14:49:22.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2665e-05 2024-09-16T14:49:22.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.190 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.190 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954322+0000", 2024-09-16T14:49:22.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9429e-05 2024-09-16T14:49:22.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.191 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.191 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954374+0000", 2024-09-16T14:49:22.191 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2182000000000003e-05 2024-09-16T14:49:22.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.191 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.191 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954384+0000", 2024-09-16T14:49:22.191 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0022e-05 2024-09-16T14:49:22.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.192 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.192 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966295+0000", 2024-09-16T14:49:22.192 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011910399 2024-09-16T14:49:22.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.192 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.192 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966317+0000", 2024-09-16T14:49:22.193 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2169999999999999e-05 2024-09-16T14:49:22.193 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.193 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.193 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.193 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:688 3.6s0 3:608d2d89:::smithi04040151-48:head [copy-from ver 4] snapc 2e=[2e,2b,29,28,25] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T14:49:22.193 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:21.913077+0000", 2024-09-16T14:49:22.193 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.175592636, 2024-09-16T14:49:22.193 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.023450634000000001, 2024-09-16T14:49:22.194 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.194 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:22.194 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:22.194 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:22.194 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:22.194 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 688 2024-09-16T14:49:22.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.194 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.194 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.194 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913077+0000", 2024-09-16T14:49:22.195 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.195 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.195 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913077+0000", 2024-09-16T14:49:22.195 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.195 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.196 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913075+0000", 2024-09-16T14:49:22.196 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:22.196 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.196 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.196 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913085+0000", 2024-09-16T14:49:22.196 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8810000000000008e-06 2024-09-16T14:49:22.196 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.197 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.197 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.197 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913099+0000", 2024-09-16T14:49:22.197 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.382e-05 2024-09-16T14:49:22.197 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.197 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.198 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.198 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913108+0000", 2024-09-16T14:49:22.198 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7390000000000005e-06 2024-09-16T14:49:22.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.198 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.198 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.199 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913174+0000", 2024-09-16T14:49:22.199 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6598000000000002e-05 2024-09-16T14:49:22.199 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.199 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.199 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:22.200 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913288+0000", 2024-09-16T14:49:22.200 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011353000000000001 2024-09-16T14:49:22.200 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.200 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.200 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.200 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.929027+0000", 2024-09-16T14:49:22.201 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015739758999999999 2024-09-16T14:49:22.201 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.201 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.201 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.201 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.931900+0000", 2024-09-16T14:49:22.202 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0028725310000000002 2024-09-16T14:49:22.202 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.202 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.202 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:22.202 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.936419+0000", 2024-09-16T14:49:22.203 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0045194340000000001 2024-09-16T14:49:22.203 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.203 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.203 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.203 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.936528+0000", 2024-09-16T14:49:22.203 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010822199999999999 2024-09-16T14:49:22.204 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.204 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.204 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.204 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.204 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.205 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:689 3.6s0 3:608d2d89:::smithi04040151-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-09-16T14:49:22.205 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:22.205 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.17554450399999999, 2024-09-16T14:49:22.205 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.018229170999999999, 2024-09-16T14:49:22.205 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.206 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:22.206 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:22.206 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:22.206 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:22.206 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 689 2024-09-16T14:49:22.206 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.207 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.207 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.207 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.207 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:22.207 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.208 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.208 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.208 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.208 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:22.208 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.209 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.209 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.209 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.209 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:22.209 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.210 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.210 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.210 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.210 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913128+0000", 2024-09-16T14:49:22.210 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.4580000000000002e-06 2024-09-16T14:49:22.210 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.211 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.211 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.211 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913137+0000", 2024-09-16T14:49:22.211 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6039999999999993e-06 2024-09-16T14:49:22.211 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.212 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.212 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.212 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913145+0000", 2024-09-16T14:49:22.212 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6200000000000005e-06 2024-09-16T14:49:22.212 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.213 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.213 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.213 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913491+0000", 2024-09-16T14:49:22.213 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034584200000000001 2024-09-16T14:49:22.213 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.214 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.214 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:22.214 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913536+0000", 2024-09-16T14:49:22.214 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.5269000000000003e-05 2024-09-16T14:49:22.214 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.214 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.215 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.215 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.930844+0000", 2024-09-16T14:49:22.215 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.017307802000000001 2024-09-16T14:49:22.215 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.215 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.216 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:22.216 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.930954+0000", 2024-09-16T14:49:22.216 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011033500000000001 2024-09-16T14:49:22.216 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.216 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.217 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.217 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.931354+0000", 2024-09-16T14:49:22.217 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00039983500000000002 2024-09-16T14:49:22.217 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.217 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.217 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.218 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.218 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.218 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:22.220 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:22.218+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:22.221 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:49:22.320 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:22.319+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:22.328 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:22.328 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:22.328 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:22.328 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:22.337 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:49:22.422 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:22.421+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:22.443 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:22.444 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:22.444 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:22.444 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:22.444 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:22.452 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:49:22.523 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:22.522+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:22.601 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:22.601 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:22.601 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:22.601 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:22.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.601 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.2s5 37/27 ECSubWrite(tid=123, reqid=osd.3.0:123, at_version=37'38, trim_to=0'0, roll_forward_to=37'34))", 2024-09-16T14:49:22.601 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:22.601 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 38.662516897000003, 2024-09-16T14:49:22.601 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0094962529999999996, 2024-09-16T14:49:22.601 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.601 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.602 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.602 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.602 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:22.602 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.602 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.602 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:22.602 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.602 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.603 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935902+0000", 2024-09-16T14:49:22.603 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:22.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.603 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.603 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935938+0000", 2024-09-16T14:49:22.603 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5731000000000001e-05 2024-09-16T14:49:22.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.603 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.603 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935949+0000", 2024-09-16T14:49:22.603 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1229999999999999e-05 2024-09-16T14:49:22.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.604 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.604 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935957+0000", 2024-09-16T14:49:22.604 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.473e-06 2024-09-16T14:49:22.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.604 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.604 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.936913+0000", 2024-09-16T14:49:22.604 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00095599900000000002 2024-09-16T14:49:22.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.604 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.605 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.936923+0000", 2024-09-16T14:49:22.605 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4059999999999994e-06 2024-09-16T14:49:22.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.605 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.605 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945371+0000", 2024-09-16T14:49:22.605 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084484279999999991 2024-09-16T14:49:22.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.605 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.605 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945399+0000", 2024-09-16T14:49:22.605 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7716e-05 2024-09-16T14:49:22.605 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.605 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.606 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.606 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s1 37/27 ECSubWrite(tid=108, reqid=osd.11.0:108, at_version=37'33, trim_to=0'0, roll_forward_to=35'31))", 2024-09-16T14:49:22.606 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.940384+0000", 2024-09-16T14:49:22.606 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 38.658036053000004, 2024-09-16T14:49:22.606 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085853549999999994, 2024-09-16T14:49:22.606 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.606 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.606 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.606 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.606 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940384+0000", 2024-09-16T14:49:22.606 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.607 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.607 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940384+0000", 2024-09-16T14:49:22.607 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.607 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.607 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940383+0000", 2024-09-16T14:49:22.607 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.607 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.608 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940406+0000", 2024-09-16T14:49:22.608 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3003999999999999e-05 2024-09-16T14:49:22.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.608 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.608 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940421+0000", 2024-09-16T14:49:22.608 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5285e-05 2024-09-16T14:49:22.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.608 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.608 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940430+0000", 2024-09-16T14:49:22.608 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.439e-06 2024-09-16T14:49:22.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.609 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.609 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940486+0000", 2024-09-16T14:49:22.609 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6266e-05 2024-09-16T14:49:22.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.609 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.609 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940497+0000", 2024-09-16T14:49:22.609 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1231e-05 2024-09-16T14:49:22.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.609 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.609 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.948934+0000", 2024-09-16T14:49:22.609 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084368450000000001 2024-09-16T14:49:22.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.610 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.610 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.948969+0000", 2024-09-16T14:49:22.610 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.4993999999999997e-05 2024-09-16T14:49:22.610 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.610 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.610 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.610 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.0s3 37/27 ECSubWrite(tid=109, reqid=osd.11.0:109, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:22.610 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:22.610 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 38.657970276999997, 2024-09-16T14:49:22.610 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090738399999999997, 2024-09-16T14:49:22.611 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.611 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.611 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.611 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.611 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:22.611 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.611 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.611 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:22.611 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.612 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.612 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:22.612 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.612 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.612 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940467+0000", 2024-09-16T14:49:22.612 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8113000000000001e-05 2024-09-16T14:49:22.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.612 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.612 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940475+0000", 2024-09-16T14:49:22.612 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8499999999999994e-06 2024-09-16T14:49:22.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.613 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.613 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940481+0000", 2024-09-16T14:49:22.613 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0979999999999999e-06 2024-09-16T14:49:22.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.613 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.613 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940569+0000", 2024-09-16T14:49:22.613 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8395999999999999e-05 2024-09-16T14:49:22.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.613 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.613 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940577+0000", 2024-09-16T14:49:22.614 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7500000000000003e-06 2024-09-16T14:49:22.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.614 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.614 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949500+0000", 2024-09-16T14:49:22.614 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089230319999999991 2024-09-16T14:49:22.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.614 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.614 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949523+0000", 2024-09-16T14:49:22.614 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3082e-05 2024-09-16T14:49:22.614 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.614 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.614 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.615 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:302 3.4s0 3:255172f3:::smithi04040151-3:head [create,setxattr qPd\\P_iff[nbYodrbXe[dQ_okmVWqe[RceaqQRNm (630) in=40b,setxattr NWYlZi[dPXnppNo]`fatPqfSaTlsQVetOmrQm^ap (138) in=40b,setxattr iVfcaeRT]TQppeYbq_`lRobpUsaVcWXphplasbgY (1507) in=40b,setxattr YeaeOf[jt`g\\k^]VfXTjOONb`jT[WYUgdiglVjYl (8841) in=40b,setxattr [iqZhSWl_sclXkajVYTXlOX]eeeWkPi\\dOYYt_Wb (1928) in=40b,setxattr \\[gaSPWYO`NQXpTccfbk`SZPaU`lbpTk]a] (1262) in=35b,setxattr ]\\]U^niYqQmjcgO[dsO`YpRZeRV]`X]bU[X^_Qis (1086) in=40b,setxattr VafomYY]aa`a^pbWebSRaithSn_dQjQYq]aTarm\\ (1402) in=40b,setxattr acj^VZhfaaikXrXq]_haYqkl[^_atWUTaloiVj`a (105) in=40b,setxattr VNUlaeZOglfodan^Ycb`dqVeZhSTmO_rabViYVch (70) in=40b,setxattr Qka\\PoUoghRc\\Qcnl`kVbj[rs\\hSdVXmlNqha`ta (594) in=40b,setxattr qsSQhb_apjdTlikdpj\\maaVdS[mVfsh_Vqi[ogWo (771) in=40b,setxattr Tm]oi_Rf_V`pSY[VhYSXSQN^bqjtgmm^fUo^f\\ZV (1087) in=40b,setxattr \\dTq (1262) in=4b,setxattr nettWfThbXj]cUpsYrTjZ^qk^\\[ZoZ[qb]Sf[ZbN (1684) in=40b,setxattr Sa\\hrdN^YombnT\\torZ`N]aRqicS_lP^XgP`e`Oc (1963) in=40b,setxattr dWhoTgYPSTh]jfWhPTs^iVdYbtfhi_WjOdes]NXh (1051) in=40b,setxattr Qbp_QrhXQf`O]blcqeqgRchO]RrhbshoV\\cW[_es (10033) in=40b,setxattr WYbQqgdRjmdbeoaP]U]n]PTWOYkP^lWOanNTnaOr (1998) in=40b,setxattr Rfbdg`[^ZYrN]cl][Ypr^tpXmb_mbh[gf\\idocW] (10804) in=40b] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:22.615 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:22.615 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 37.6660045, 2024-09-16T14:49:22.615 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.009468509, 2024-09-16T14:49:22.615 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.615 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:22.615 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:22.615 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:22.615 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:22.615 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 302 2024-09-16T14:49:22.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.616 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.616 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.616 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:22.616 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.616 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.616 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:22.616 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.616 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.617 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932413+0000", 2024-09-16T14:49:22.617 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:22.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.617 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.617 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932459+0000", 2024-09-16T14:49:22.617 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6414e-05 2024-09-16T14:49:22.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.617 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.617 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932491+0000", 2024-09-16T14:49:22.617 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1656000000000003e-05 2024-09-16T14:49:22.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.618 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.618 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932499+0000", 2024-09-16T14:49:22.618 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4940000000000003e-06 2024-09-16T14:49:22.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.618 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.618 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932557+0000", 2024-09-16T14:49:22.618 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8044000000000002e-05 2024-09-16T14:49:22.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.618 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:22.618 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932704+0000", 2024-09-16T14:49:22.618 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00014615 2024-09-16T14:49:22.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.619 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.619 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.934021+0000", 2024-09-16T14:49:22.619 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001317052 2024-09-16T14:49:22.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.619 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.619 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941512+0000", 2024-09-16T14:49:22.619 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074917789999999996 2024-09-16T14:49:22.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.619 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:22.620 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941804+0000", 2024-09-16T14:49:22.620 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029206099999999998 2024-09-16T14:49:22.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.620 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.620 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941884+0000", 2024-09-16T14:49:22.620 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9231000000000001e-05 2024-09-16T14:49:22.620 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.620 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.620 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.620 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es2 38/27 ECSubWrite(tid=113, reqid=client.4500.0:298, at_version=38'12, trim_to=0'0, roll_forward_to=37'11))", 2024-09-16T14:49:22.621 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.933568+0000", 2024-09-16T14:49:22.621 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 37.664851976999998, 2024-09-16T14:49:22.621 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089983270000000004, 2024-09-16T14:49:22.621 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.621 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.621 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.621 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.621 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933568+0000", 2024-09-16T14:49:22.621 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.621 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.622 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933568+0000", 2024-09-16T14:49:22.622 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.622 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.622 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933567+0000", 2024-09-16T14:49:22.622 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.622 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.622 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933833+0000", 2024-09-16T14:49:22.622 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00026602700000000001 2024-09-16T14:49:22.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.623 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.623 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933846+0000", 2024-09-16T14:49:22.623 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.223e-05 2024-09-16T14:49:22.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.623 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.623 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933854+0000", 2024-09-16T14:49:22.623 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5369999999999997e-06 2024-09-16T14:49:22.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.623 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.624 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933912+0000", 2024-09-16T14:49:22.624 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8103999999999998e-05 2024-09-16T14:49:22.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.624 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.624 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.933922+0000", 2024-09-16T14:49:22.624 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.978e-06 2024-09-16T14:49:22.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.624 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.624 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.942550+0000", 2024-09-16T14:49:22.624 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0086275210000000008 2024-09-16T14:49:22.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.625 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.625 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.942566+0000", 2024-09-16T14:49:22.625 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6268000000000001e-05 2024-09-16T14:49:22.625 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.625 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.625 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.625 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:365 3.4s0 3:2b1464a3:::smithi04040151-28:head [copy-from ver 15] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:22.625 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:22.625 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.659151102999999, 2024-09-16T14:49:22.626 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.019409261000000001, 2024-09-16T14:49:22.626 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.626 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:22.626 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:22.626 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:22.626 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:22.626 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 365 2024-09-16T14:49:22.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.626 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.626 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.626 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:22.626 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939267+0000", 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939278+0000", 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.093e-05 2024-09-16T14:49:22.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.628 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.628 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.628 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939295+0000", 2024-09-16T14:49:22.628 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7161000000000001e-05 2024-09-16T14:49:22.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.628 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.628 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.628 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939304+0000", 2024-09-16T14:49:22.628 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.329e-06 2024-09-16T14:49:22.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.628 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.628 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.629 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939378+0000", 2024-09-16T14:49:22.629 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3456999999999998e-05 2024-09-16T14:49:22.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.629 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.629 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:22.629 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939387+0000", 2024-09-16T14:49:22.629 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1449999999999997e-06 2024-09-16T14:49:22.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.629 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.629 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.629 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.941885+0000", 2024-09-16T14:49:22.629 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0024978349999999999 2024-09-16T14:49:22.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.630 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:22.630 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942001+0000", 2024-09-16T14:49:22.630 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011674499999999999 2024-09-16T14:49:22.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.630 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.630 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.953426+0000", 2024-09-16T14:49:22.630 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011424808999999999 2024-09-16T14:49:22.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.630 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.630 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956855+0000", 2024-09-16T14:49:22.631 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0034292160000000001 2024-09-16T14:49:22.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.631 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.631 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:22.631 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.958596+0000", 2024-09-16T14:49:22.631 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001740969 2024-09-16T14:49:22.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.631 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.631 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.631 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.958678+0000", 2024-09-16T14:49:22.631 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1592000000000001e-05 2024-09-16T14:49:22.631 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.631 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.631 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.632 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.632 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:366 3.4s0 3:2b1464a3:::smithi04040151-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:22.632 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:22.632 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.659101532999998, 2024-09-16T14:49:22.632 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.017101175, 2024-09-16T14:49:22.632 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.632 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:22.632 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:22.632 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:22.632 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:22.632 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 366 2024-09-16T14:49:22.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.633 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.633 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.633 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.633 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:22.633 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.633 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.633 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.633 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:22.633 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.633 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.633 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.634 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:22.634 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.634 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.634 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939321+0000", 2024-09-16T14:49:22.634 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.49e-06 2024-09-16T14:49:22.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.634 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.634 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939331+0000", 2024-09-16T14:49:22.634 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.006e-05 2024-09-16T14:49:22.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.635 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.635 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939336+0000", 2024-09-16T14:49:22.635 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2959999999999998e-06 2024-09-16T14:49:22.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.635 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.635 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939454+0000", 2024-09-16T14:49:22.635 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000117726 2024-09-16T14:49:22.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.635 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:22.635 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939461+0000", 2024-09-16T14:49:22.635 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1940000000000001e-06 2024-09-16T14:49:22.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.636 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.636 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942256+0000", 2024-09-16T14:49:22.636 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0027949350000000001 2024-09-16T14:49:22.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.636 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:22.636 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942325+0000", 2024-09-16T14:49:22.636 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.9119e-05 2024-09-16T14:49:22.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.636 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.636 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956179+0000", 2024-09-16T14:49:22.636 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013853343000000001 2024-09-16T14:49:22.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.637 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:22.637 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956295+0000", 2024-09-16T14:49:22.637 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000115899 2024-09-16T14:49:22.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.637 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.637 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956419+0000", 2024-09-16T14:49:22.637 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012467399999999999 2024-09-16T14:49:22.637 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.637 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.637 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:391 3.4s0 3:20ea1aa7:::smithi04040151-17:head [append 0~0] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.733889486999999, 2024-09-16T14:49:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011589693999999999, 2024-09-16T14:49:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 391 2024-09-16T14:49:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.638 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.639 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.639 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:22.639 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.639 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.639 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:22.639 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.639 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.639 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864528+0000", 2024-09-16T14:49:22.639 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:22.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.640 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.640 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864540+0000", 2024-09-16T14:49:22.640 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2514000000000001e-05 2024-09-16T14:49:22.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.640 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.640 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864569+0000", 2024-09-16T14:49:22.640 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8716000000000001e-05 2024-09-16T14:49:22.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.640 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.640 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864580+0000", 2024-09-16T14:49:22.641 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1564e-05 2024-09-16T14:49:22.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.641 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.641 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864681+0000", 2024-09-16T14:49:22.641 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010063100000000001 2024-09-16T14:49:22.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.641 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:22.641 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864690+0000", 2024-09-16T14:49:22.641 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6300000000000004e-06 2024-09-16T14:49:22.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.641 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.642 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.869283+0000", 2024-09-16T14:49:22.642 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.004593186 2024-09-16T14:49:22.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.642 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:22.642 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.869365+0000", 2024-09-16T14:49:22.642 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.174e-05 2024-09-16T14:49:22.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.642 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.642 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870206+0000", 2024-09-16T14:49:22.642 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00084176100000000001 2024-09-16T14:49:22.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.873261+0000", 2024-09-16T14:49:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0030546549999999999 2024-09-16T14:49:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.876069+0000", 2024-09-16T14:49:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0028082210000000001 2024-09-16T14:49:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.876120+0000", 2024-09-16T14:49:22.643 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.0581999999999998e-05 2024-09-16T14:49:22.644 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.644 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.644 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.644 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.644 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:392 3.4s0 3:20ea1aa7:::smithi04040151-17:head [rollback c] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:22.644 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:22.644 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.733798907000001, 2024-09-16T14:49:22.644 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013845265000000001, 2024-09-16T14:49:22.644 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.644 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:22.644 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:22.644 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:22.645 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:22.645 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 392 2024-09-16T14:49:22.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.645 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.645 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.645 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:22.645 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.645 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.645 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:22.645 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.646 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.646 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864620+0000", 2024-09-16T14:49:22.646 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:22.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.646 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.646 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864626+0000", 2024-09-16T14:49:22.646 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6699999999999999e-06 2024-09-16T14:49:22.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.646 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.646 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864643+0000", 2024-09-16T14:49:22.647 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7404000000000001e-05 2024-09-16T14:49:22.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.647 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.647 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864651+0000", 2024-09-16T14:49:22.647 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0860000000000002e-06 2024-09-16T14:49:22.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.647 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.647 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864759+0000", 2024-09-16T14:49:22.648 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010751000000000001 2024-09-16T14:49:22.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.648 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:22.648 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864765+0000", 2024-09-16T14:49:22.648 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.737e-06 2024-09-16T14:49:22.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.648 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.648 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870832+0000", 2024-09-16T14:49:22.648 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060664930000000001 2024-09-16T14:49:22.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.649 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:22.649 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870972+0000", 2024-09-16T14:49:22.649 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00014033599999999999 2024-09-16T14:49:22.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.649 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.649 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.649 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.871637+0000", 2024-09-16T14:49:22.649 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00066502800000000002 2024-09-16T14:49:22.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.649 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.649 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.649 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.873263+0000", 2024-09-16T14:49:22.649 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0016256409999999999 2024-09-16T14:49:22.650 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.650 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:22.650 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.878403+0000", 2024-09-16T14:49:22.650 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0051401600000000004 2024-09-16T14:49:22.650 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.650 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.650 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.878466+0000", 2024-09-16T14:49:22.650 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.2964000000000002e-05 2024-09-16T14:49:22.650 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.650 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.650 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.650 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.651 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.651 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:393 3.4s0 3:20ea1aa7:::smithi04040151-17:head [append 0~0] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:22.651 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:22.651 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.733739313000001, 2024-09-16T14:49:22.651 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.016150277000000001, 2024-09-16T14:49:22.651 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.651 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:22.651 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:22.651 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:22.651 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:22.651 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 393 2024-09-16T14:49:22.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.651 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.651 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.652 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.652 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:22.652 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.652 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.652 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.652 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.652 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:22.652 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.652 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.652 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.652 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.652 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:22.652 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.652 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.653 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.653 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.653 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864688+0000", 2024-09-16T14:49:22.653 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5259999999999993e-06 2024-09-16T14:49:22.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.653 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.653 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.653 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864705+0000", 2024-09-16T14:49:22.653 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6368999999999999e-05 2024-09-16T14:49:22.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.653 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.653 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.653 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864712+0000", 2024-09-16T14:49:22.654 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4749999999999996e-06 2024-09-16T14:49:22.654 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.654 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.654 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.654 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864791+0000", 2024-09-16T14:49:22.654 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9159000000000006e-05 2024-09-16T14:49:22.654 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.654 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.654 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:22.654 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864797+0000", 2024-09-16T14:49:22.654 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5400000000000003e-06 2024-09-16T14:49:22.654 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.654 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.654 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.655 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872109+0000", 2024-09-16T14:49:22.655 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073124569999999996 2024-09-16T14:49:22.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.655 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.655 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:22.655 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872237+0000", 2024-09-16T14:49:22.655 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012735099999999999 2024-09-16T14:49:22.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.655 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.655 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.655 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872865+0000", 2024-09-16T14:49:22.655 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00062808000000000004 2024-09-16T14:49:22.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.655 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.656 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.656 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.874081+0000", 2024-09-16T14:49:22.656 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001216186 2024-09-16T14:49:22.656 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.656 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.656 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:22.656 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.880752+0000", 2024-09-16T14:49:22.656 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0066711339999999996 2024-09-16T14:49:22.656 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.656 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.656 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.656 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.880831+0000", 2024-09-16T14:49:22.656 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8681000000000001e-05 2024-09-16T14:49:22.656 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.657 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.657 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.657 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.657 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.657 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.9s5 51/27 ECSubWrite(tid=35, reqid=osd.8.0:35, at_version=51'21, trim_to=0'0, roll_forward_to=47'19))", 2024-09-16T14:49:22.657 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:22.657 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.709294060000001, 2024-09-16T14:49:22.657 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010449557999999999, 2024-09-16T14:49:22.657 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.657 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.657 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.657 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.657 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.658 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:22.658 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.658 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.658 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.658 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.658 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:22.658 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.658 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.658 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.658 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.658 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889125+0000", 2024-09-16T14:49:22.658 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.658 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.658 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.659 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.659 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889142+0000", 2024-09-16T14:49:22.659 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7186000000000001e-05 2024-09-16T14:49:22.659 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.659 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.659 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.659 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889152+0000", 2024-09-16T14:49:22.659 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7920000000000004e-06 2024-09-16T14:49:22.659 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.659 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.659 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.659 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889159+0000", 2024-09-16T14:49:22.659 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.0319999999999998e-06 2024-09-16T14:49:22.659 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.660 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.660 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.660 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889203+0000", 2024-09-16T14:49:22.660 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.4264000000000002e-05 2024-09-16T14:49:22.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.660 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.660 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.660 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889212+0000", 2024-09-16T14:49:22.660 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6940000000000001e-06 2024-09-16T14:49:22.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.660 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.660 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.661 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.899537+0000", 2024-09-16T14:49:22.661 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010325328 2024-09-16T14:49:22.661 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.661 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.661 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.661 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.899575+0000", 2024-09-16T14:49:22.661 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.7741999999999999e-05 2024-09-16T14:49:22.661 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.661 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.661 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.661 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.661 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.661 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.2s5 51/27 ECSubWrite(tid=175, reqid=client.4500.0:460, at_version=51'53, trim_to=0'0, roll_forward_to=51'51))", 2024-09-16T14:49:22.662 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.994224+0000", 2024-09-16T14:49:22.662 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.604195208, 2024-09-16T14:49:22.662 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085171090000000001, 2024-09-16T14:49:22.662 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.662 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.662 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.662 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.662 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.662 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994224+0000", 2024-09-16T14:49:22.662 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.662 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.662 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.662 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.662 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994224+0000", 2024-09-16T14:49:22.663 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.663 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.663 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.663 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.663 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994223+0000", 2024-09-16T14:49:22.663 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:22.663 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.663 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.663 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.663 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994266+0000", 2024-09-16T14:49:22.663 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2602000000000003e-05 2024-09-16T14:49:22.663 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.663 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.663 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.664 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994293+0000", 2024-09-16T14:49:22.664 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7773999999999999e-05 2024-09-16T14:49:22.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.664 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.664 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.664 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994304+0000", 2024-09-16T14:49:22.664 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0932e-05 2024-09-16T14:49:22.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.664 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.664 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.664 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994371+0000", 2024-09-16T14:49:22.664 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6202000000000001e-05 2024-09-16T14:49:22.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.665 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.665 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994383+0000", 2024-09-16T14:49:22.665 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2685e-05 2024-09-16T14:49:22.665 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.665 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.665 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002711+0000", 2024-09-16T14:49:22.665 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0083273529999999991 2024-09-16T14:49:22.665 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.665 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.665 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002742+0000", 2024-09-16T14:49:22.666 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0936000000000003e-05 2024-09-16T14:49:22.666 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.666 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.666 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.666 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.666 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.666 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds4 51/27 ECSubWrite(tid=211, reqid=client.4500.0:459, at_version=51'31, trim_to=0'0, roll_forward_to=51'29))", 2024-09-16T14:49:22.666 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.994919+0000", 2024-09-16T14:49:22.666 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.603500599, 2024-09-16T14:49:22.666 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0087600569999999999, 2024-09-16T14:49:22.666 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.666 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.666 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.667 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.667 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.667 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994919+0000", 2024-09-16T14:49:22.667 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.667 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.667 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.667 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994919+0000", 2024-09-16T14:49:22.667 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.667 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.667 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.667 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994918+0000", 2024-09-16T14:49:22.668 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:22.668 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.668 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.668 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995629+0000", 2024-09-16T14:49:22.668 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000711245 2024-09-16T14:49:22.668 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.668 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.668 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995643+0000", 2024-09-16T14:49:22.668 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.398e-05 2024-09-16T14:49:22.668 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.669 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.669 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995651+0000", 2024-09-16T14:49:22.669 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0479999999999999e-06 2024-09-16T14:49:22.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.669 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.669 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.669 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995722+0000", 2024-09-16T14:49:22.669 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.0137999999999998e-05 2024-09-16T14:49:22.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.669 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.669 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.669 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995731+0000", 2024-09-16T14:49:22.669 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5279999999999992e-06 2024-09-16T14:49:22.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003653+0000", 2024-09-16T14:49:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0079221599999999993 2024-09-16T14:49:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003679+0000", 2024-09-16T14:49:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5882999999999999e-05 2024-09-16T14:49:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.670 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.671 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s5 51/27 ECSubWrite(tid=60, reqid=client.4500.0:467, at_version=51'19, trim_to=0'0, roll_forward_to=51'16))", 2024-09-16T14:49:22.671 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:22.671 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.600494855000001, 2024-09-16T14:49:22.671 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0099497439999999999, 2024-09-16T14:49:22.671 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.671 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.671 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.671 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.671 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.671 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:22.671 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.671 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.671 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.671 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.672 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:22.672 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.672 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.672 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.672 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.672 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997924+0000", 2024-09-16T14:49:22.672 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:22.672 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.672 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.672 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.672 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997944+0000", 2024-09-16T14:49:22.672 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0404e-05 2024-09-16T14:49:22.673 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.673 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.673 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.673 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997957+0000", 2024-09-16T14:49:22.673 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2123000000000001e-05 2024-09-16T14:49:22.673 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.673 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.673 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.673 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997966+0000", 2024-09-16T14:49:22.673 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6549999999999999e-06 2024-09-16T14:49:22.673 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.673 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.673 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.674 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002158+0000", 2024-09-16T14:49:22.674 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0041915930000000004 2024-09-16T14:49:22.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.674 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.674 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.674 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002168+0000", 2024-09-16T14:49:22.674 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0071000000000001e-05 2024-09-16T14:49:22.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.674 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.674 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.674 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007845+0000", 2024-09-16T14:49:22.674 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056765690000000002 2024-09-16T14:49:22.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.674 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.675 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.675 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007875+0000", 2024-09-16T14:49:22.675 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0077e-05 2024-09-16T14:49:22.675 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.675 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.675 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.675 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.675 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.675 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s5 51/27 ECSubWrite(tid=62, reqid=client.4500.0:468, at_version=51'20, trim_to=0'0, roll_forward_to=51'16))", 2024-09-16T14:49:22.675 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.999395+0000", 2024-09-16T14:49:22.675 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.599024905, 2024-09-16T14:49:22.675 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084961840000000004, 2024-09-16T14:49:22.675 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.676 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:22.676 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.676 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.676 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999395+0000", 2024-09-16T14:49:22.676 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.676 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.676 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.676 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999395+0000", 2024-09-16T14:49:22.676 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.676 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.676 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.677 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999394+0000", 2024-09-16T14:49:22.677 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:22.677 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.677 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.677 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.677 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999413+0000", 2024-09-16T14:49:22.677 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9508999999999999e-05 2024-09-16T14:49:22.677 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.677 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.677 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.677 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999436+0000", 2024-09-16T14:49:22.677 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2512999999999999e-05 2024-09-16T14:49:22.677 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.678 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.678 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.678 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.999445+0000", 2024-09-16T14:49:22.678 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4390000000000006e-06 2024-09-16T14:49:22.678 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.678 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.678 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.678 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003294+0000", 2024-09-16T14:49:22.678 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0038483250000000001 2024-09-16T14:49:22.678 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.678 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.678 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.678 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.003304+0000", 2024-09-16T14:49:22.678 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:49:22.679 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.679 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.679 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.679 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007879+0000", 2024-09-16T14:49:22.679 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0045754569999999998 2024-09-16T14:49:22.679 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.679 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.679 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.679 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007891+0000", 2024-09-16T14:49:22.679 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1953000000000001e-05 2024-09-16T14:49:22.679 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.679 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.679 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.680 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.680 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.680 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:506 3.4s0 3:255172f3:::smithi04040151-3:head [rollback 13] snapc 1b=[1b,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-16T14:49:22.680 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:22.680 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.704057225, 2024-09-16T14:49:22.680 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0092517710000000006, 2024-09-16T14:49:22.680 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.680 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:22.680 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:22.680 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:22.680 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:22.680 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 506 2024-09-16T14:49:22.680 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.680 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.681 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.681 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.681 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:22.681 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.681 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.681 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.681 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.681 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:22.681 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.681 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.681 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.681 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.681 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:22.682 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.682 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.682 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.682 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.682 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894365+0000", 2024-09-16T14:49:22.682 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.571e-06 2024-09-16T14:49:22.682 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.682 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.682 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.682 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894370+0000", 2024-09-16T14:49:22.682 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8279999999999999e-06 2024-09-16T14:49:22.682 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.682 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.682 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.683 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894374+0000", 2024-09-16T14:49:22.683 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5920000000000001e-06 2024-09-16T14:49:22.683 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.683 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.683 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.683 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894483+0000", 2024-09-16T14:49:22.683 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010895699999999999 2024-09-16T14:49:22.683 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.683 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.683 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:22.683 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894491+0000", 2024-09-16T14:49:22.683 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9389999999999997e-06 2024-09-16T14:49:22.683 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.683 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.684 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.684 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.897713+0000", 2024-09-16T14:49:22.684 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0032221060000000002 2024-09-16T14:49:22.684 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.684 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.684 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:22.684 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.897829+0000", 2024-09-16T14:49:22.684 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000116082 2024-09-16T14:49:22.684 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.684 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.684 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.684 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.898559+0000", 2024-09-16T14:49:22.684 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00072999800000000004 2024-09-16T14:49:22.684 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.685 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.685 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.685 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.901829+0000", 2024-09-16T14:49:22.685 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0032702159999999998 2024-09-16T14:49:22.685 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.685 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.685 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:22.685 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903546+0000", 2024-09-16T14:49:22.685 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001716827 2024-09-16T14:49:22.685 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.685 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.685 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.685 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903614+0000", 2024-09-16T14:49:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8058999999999993e-05 2024-09-16T14:49:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:507 3.4s0 3:255172f3:::smithi04040151-3:head [append 0~0] snapc 1b=[1b,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-16T14:49:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.704025250000001, 2024-09-16T14:49:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0095393700000000001, 2024-09-16T14:49:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:22.686 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 507 2024-09-16T14:49:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.688 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.688 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:22.688 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.688 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.688 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.688 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894397+0000", 2024-09-16T14:49:22.688 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.774e-06 2024-09-16T14:49:22.688 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.688 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.688 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894403+0000", 2024-09-16T14:49:22.689 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1079999999999998e-06 2024-09-16T14:49:22.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.689 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.689 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.689 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894406+0000", 2024-09-16T14:49:22.689 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.337e-06 2024-09-16T14:49:22.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.689 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.689 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.689 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894536+0000", 2024-09-16T14:49:22.689 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012969199999999999 2024-09-16T14:49:22.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.689 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.689 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:22.690 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894545+0000", 2024-09-16T14:49:22.690 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7499999999999992e-06 2024-09-16T14:49:22.690 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.690 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.690 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.690 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899040+0000", 2024-09-16T14:49:22.690 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044951289999999996 2024-09-16T14:49:22.690 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.690 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.690 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:22.690 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899164+0000", 2024-09-16T14:49:22.690 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012377699999999999 2024-09-16T14:49:22.690 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.690 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.691 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.691 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899721+0000", 2024-09-16T14:49:22.691 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00055683800000000004 2024-09-16T14:49:22.691 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.691 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.691 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.901911+0000", 2024-09-16T14:49:22.691 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0021908650000000002 2024-09-16T14:49:22.691 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.691 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:22.691 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903859+0000", 2024-09-16T14:49:22.691 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001947383 2024-09-16T14:49:22.691 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.692 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.692 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.692 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903934+0000", 2024-09-16T14:49:22.692 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5056999999999996e-05 2024-09-16T14:49:22.692 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.692 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.692 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.692 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.692 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.692 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:588 3.4s0 3:255172f3:::smithi04040151-3:head [append 0~0] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:22.692 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:11.975961+0000", 2024-09-16T14:49:22.692 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.622458760000001, 2024-09-16T14:49:22.692 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0258510810000001, 2024-09-16T14:49:22.692 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 588 2024-09-16T14:49:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975961+0000", 2024-09-16T14:49:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.693 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975961+0000", 2024-09-16T14:49:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975959+0000", 2024-09-16T14:49:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975968+0000", 2024-09-16T14:49:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1020000000000003e-06 2024-09-16T14:49:22.694 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.695 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.695 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.695 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975982+0000", 2024-09-16T14:49:22.695 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.33e-05 2024-09-16T14:49:22.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.695 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.695 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.695 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975990+0000", 2024-09-16T14:49:22.695 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4679999999999992e-06 2024-09-16T14:49:22.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.695 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.695 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.695 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976080+0000", 2024-09-16T14:49:22.695 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9642999999999995e-05 2024-09-16T14:49:22.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.696 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.696 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for peered", 2024-09-16T14:49:22.696 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976083+0000", 2024-09-16T14:49:22.696 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.92e-06 2024-09-16T14:49:22.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.696 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.696 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.696 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.983734+0000", 2024-09-16T14:49:22.696 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.007651056 2024-09-16T14:49:22.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.696 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.696 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:22.697 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.984659+0000", 2024-09-16T14:49:22.697 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000925493 2024-09-16T14:49:22.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.697 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.697 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.697 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.991407+0000", 2024-09-16T14:49:22.697 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0067480969999999998 2024-09-16T14:49:22.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.697 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.697 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:22.697 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.991550+0000", 2024-09-16T14:49:22.697 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00014299700000000001 2024-09-16T14:49:22.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.698 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.698 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.698 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.992634+0000", 2024-09-16T14:49:22.698 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001083894 2024-09-16T14:49:22.698 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.698 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.698 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.698 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.998056+0000", 2024-09-16T14:49:22.698 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0054220070000000004 2024-09-16T14:49:22.698 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.698 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.698 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:22.698 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.001429+0000", 2024-09-16T14:49:22.698 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0033727480000000001 2024-09-16T14:49:22.699 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.699 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.699 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.699 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.001812+0000", 2024-09-16T14:49:22.699 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00038303600000000001 2024-09-16T14:49:22.699 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.699 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.699 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.699 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.699 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.699 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:589 3.4s0 3:255172f3:::smithi04040151-3:head [rollback 24] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:22.699 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:11.976008+0000", 2024-09-16T14:49:22.699 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.622411815, 2024-09-16T14:49:22.700 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.027673222, 2024-09-16T14:49:22.700 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.700 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:22.700 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:22.700 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:22.700 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:22.700 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 589 2024-09-16T14:49:22.700 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.700 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.700 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.700 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.700 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976008+0000", 2024-09-16T14:49:22.700 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.701 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.701 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.701 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.701 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976008+0000", 2024-09-16T14:49:22.701 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.701 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.701 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.701 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.701 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976007+0000", 2024-09-16T14:49:22.701 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.701 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.701 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.701 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.701 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976012+0000", 2024-09-16T14:49:22.702 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9760000000000003e-06 2024-09-16T14:49:22.702 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.702 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.702 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.702 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976025+0000", 2024-09-16T14:49:22.702 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2884000000000001e-05 2024-09-16T14:49:22.702 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.702 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.702 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.702 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976031+0000", 2024-09-16T14:49:22.702 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0079999999999999e-06 2024-09-16T14:49:22.702 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.702 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.702 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.703 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976122+0000", 2024-09-16T14:49:22.703 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0946000000000006e-05 2024-09-16T14:49:22.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.703 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for peered", 2024-09-16T14:49:22.703 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976123+0000", 2024-09-16T14:49:22.703 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1489999999999999e-06 2024-09-16T14:49:22.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.703 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.703 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.985902+0000", 2024-09-16T14:49:22.703 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.009778528 2024-09-16T14:49:22.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.704 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:22.704 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.986100+0000", 2024-09-16T14:49:22.704 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00019825499999999999 2024-09-16T14:49:22.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.704 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.704 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.993202+0000", 2024-09-16T14:49:22.704 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071018089999999997 2024-09-16T14:49:22.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.704 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:22.704 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.993327+0000", 2024-09-16T14:49:22.704 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000125031 2024-09-16T14:49:22.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.705 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.705 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.994109+0000", 2024-09-16T14:49:22.705 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00078209799999999995 2024-09-16T14:49:22.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.705 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.705 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.999045+0000", 2024-09-16T14:49:22.705 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0049362879999999996 2024-09-16T14:49:22.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.705 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:22.705 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.003322+0000", 2024-09-16T14:49:22.705 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0042765879999999996 2024-09-16T14:49:22.706 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.706 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.706 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.003681+0000", 2024-09-16T14:49:22.706 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00035919199999999999 2024-09-16T14:49:22.706 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.706 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.706 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.706 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.706 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:590 3.4s0 3:255172f3:::smithi04040151-3:head [append 0~0] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:22.706 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:11.976073+0000", 2024-09-16T14:49:22.706 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.622346937, 2024-09-16T14:49:22.706 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0282184670000001, 2024-09-16T14:49:22.707 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:22.707 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:22.707 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:22.707 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:22.707 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:22.707 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 590 2024-09-16T14:49:22.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.707 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:22.707 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.707 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:22.707 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976073+0000", 2024-09-16T14:49:22.707 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.708 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.708 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:22.708 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976073+0000", 2024-09-16T14:49:22.708 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:22.708 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.708 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.708 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:22.708 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976072+0000", 2024-09-16T14:49:22.708 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:22.708 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.708 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976076+0000", 2024-09-16T14:49:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.985e-06 2024-09-16T14:49:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976084+0000", 2024-09-16T14:49:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0080000000000002e-06 2024-09-16T14:49:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976095+0000", 2024-09-16T14:49:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0413e-05 2024-09-16T14:49:22.709 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976150+0000", 2024-09-16T14:49:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5588999999999999e-05 2024-09-16T14:49:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for peered", 2024-09-16T14:49:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976151+0000", 2024-09-16T14:49:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8700000000000004e-07 2024-09-16T14:49:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.986165+0000", 2024-09-16T14:49:22.710 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.010013982 2024-09-16T14:49:22.711 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.711 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.711 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:22.711 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.986260+0000", 2024-09-16T14:49:22.711 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4545999999999999e-05 2024-09-16T14:49:22.711 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.711 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.711 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:22.711 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.994573+0000", 2024-09-16T14:49:22.711 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0083134779999999991 2024-09-16T14:49:22.711 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.711 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.711 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:22.711 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.994693+0000", 2024-09-16T14:49:22.712 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000119707 2024-09-16T14:49:22.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.712 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.712 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:22.712 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.995284+0000", 2024-09-16T14:49:22.712 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00059083400000000004 2024-09-16T14:49:22.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.712 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.712 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:22.712 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.999048+0000", 2024-09-16T14:49:22.712 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.003763878 2024-09-16T14:49:22.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.712 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.713 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:22.713 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.003931+0000", 2024-09-16T14:49:22.713 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0048833339999999996 2024-09-16T14:49:22.713 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:22.713 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:22.713 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:22.713 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.004291+0000", 2024-09-16T14:49:22.713 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00036028999999999998 2024-09-16T14:49:22.713 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.713 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.713 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.713 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:22.713 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:22.713 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:22.715 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:49:22.715 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:22.713+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:22.816 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:22.815+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:22.822 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:22.822 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:22.822 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:22.822 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:22.830 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:49:22.890 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T14:49:22.891 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is set 2024-09-16T14:49:22.891 INFO:tasks.rados.rados.0.smithi040.stdout:286: done (2 left) 2024-09-16T14:49:22.891 INFO:tasks.rados.rados.0.smithi040.stdout:287: done (1 left) 2024-09-16T14:49:22.891 INFO:tasks.rados.rados.0.smithi040.stdout:288: done (0 left) 2024-09-16T14:49:22.892 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 34 current snap is 26 2024-09-16T14:49:22.892 INFO:tasks.rados.rados.0.smithi040.stdout:289: seq_num 113 ranges {557056=737280,1294336=557056,1851392=606208} 2024-09-16T14:49:22.898 INFO:tasks.rados.rados.0.smithi040.stdout:289: writing smithi04040151-34 from 557056 to 1294336 tid 1 2024-09-16T14:49:22.901 INFO:tasks.rados.rados.0.smithi040.stdout:289: writing smithi04040151-34 from 1294336 to 1851392 tid 2 2024-09-16T14:49:22.905 INFO:tasks.rados.rados.0.smithi040.stdout:289: writing smithi04040151-34 from 1851392 to 2457600 tid 3 2024-09-16T14:49:22.905 INFO:tasks.rados.rados.0.smithi040.stdout:290: snap_remove snap 23 2024-09-16T14:49:22.909 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T14:49:22.916 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:22.915+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:22.946 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:22.947 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:22.947 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:22.947 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:22.947 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:22.956 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:49:23.018 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:23.016+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:23.106 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:23.106 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:23.106 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:23.106 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:23.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.106 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=128, reqid=client.4500.0:369, at_version=42'38, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:23.106 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:23.106 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 34.144144109999999, 2024-09-16T14:49:23.106 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060794769999999998, 2024-09-16T14:49:23.106 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.107 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.107 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.107 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.107 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:23.107 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.107 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.107 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:23.107 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.107 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.108 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959976+0000", 2024-09-16T14:49:23.108 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.108 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.108 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960436+0000", 2024-09-16T14:49:23.108 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00046022400000000002 2024-09-16T14:49:23.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.108 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.108 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960455+0000", 2024-09-16T14:49:23.108 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8162999999999999e-05 2024-09-16T14:49:23.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.109 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.109 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960468+0000", 2024-09-16T14:49:23.109 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3135999999999999e-05 2024-09-16T14:49:23.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.109 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.109 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960527+0000", 2024-09-16T14:49:23.109 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8859e-05 2024-09-16T14:49:23.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.109 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.109 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960533+0000", 2024-09-16T14:49:23.109 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:49:23.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.110 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.110 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.966018+0000", 2024-09-16T14:49:23.110 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0054845069999999996 2024-09-16T14:49:23.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.110 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.110 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.966057+0000", 2024-09-16T14:49:23.110 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.8927999999999997e-05 2024-09-16T14:49:23.110 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.110 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.110 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.111 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=129, reqid=client.4500.0:370, at_version=42'39, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:23.111 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:23.111 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 34.142888352, 2024-09-16T14:49:23.111 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074279639999999996, 2024-09-16T14:49:23.111 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.111 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.111 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.111 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.111 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:23.111 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.111 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.112 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:23.112 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.112 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.112 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961232+0000", 2024-09-16T14:49:23.112 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.112 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.112 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962118+0000", 2024-09-16T14:49:23.112 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00088592399999999996 2024-09-16T14:49:23.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.113 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.113 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962132+0000", 2024-09-16T14:49:23.113 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4127e-05 2024-09-16T14:49:23.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.113 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.113 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962142+0000", 2024-09-16T14:49:23.113 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0181e-05 2024-09-16T14:49:23.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.113 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.113 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962192+0000", 2024-09-16T14:49:23.113 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9826e-05 2024-09-16T14:49:23.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962202+0000", 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8400000000000007e-06 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968622+0000", 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00642021 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968661+0000", 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.8670000000000001e-05 2024-09-16T14:49:23.114 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=130, reqid=client.4500.0:371, at_version=42'40, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 34.141553096999999, 2024-09-16T14:49:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0061144900000000002, 2024-09-16T14:49:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.115 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962567+0000", 2024-09-16T14:49:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.116 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.117 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962595+0000", 2024-09-16T14:49:23.117 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7824999999999999e-05 2024-09-16T14:49:23.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.117 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.117 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962611+0000", 2024-09-16T14:49:23.117 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:49:23.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.117 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.117 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962620+0000", 2024-09-16T14:49:23.117 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8259999999999999e-06 2024-09-16T14:49:23.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.963285+0000", 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00066560100000000004 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.963295+0000", 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0036e-05 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968667+0000", 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0053715960000000002 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.119 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.119 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968683+0000", 2024-09-16T14:49:23.119 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5522000000000001e-05 2024-09-16T14:49:23.119 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.119 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.119 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.119 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 43/27 ECSubWrite(tid=133, reqid=osd.11.0:133, at_version=43'42, trim_to=0'0, roll_forward_to=42'40))", 2024-09-16T14:49:23.119 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:23.119 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.250345303000003, 2024-09-16T14:49:23.119 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073093740000000004, 2024-09-16T14:49:23.119 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.119 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.120 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.120 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.120 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:23.120 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.120 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.120 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:23.120 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.120 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.121 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:23.121 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:23.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.121 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.121 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854515+0000", 2024-09-16T14:49:23.121 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00073950999999999997 2024-09-16T14:49:23.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.121 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.121 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854527+0000", 2024-09-16T14:49:23.121 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2012e-05 2024-09-16T14:49:23.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854535+0000", 2024-09-16T14:49:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:49:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854952+0000", 2024-09-16T14:49:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00041779399999999998 2024-09-16T14:49:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854961+0000", 2024-09-16T14:49:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7749999999999998e-06 2024-09-16T14:49:23.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.861052+0000", 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006090608 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.861085+0000", 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3442999999999998e-05 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 51/27 ECSubWrite(tid=169, reqid=osd.11.0:169, at_version=51'52, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.222935381999999, 2024-09-16T14:49:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006781848, 2024-09-16T14:49:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881185+0000", 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881208+0000", 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2823000000000002e-05 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881218+0000", 2024-09-16T14:49:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3929999999999997e-06 2024-09-16T14:49:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881225+0000", 2024-09-16T14:49:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1629999999999999e-06 2024-09-16T14:49:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881279+0000", 2024-09-16T14:49:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4147000000000002e-05 2024-09-16T14:49:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.127 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.127 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881288+0000", 2024-09-16T14:49:23.127 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4630000000000007e-06 2024-09-16T14:49:23.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.127 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.127 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.127 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.887941+0000", 2024-09-16T14:49:23.127 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0066530210000000003 2024-09-16T14:49:23.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.127 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.127 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.127 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.887968+0000", 2024-09-16T14:49:23.127 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6228999999999999e-05 2024-09-16T14:49:23.127 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 51/27 ECSubWrite(tid=173, reqid=osd.11.0:173, at_version=51'54, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.222784154999999, 2024-09-16T14:49:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0080949799999999999, 2024-09-16T14:49:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881336+0000", 2024-09-16T14:49:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881356+0000", 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9334000000000002e-05 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881366+0000", 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0742e-05 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881374+0000", 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5020000000000003e-06 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.882087+0000", 2024-09-16T14:49:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00071271300000000002 2024-09-16T14:49:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.882096+0000", 2024-09-16T14:49:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4550000000000001e-06 2024-09-16T14:49:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889403+0000", 2024-09-16T14:49:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073067749999999997 2024-09-16T14:49:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889432+0000", 2024-09-16T14:49:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9065999999999999e-05 2024-09-16T14:49:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 52/27 ECSubWrite(tid=50, reqid=osd.1.0:50, at_version=52'40, trim_to=0'0, roll_forward_to=51'36))", 2024-09-16T14:49:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.221581285999999, 2024-09-16T14:49:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060730380000000002, 2024-09-16T14:49:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882539+0000", 2024-09-16T14:49:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882566+0000", 2024-09-16T14:49:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7101999999999999e-05 2024-09-16T14:49:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882580+0000", 2024-09-16T14:49:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4605e-05 2024-09-16T14:49:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882589+0000", 2024-09-16T14:49:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1020000000000003e-06 2024-09-16T14:49:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.884101+0000", 2024-09-16T14:49:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001511601 2024-09-16T14:49:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.884112+0000", 2024-09-16T14:49:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1055e-05 2024-09-16T14:49:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888581+0000", 2024-09-16T14:49:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044684240000000004 2024-09-16T14:49:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888613+0000", 2024-09-16T14:49:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2354000000000002e-05 2024-09-16T14:49:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=235, reqid=osd.7.0:235, at_version=61'21, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.172595278999999, 2024-09-16T14:49:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0080251469999999998, 2024-09-16T14:49:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931525+0000", 2024-09-16T14:49:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931556+0000", 2024-09-16T14:49:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0713000000000002e-05 2024-09-16T14:49:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931577+0000", 2024-09-16T14:49:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0879000000000001e-05 2024-09-16T14:49:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931599+0000", 2024-09-16T14:49:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2107999999999999e-05 2024-09-16T14:49:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932715+0000", 2024-09-16T14:49:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0011159119999999999 2024-09-16T14:49:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932725+0000", 2024-09-16T14:49:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8239999999999995e-06 2024-09-16T14:49:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939520+0000", 2024-09-16T14:49:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00679556 2024-09-16T14:49:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939551+0000", 2024-09-16T14:49:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0582e-05 2024-09-16T14:49:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 61/27 ECSubWrite(tid=63, reqid=osd.1.0:63, at_version=61'50, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.172474489000001, 2024-09-16T14:49:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090916260000000002, 2024-09-16T14:49:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931646+0000", 2024-09-16T14:49:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931665+0000", 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8459999999999999e-05 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931677+0000", 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2150000000000001e-05 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931684+0000", 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3599999999999998e-06 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931754+0000", 2024-09-16T14:49:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.9733999999999993e-05 2024-09-16T14:49:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931766+0000", 2024-09-16T14:49:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1873e-05 2024-09-16T14:49:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940712+0000", 2024-09-16T14:49:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089461669999999997 2024-09-16T14:49:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940738+0000", 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6092999999999999e-05 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.172074476000001, 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010994122, 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932066+0000", 2024-09-16T14:49:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9802000000000001e-05 2024-09-16T14:49:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932076+0000", 2024-09-16T14:49:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8470000000000008e-06 2024-09-16T14:49:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932086+0000", 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7010000000000008e-06 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933307+0000", 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0012207450000000001 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933319+0000", 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2473000000000001e-05 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.150 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943013+0000", 2024-09-16T14:49:23.150 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0096943700000000008 2024-09-16T14:49:23.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.150 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.150 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943041+0000", 2024-09-16T14:49:23.150 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7376999999999999e-05 2024-09-16T14:49:23.150 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.150 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.150 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.150 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:23.150 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:23.151 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.170949648000001, 2024-09-16T14:49:23.151 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0099360779999999992, 2024-09-16T14:49:23.151 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.151 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.151 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.151 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.151 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:23.151 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.151 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.151 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:23.151 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.152 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.152 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933171+0000", 2024-09-16T14:49:23.152 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.152 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.152 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933205+0000", 2024-09-16T14:49:23.152 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3630999999999997e-05 2024-09-16T14:49:23.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.152 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.152 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933214+0000", 2024-09-16T14:49:23.153 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0620000000000007e-06 2024-09-16T14:49:23.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.153 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.153 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933222+0000", 2024-09-16T14:49:23.153 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:23.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.153 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.153 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933271+0000", 2024-09-16T14:49:23.153 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9251e-05 2024-09-16T14:49:23.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.153 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.154 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933280+0000", 2024-09-16T14:49:23.154 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5419999999999994e-06 2024-09-16T14:49:23.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.154 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.154 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943089+0000", 2024-09-16T14:49:23.154 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0098082529999999994 2024-09-16T14:49:23.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.154 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.154 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943108+0000", 2024-09-16T14:49:23.154 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8916e-05 2024-09-16T14:49:23.154 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s2 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.170811119, 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01234993, 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:23.155 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.156 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.156 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.156 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:23.156 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.156 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.156 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.156 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:23.156 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.156 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.156 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.156 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933339+0000", 2024-09-16T14:49:23.157 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9107000000000001e-05 2024-09-16T14:49:23.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.157 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.157 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933347+0000", 2024-09-16T14:49:23.157 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6219999999999998e-06 2024-09-16T14:49:23.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.157 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.157 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933354+0000", 2024-09-16T14:49:23.157 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.72e-06 2024-09-16T14:49:23.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.157 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933438+0000", 2024-09-16T14:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.3666999999999997e-05 2024-09-16T14:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933453+0000", 2024-09-16T14:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4814000000000001e-05 2024-09-16T14:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945638+0000", 2024-09-16T14:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012185367000000001 2024-09-16T14:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945660+0000", 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2033999999999999e-05 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.170765998, 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012249365, 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.159 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.160 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.160 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.160 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:23.160 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.160 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.160 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.160 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:23.160 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.160 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.160 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.160 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933354+0000", 2024-09-16T14:49:23.160 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.161 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.161 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933377+0000", 2024-09-16T14:49:23.161 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3087000000000001e-05 2024-09-16T14:49:23.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.161 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.161 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933393+0000", 2024-09-16T14:49:23.161 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5231e-05 2024-09-16T14:49:23.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.161 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.161 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933400+0000", 2024-09-16T14:49:23.162 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.6190000000000002e-06 2024-09-16T14:49:23.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.162 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.162 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.162 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933470+0000", 2024-09-16T14:49:23.162 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.936e-05 2024-09-16T14:49:23.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.162 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.162 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.162 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933479+0000", 2024-09-16T14:49:23.162 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2609999999999999e-06 2024-09-16T14:49:23.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.162 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.162 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.163 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945575+0000", 2024-09-16T14:49:23.163 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012095896 2024-09-16T14:49:23.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.163 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.163 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945605+0000", 2024-09-16T14:49:23.163 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9765e-05 2024-09-16T14:49:23.163 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.163 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.163 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.163 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:23.163 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:23.164 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.170723704, 2024-09-16T14:49:23.164 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013525466, 2024-09-16T14:49:23.164 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.164 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.164 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.164 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.164 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:23.164 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.164 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.164 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:23.164 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.165 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.165 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.165 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:23.165 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.165 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.165 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.165 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933451+0000", 2024-09-16T14:49:23.165 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4162000000000001e-05 2024-09-16T14:49:23.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.165 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.165 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.165 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933459+0000", 2024-09-16T14:49:23.166 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1110000000000008e-06 2024-09-16T14:49:23.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.166 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.166 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.166 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:23.166 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5050000000000004e-06 2024-09-16T14:49:23.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.167 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.167 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.936478+0000", 2024-09-16T14:49:23.167 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0030122790000000001 2024-09-16T14:49:23.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.167 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.167 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.936488+0000", 2024-09-16T14:49:23.167 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0197000000000001e-05 2024-09-16T14:49:23.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.167 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.168 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946886+0000", 2024-09-16T14:49:23.168 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010397912 2024-09-16T14:49:23.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.168 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.168 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946923+0000", 2024-09-16T14:49:23.168 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6699999999999998e-05 2024-09-16T14:49:23.169 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.170 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.170 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.170 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:23.170 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:23.171 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.169566243, 2024-09-16T14:49:23.171 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0083397039999999999, 2024-09-16T14:49:23.171 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.171 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.171 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.171 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.171 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.171 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:23.171 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.171 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.171 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.171 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.171 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:23.171 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.172 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934554+0000", 2024-09-16T14:49:23.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.172 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935547+0000", 2024-09-16T14:49:23.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00099309300000000001 2024-09-16T14:49:23.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.172 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935565+0000", 2024-09-16T14:49:23.173 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8335999999999999e-05 2024-09-16T14:49:23.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.173 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.173 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935578+0000", 2024-09-16T14:49:23.173 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2831e-05 2024-09-16T14:49:23.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.173 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.173 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935653+0000", 2024-09-16T14:49:23.173 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4411000000000003e-05 2024-09-16T14:49:23.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935663+0000", 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0869999999999999e-05 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942652+0000", 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006988129 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942895+0000", 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00024308899999999999 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.174 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.175 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.175 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s2 61/53 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=61'38, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:23.175 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:23.175 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.166227363000001, 2024-09-16T14:49:23.175 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090892060000000007, 2024-09-16T14:49:23.175 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.175 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.175 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.175 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.175 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:23.175 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.176 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.176 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.176 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:23.176 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.176 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.176 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.176 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937893+0000", 2024-09-16T14:49:23.176 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.176 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.176 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.176 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.177 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937915+0000", 2024-09-16T14:49:23.177 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2566e-05 2024-09-16T14:49:23.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.177 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.177 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937929+0000", 2024-09-16T14:49:23.177 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3314e-05 2024-09-16T14:49:23.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.177 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.177 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937938+0000", 2024-09-16T14:49:23.177 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2050000000000008e-06 2024-09-16T14:49:23.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940150+0000", 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0022119040000000002 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940158+0000", 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2930000000000001e-06 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946972+0000", 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0068139150000000002 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.178 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.179 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.179 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946983+0000", 2024-09-16T14:49:23.179 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0907000000000001e-05 2024-09-16T14:49:23.179 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.179 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.179 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.179 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 61/27 ECSubWrite(tid=226, reqid=osd.11.0:226, at_version=61'82, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:23.179 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:23.179 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.165704748, 2024-09-16T14:49:23.179 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085452879999999998, 2024-09-16T14:49:23.179 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.180 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.180 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.180 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.180 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.180 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:23.180 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.180 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.180 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.180 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:23.180 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.180 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.180 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.181 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:23.181 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.181 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.181 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938432+0000", 2024-09-16T14:49:23.181 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5931e-05 2024-09-16T14:49:23.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.181 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.181 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938444+0000", 2024-09-16T14:49:23.181 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1933000000000001e-05 2024-09-16T14:49:23.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.182 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.182 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938449+0000", 2024-09-16T14:49:23.182 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2070000000000003e-06 2024-09-16T14:49:23.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.182 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.182 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938492+0000", 2024-09-16T14:49:23.182 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2827999999999997e-05 2024-09-16T14:49:23.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.182 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.182 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.182 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938500+0000", 2024-09-16T14:49:23.182 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7600000000000008e-06 2024-09-16T14:49:23.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.183 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.183 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.183 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946932+0000", 2024-09-16T14:49:23.183 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084318099999999997 2024-09-16T14:49:23.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.183 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.183 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.183 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946962+0000", 2024-09-16T14:49:23.183 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9603000000000001e-05 2024-09-16T14:49:23.183 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.183 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.183 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.184 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=238, reqid=client.4500.0:536, at_version=61'23, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:23.184 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:23.184 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.164881776, 2024-09-16T14:49:23.184 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0064046049999999998, 2024-09-16T14:49:23.184 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.184 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.184 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.184 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.184 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:23.184 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.185 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.185 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:23.185 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.185 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.185 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939238+0000", 2024-09-16T14:49:23.185 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.185 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.185 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939537+0000", 2024-09-16T14:49:23.185 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029911299999999999 2024-09-16T14:49:23.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.186 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.186 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939551+0000", 2024-09-16T14:49:23.186 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4003e-05 2024-09-16T14:49:23.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.186 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.186 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939561+0000", 2024-09-16T14:49:23.186 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4269999999999997e-06 2024-09-16T14:49:23.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.186 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.186 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939777+0000", 2024-09-16T14:49:23.186 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00021644599999999999 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939790+0000", 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2381999999999999e-05 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945601+0000", 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0058117139999999999 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945644+0000", 2024-09-16T14:49:23.187 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.265e-05 2024-09-16T14:49:23.188 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.188 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.188 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.188 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 75/65 ECSubWrite(tid=135, reqid=client.4500.0:690, at_version=75'49, trim_to=0'0, roll_forward_to=73'47))", 2024-09-16T14:49:23.188 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:21.922611+0000", 2024-09-16T14:49:23.188 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.1815099019999999, 2024-09-16T14:49:23.188 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090307089999999996, 2024-09-16T14:49:23.188 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.188 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.188 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.189 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.189 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922611+0000", 2024-09-16T14:49:23.189 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.189 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.189 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922611+0000", 2024-09-16T14:49:23.189 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.189 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.189 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922611+0000", 2024-09-16T14:49:23.189 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.190 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.190 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922966+0000", 2024-09-16T14:49:23.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00035514900000000002 2024-09-16T14:49:23.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.190 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.190 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922984+0000", 2024-09-16T14:49:23.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7747000000000001e-05 2024-09-16T14:49:23.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.190 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.190 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922993+0000", 2024-09-16T14:49:23.190 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8850000000000005e-06 2024-09-16T14:49:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.923096+0000", 2024-09-16T14:49:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000103626 2024-09-16T14:49:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.923107+0000", 2024-09-16T14:49:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0370999999999999e-05 2024-09-16T14:49:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.191 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.931610+0000", 2024-09-16T14:49:23.192 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085030420000000006 2024-09-16T14:49:23.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.192 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.192 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.931642+0000", 2024-09-16T14:49:23.192 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.235e-05 2024-09-16T14:49:23.192 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.192 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.192 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.192 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 75/65 ECSubWrite(tid=136, reqid=client.4500.0:691, at_version=75'50, trim_to=0'0, roll_forward_to=73'47))", 2024-09-16T14:49:23.192 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:21.926141+0000", 2024-09-16T14:49:23.192 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 1.1779801219999999, 2024-09-16T14:49:23.193 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0063006299999999998, 2024-09-16T14:49:23.193 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.193 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.193 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.193 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.193 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926141+0000", 2024-09-16T14:49:23.193 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.193 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.193 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926141+0000", 2024-09-16T14:49:23.193 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.194 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.194 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926140+0000", 2024-09-16T14:49:23.194 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.194 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.194 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926912+0000", 2024-09-16T14:49:23.194 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00077166799999999999 2024-09-16T14:49:23.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.194 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.194 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926932+0000", 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0273000000000001e-05 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926943+0000", 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0934000000000001e-05 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.927015+0000", 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2012000000000001e-05 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.195 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.927027+0000", 2024-09-16T14:49:23.196 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2490000000000001e-05 2024-09-16T14:49:23.196 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.196 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.196 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.932418+0000", 2024-09-16T14:49:23.196 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0053903349999999996 2024-09-16T14:49:23.196 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.196 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.196 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.932442+0000", 2024-09-16T14:49:23.196 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4043e-05 2024-09-16T14:49:23.196 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.196 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.197 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.197 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.197 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.197 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:23.198 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:49:23.199 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:23.197+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:23.299 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:23.297+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:23.305 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:23.305 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:23.306 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:23.306 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:23.314 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:49:23.400 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:23.399+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:23.421 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:23.421 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:23.421 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:23.421 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:23.421 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:23.431 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:49:23.501 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:23.500+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:23.540 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:23.540 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:23.540 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:23.540 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:23.540 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.540 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:303 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:23.540 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:23.540 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 38.605149505, 2024-09-16T14:49:23.540 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014763235, 2024-09-16T14:49:23.541 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.541 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:23.541 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:23.541 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:23.541 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:23.541 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 303 2024-09-16T14:49:23.541 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.541 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.541 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.541 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.541 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:23.541 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.542 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.542 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.542 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.542 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:23.542 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.542 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.542 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.542 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.542 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932467+0000", 2024-09-16T14:49:23.542 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:23.542 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.543 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.543 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.543 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932474+0000", 2024-09-16T14:49:23.543 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3189999999999997e-06 2024-09-16T14:49:23.543 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.543 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.543 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.543 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932490+0000", 2024-09-16T14:49:23.544 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:49:23.544 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.544 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.544 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.544 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932498+0000", 2024-09-16T14:49:23.544 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7789999999999996e-06 2024-09-16T14:49:23.544 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.544 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.544 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.544 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932544+0000", 2024-09-16T14:49:23.545 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6486000000000002e-05 2024-09-16T14:49:23.545 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.545 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.545 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:23.545 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932550+0000", 2024-09-16T14:49:23.545 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.3149999999999999e-06 2024-09-16T14:49:23.545 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.545 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.545 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.545 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.939655+0000", 2024-09-16T14:49:23.545 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071050039999999998 2024-09-16T14:49:23.546 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.546 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.546 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:23.546 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.939762+0000", 2024-09-16T14:49:23.546 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000106885 2024-09-16T14:49:23.546 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.546 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.546 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.546 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.947232+0000", 2024-09-16T14:49:23.546 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074701619999999998 2024-09-16T14:49:23.546 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.547 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.547 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.547 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.547 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.547 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:304 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:23.547 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:23.547 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 38.605111764, 2024-09-16T14:49:23.547 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015391726, 2024-09-16T14:49:23.547 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.547 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:23.547 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:23.548 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:23.548 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:23.548 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 304 2024-09-16T14:49:23.548 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.548 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.548 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.548 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.548 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:23.549 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.549 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.549 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.549 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.549 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:23.549 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.549 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.549 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.549 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.549 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:23.549 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.549 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.549 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.550 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.550 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932510+0000", 2024-09-16T14:49:23.550 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.1069999999999998e-06 2024-09-16T14:49:23.550 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.550 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.550 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.550 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932516+0000", 2024-09-16T14:49:23.550 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4699999999999999e-06 2024-09-16T14:49:23.550 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.551 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.551 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.551 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932522+0000", 2024-09-16T14:49:23.551 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4750000000000001e-06 2024-09-16T14:49:23.551 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.551 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.551 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.551 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932610+0000", 2024-09-16T14:49:23.551 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8623999999999997e-05 2024-09-16T14:49:23.552 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.552 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.552 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:23.552 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932616+0000", 2024-09-16T14:49:23.552 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.9649999999999996e-06 2024-09-16T14:49:23.552 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.552 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.552 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.552 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940205+0000", 2024-09-16T14:49:23.552 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0075882570000000002 2024-09-16T14:49:23.552 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.552 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.553 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:23.553 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940285+0000", 2024-09-16T14:49:23.553 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0783000000000002e-05 2024-09-16T14:49:23.553 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.553 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.553 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.553 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.947898+0000", 2024-09-16T14:49:23.553 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0076124820000000003 2024-09-16T14:49:23.553 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.553 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.553 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.554 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.554 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.554 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:305 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:23.554 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:23.554 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 38.605030974000002, 2024-09-16T14:49:23.554 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015660833999999998, 2024-09-16T14:49:23.554 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.554 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:23.554 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:23.554 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:23.554 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:23.555 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 305 2024-09-16T14:49:23.555 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.555 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.555 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.555 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.555 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:23.555 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.555 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.555 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.555 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.555 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:23.555 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.556 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.556 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932586+0000", 2024-09-16T14:49:23.556 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.556 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.556 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932591+0000", 2024-09-16T14:49:23.556 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2470000000000002e-06 2024-09-16T14:49:23.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.557 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.557 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932602+0000", 2024-09-16T14:49:23.557 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1396000000000001e-05 2024-09-16T14:49:23.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.557 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.557 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.557 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932608+0000", 2024-09-16T14:49:23.557 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.7169999999999996e-06 2024-09-16T14:49:23.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.557 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.557 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.557 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932646+0000", 2024-09-16T14:49:23.558 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.7978999999999997e-05 2024-09-16T14:49:23.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.558 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:23.558 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932652+0000", 2024-09-16T14:49:23.558 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0310000000000004e-06 2024-09-16T14:49:23.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.558 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.558 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941247+0000", 2024-09-16T14:49:23.559 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085954180000000005 2024-09-16T14:49:23.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.559 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:23.559 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941327+0000", 2024-09-16T14:49:23.559 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9354e-05 2024-09-16T14:49:23.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.559 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.559 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.948248+0000", 2024-09-16T14:49:23.560 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0069212400000000004 2024-09-16T14:49:23.560 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.560 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.560 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.560 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:403 3.2s0 3:421e4b92:::smithi04040151-11:head [copy-from ver 2] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:23.560 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:23.560 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.654053077, 2024-09-16T14:49:23.560 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015904270000000002, 2024-09-16T14:49:23.560 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.561 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:23.561 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:23.561 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:23.561 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:23.561 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 403 2024-09-16T14:49:23.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.561 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.561 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.561 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:23.562 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.562 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.562 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:23.562 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.562 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.562 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883563+0000", 2024-09-16T14:49:23.563 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:23.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.563 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.563 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.563 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883573+0000", 2024-09-16T14:49:23.563 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0583e-05 2024-09-16T14:49:23.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.563 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.563 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.563 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883593+0000", 2024-09-16T14:49:23.563 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0052e-05 2024-09-16T14:49:23.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.564 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.564 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883603+0000", 2024-09-16T14:49:23.564 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4409999999999999e-06 2024-09-16T14:49:23.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.564 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.564 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883667+0000", 2024-09-16T14:49:23.564 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4671000000000005e-05 2024-09-16T14:49:23.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.565 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:23.565 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883775+0000", 2024-09-16T14:49:23.565 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000108068 2024-09-16T14:49:23.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.565 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.565 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.896809+0000", 2024-09-16T14:49:23.565 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01303355 2024-09-16T14:49:23.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.566 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.566 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897717+0000", 2024-09-16T14:49:23.566 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00090801599999999999 2024-09-16T14:49:23.566 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.566 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.567 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:23.567 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.899408+0000", 2024-09-16T14:49:23.567 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0016911459999999999 2024-09-16T14:49:23.567 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.568 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.568 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.899469+0000", 2024-09-16T14:49:23.568 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0968999999999999e-05 2024-09-16T14:49:23.568 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.569 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.570 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.570 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:404 3.2s0 3:421e4b92:::smithi04040151-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:23.570 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:23.570 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.653999779999999, 2024-09-16T14:49:23.570 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0139831, 2024-09-16T14:49:23.570 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.570 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:23.570 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:23.571 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:23.571 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:23.571 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 404 2024-09-16T14:49:23.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.571 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.571 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.571 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.571 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:23.571 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.571 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.571 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.571 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:23.572 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.572 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.572 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883617+0000", 2024-09-16T14:49:23.572 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.572 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.572 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883622+0000", 2024-09-16T14:49:23.572 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7489999999999998e-06 2024-09-16T14:49:23.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.572 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.573 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883631+0000", 2024-09-16T14:49:23.573 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6489999999999997e-06 2024-09-16T14:49:23.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.573 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.573 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883639+0000", 2024-09-16T14:49:23.573 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2120000000000003e-06 2024-09-16T14:49:23.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.573 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.573 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883930+0000", 2024-09-16T14:49:23.573 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029138200000000002 2024-09-16T14:49:23.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883961+0000", 2024-09-16T14:49:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0190999999999999e-05 2024-09-16T14:49:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897411+0000", 2024-09-16T14:49:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013450037999999999 2024-09-16T14:49:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897506+0000", 2024-09-16T14:49:23.574 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5785000000000005e-05 2024-09-16T14:49:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897601+0000", 2024-09-16T14:49:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4868e-05 2024-09-16T14:49:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:461 3.2s0 3:421e4b92:::smithi04040151-11:head [rollback 10] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.544009737, 2024-09-16T14:49:23.575 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011419016000000001, 2024-09-16T14:49:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 461 2024-09-16T14:49:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.576 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993608+0000", 2024-09-16T14:49:23.577 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993611+0000", 2024-09-16T14:49:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6440000000000002e-06 2024-09-16T14:49:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993619+0000", 2024-09-16T14:49:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.199e-06 2024-09-16T14:49:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.578 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.993624+0000", 2024-09-16T14:49:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.8790000000000002e-06 2024-09-16T14:49:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994734+0000", 2024-09-16T14:49:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0011103339999999999 2024-09-16T14:49:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.994830+0000", 2024-09-16T14:49:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6125000000000005e-05 2024-09-16T14:49:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.579 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995335+0000", 2024-09-16T14:49:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00050461799999999997 2024-09-16T14:49:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.001584+0000", 2024-09-16T14:49:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0062489609999999999 2024-09-16T14:49:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.004966+0000", 2024-09-16T14:49:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.003382674 2024-09-16T14:49:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.580 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.005027+0000", 2024-09-16T14:49:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0974000000000003e-05 2024-09-16T14:49:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:470 3.2s0 3:4ee78709:::smithi04040151-40:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.541755273, 2024-09-16T14:49:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011056359999999999, 2024-09-16T14:49:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.581 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 470 2024-09-16T14:49:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.582 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995863+0000", 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995862+0000", 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995865+0000", 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2440000000000002e-06 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.583 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995872+0000", 2024-09-16T14:49:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5640000000000002e-06 2024-09-16T14:49:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995877+0000", 2024-09-16T14:49:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1680000000000003e-06 2024-09-16T14:49:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.996768+0000", 2024-09-16T14:49:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00089129699999999999 2024-09-16T14:49:23.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.996867+0000", 2024-09-16T14:49:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8259999999999998e-05 2024-09-16T14:49:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.006919+0000", 2024-09-16T14:49:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010052284 2024-09-16T14:49:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.585 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:471 3.2s0 3:4ee78709:::smithi04040151-40:head [read 0~1474560,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.541736546999999, 2024-09-16T14:49:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012013398999999999, 2024-09-16T14:49:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 471 2024-09-16T14:49:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.586 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.587 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.587 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.587 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:23.587 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.587 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.587 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.587 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.587 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:23.587 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.587 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.587 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.588 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.588 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995881+0000", 2024-09-16T14:49:23.588 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.588 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.588 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.588 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995884+0000", 2024-09-16T14:49:23.588 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8600000000000001e-06 2024-09-16T14:49:23.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.588 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.588 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.588 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995891+0000", 2024-09-16T14:49:23.588 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.72e-06 2024-09-16T14:49:23.589 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.589 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.589 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.589 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.995896+0000", 2024-09-16T14:49:23.589 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1889999999999998e-06 2024-09-16T14:49:23.589 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.589 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.589 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.589 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997239+0000", 2024-09-16T14:49:23.589 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0013430460000000001 2024-09-16T14:49:23.589 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.589 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.589 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:23.589 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997352+0000", 2024-09-16T14:49:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011287500000000001 2024-09-16T14:49:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007895+0000", 2024-09-16T14:49:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010543113999999999 2024-09-16T14:49:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s4 52/27 ECSubWrite(tid=183, reqid=osd.11.0:183, at_version=52'60, trim_to=0'0, roll_forward_to=51'56))", 2024-09-16T14:49:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:23.590 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.650347675999999, 2024-09-16T14:49:23.591 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011901715, 2024-09-16T14:49:23.591 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.591 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.591 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.591 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.591 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:23.591 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.591 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.591 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:23.591 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.592 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.592 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887270+0000", 2024-09-16T14:49:23.592 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.592 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.592 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887292+0000", 2024-09-16T14:49:23.592 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1920999999999999e-05 2024-09-16T14:49:23.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.592 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.592 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887303+0000", 2024-09-16T14:49:23.592 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1472e-05 2024-09-16T14:49:23.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.593 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.593 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887311+0000", 2024-09-16T14:49:23.593 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:23.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.593 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.593 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887360+0000", 2024-09-16T14:49:23.593 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9002999999999998e-05 2024-09-16T14:49:23.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.593 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887369+0000", 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9299999999999992e-06 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.899136+0000", 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011767123000000001 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.899172+0000", 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6161000000000002e-05 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.594 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.595 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s4 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:23.595 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.931930+0000", 2024-09-16T14:49:23.595 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.605687879, 2024-09-16T14:49:23.595 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014515299000000001, 2024-09-16T14:49:23.595 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.595 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.595 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.595 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.595 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931930+0000", 2024-09-16T14:49:23.595 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.596 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.596 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931930+0000", 2024-09-16T14:49:23.596 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.596 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.596 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931929+0000", 2024-09-16T14:49:23.596 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.596 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.596 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931950+0000", 2024-09-16T14:49:23.597 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0661000000000001e-05 2024-09-16T14:49:23.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.597 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.597 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931962+0000", 2024-09-16T14:49:23.597 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1949e-05 2024-09-16T14:49:23.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.597 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.598 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931970+0000", 2024-09-16T14:49:23.598 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9510000000000006e-06 2024-09-16T14:49:23.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.598 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.598 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932022+0000", 2024-09-16T14:49:23.598 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1990000000000002e-05 2024-09-16T14:49:23.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.598 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.598 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932032+0000", 2024-09-16T14:49:23.598 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0474e-05 2024-09-16T14:49:23.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.599 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.599 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946420+0000", 2024-09-16T14:49:23.599 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0143874 2024-09-16T14:49:23.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.599 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.599 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946445+0000", 2024-09-16T14:49:23.599 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5627e-05 2024-09-16T14:49:23.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.599 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.600 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds5 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:23.600 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932528+0000", 2024-09-16T14:49:23.600 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.605089612, 2024-09-16T14:49:23.600 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011203032999999999, 2024-09-16T14:49:23.600 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.600 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.600 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.600 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.600 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932528+0000", 2024-09-16T14:49:23.600 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.600 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.601 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.601 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932528+0000", 2024-09-16T14:49:23.601 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.601 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.601 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932527+0000", 2024-09-16T14:49:23.601 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.601 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.601 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932559+0000", 2024-09-16T14:49:23.601 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1553000000000001e-05 2024-09-16T14:49:23.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.602 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.602 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932575+0000", 2024-09-16T14:49:23.602 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6188000000000002e-05 2024-09-16T14:49:23.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.602 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.602 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932600+0000", 2024-09-16T14:49:23.602 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4505e-05 2024-09-16T14:49:23.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.602 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.602 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932684+0000", 2024-09-16T14:49:23.603 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4512999999999995e-05 2024-09-16T14:49:23.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.603 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.603 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932694+0000", 2024-09-16T14:49:23.603 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5510000000000005e-06 2024-09-16T14:49:23.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.603 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.603 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943713+0000", 2024-09-16T14:49:23.603 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011019340000000001 2024-09-16T14:49:23.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.603 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.604 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943731+0000", 2024-09-16T14:49:23.604 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8260000000000001e-05 2024-09-16T14:49:23.604 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.604 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.604 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.604 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.as6 61/53 ECSubWrite(tid=225, reqid=osd.11.0:225, at_version=61'26, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:23.604 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932886+0000", 2024-09-16T14:49:23.604 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.604732386, 2024-09-16T14:49:23.604 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014177248, 2024-09-16T14:49:23.604 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.604 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.605 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.605 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.605 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932886+0000", 2024-09-16T14:49:23.605 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.605 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.605 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932886+0000", 2024-09-16T14:49:23.605 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.605 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.605 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932885+0000", 2024-09-16T14:49:23.606 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.606 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.606 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932916+0000", 2024-09-16T14:49:23.606 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1257999999999998e-05 2024-09-16T14:49:23.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.606 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.606 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932934+0000", 2024-09-16T14:49:23.606 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7958999999999999e-05 2024-09-16T14:49:23.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932942+0000", 2024-09-16T14:49:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9189999999999999e-06 2024-09-16T14:49:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933003+0000", 2024-09-16T14:49:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1301000000000001e-05 2024-09-16T14:49:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933023+0000", 2024-09-16T14:49:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0258000000000002e-05 2024-09-16T14:49:23.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.608 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.608 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946638+0000", 2024-09-16T14:49:23.608 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013614924 2024-09-16T14:49:23.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.608 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.608 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947063+0000", 2024-09-16T14:49:23.608 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00042440400000000002 2024-09-16T14:49:23.608 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.608 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.608 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.609 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs1 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:23.609 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933121+0000", 2024-09-16T14:49:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.604497365, 2024-09-16T14:49:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013854730000000001, 2024-09-16T14:49:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933121+0000", 2024-09-16T14:49:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933121+0000", 2024-09-16T14:49:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933120+0000", 2024-09-16T14:49:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933143+0000", 2024-09-16T14:49:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3048999999999999e-05 2024-09-16T14:49:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933158+0000", 2024-09-16T14:49:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.543e-05 2024-09-16T14:49:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933166+0000", 2024-09-16T14:49:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0839999999999992e-06 2024-09-16T14:49:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933239+0000", 2024-09-16T14:49:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2447999999999995e-05 2024-09-16T14:49:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933250+0000", 2024-09-16T14:49:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1143e-05 2024-09-16T14:49:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946560+0000", 2024-09-16T14:49:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013309642999999999 2024-09-16T14:49:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946975+0000", 2024-09-16T14:49:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00041567600000000002 2024-09-16T14:49:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s1 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933175+0000", 2024-09-16T14:49:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.604443056999999, 2024-09-16T14:49:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014250087999999999, 2024-09-16T14:49:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933175+0000", 2024-09-16T14:49:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933175+0000", 2024-09-16T14:49:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933174+0000", 2024-09-16T14:49:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933196+0000", 2024-09-16T14:49:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1577999999999999e-05 2024-09-16T14:49:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933210+0000", 2024-09-16T14:49:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3864e-05 2024-09-16T14:49:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933217+0000", 2024-09-16T14:49:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1999999999999997e-06 2024-09-16T14:49:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933306+0000", 2024-09-16T14:49:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8985000000000003e-05 2024-09-16T14:49:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933315+0000", 2024-09-16T14:49:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5730000000000008e-06 2024-09-16T14:49:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947397+0000", 2024-09-16T14:49:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014082643000000001 2024-09-16T14:49:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947425+0000", 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7793e-05 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs1 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933463+0000", 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.60415478, 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014016617, 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933463+0000", 2024-09-16T14:49:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933463+0000", 2024-09-16T14:49:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933462+0000", 2024-09-16T14:49:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933487+0000", 2024-09-16T14:49:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4287e-05 2024-09-16T14:49:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933501+0000", 2024-09-16T14:49:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4202999999999999e-05 2024-09-16T14:49:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933509+0000", 2024-09-16T14:49:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5979999999999999e-06 2024-09-16T14:49:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940824+0000", 2024-09-16T14:49:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073150840000000003 2024-09-16T14:49:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940836+0000", 2024-09-16T14:49:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2002e-05 2024-09-16T14:49:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947393+0000", 2024-09-16T14:49:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006557223 2024-09-16T14:49:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947480+0000", 2024-09-16T14:49:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6948999999999997e-05 2024-09-16T14:49:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s1 61/53 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=61'38, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.937911+0000", 2024-09-16T14:49:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.599706898999999, 2024-09-16T14:49:23.626 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012160381, 2024-09-16T14:49:23.626 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.626 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.626 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.626 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.626 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937911+0000", 2024-09-16T14:49:23.626 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.626 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.626 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937911+0000", 2024-09-16T14:49:23.626 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.627 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.627 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937910+0000", 2024-09-16T14:49:23.627 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:23.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.627 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.627 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937945+0000", 2024-09-16T14:49:23.627 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.4999999999999997e-05 2024-09-16T14:49:23.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.627 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.627 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937979+0000", 2024-09-16T14:49:23.628 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3309999999999998e-05 2024-09-16T14:49:23.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.628 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.628 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.628 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937988+0000", 2024-09-16T14:49:23.628 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8509999999999994e-06 2024-09-16T14:49:23.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.628 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.628 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.628 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943764+0000", 2024-09-16T14:49:23.628 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0057755419999999998 2024-09-16T14:49:23.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.628 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.628 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.629 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943771+0000", 2024-09-16T14:49:23.629 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8399999999999997e-06 2024-09-16T14:49:23.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.629 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.629 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.629 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.950049+0000", 2024-09-16T14:49:23.629 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0062787390000000002 2024-09-16T14:49:23.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.629 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.629 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.629 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.950071+0000", 2024-09-16T14:49:23.629 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1940999999999999e-05 2024-09-16T14:49:23.629 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.630 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.630 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.630 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s4 61/27 ECSubWrite(tid=229, reqid=osd.11.0:229, at_version=61'84, trim_to=0'0, roll_forward_to=61'82))", 2024-09-16T14:49:23.630 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.953930+0000", 2024-09-16T14:49:23.630 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.583687514999999, 2024-09-16T14:49:23.630 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012298072, 2024-09-16T14:49:23.630 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.630 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.630 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.630 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.630 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953930+0000", 2024-09-16T14:49:23.631 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.631 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.631 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.631 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953930+0000", 2024-09-16T14:49:23.631 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.631 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.631 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.631 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953930+0000", 2024-09-16T14:49:23.631 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:23.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.631 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.632 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.632 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953948+0000", 2024-09-16T14:49:23.632 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8253999999999998e-05 2024-09-16T14:49:23.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.632 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.632 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.632 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953964+0000", 2024-09-16T14:49:23.632 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5101e-05 2024-09-16T14:49:23.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.632 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.632 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.632 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953971+0000", 2024-09-16T14:49:23.632 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8250000000000005e-06 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954019+0000", 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7250000000000003e-05 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954029+0000", 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0154e-05 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966203+0000", 2024-09-16T14:49:23.633 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01217391 2024-09-16T14:49:23.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.634 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.634 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966228+0000", 2024-09-16T14:49:23.634 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5799000000000002e-05 2024-09-16T14:49:23.634 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.634 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.634 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.634 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es6 61/27 ECSubWrite(tid=230, reqid=osd.11.0:230, at_version=61'35, trim_to=0'0, roll_forward_to=60'33))", 2024-09-16T14:49:23.634 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.953982+0000", 2024-09-16T14:49:23.634 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.583635963000001, 2024-09-16T14:49:23.634 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012264608, 2024-09-16T14:49:23.635 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.635 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.635 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.635 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.635 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953982+0000", 2024-09-16T14:49:23.635 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.635 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.635 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953982+0000", 2024-09-16T14:49:23.635 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953982+0000", 2024-09-16T14:49:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954000+0000", 2024-09-16T14:49:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8175000000000001e-05 2024-09-16T14:49:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954009+0000", 2024-09-16T14:49:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2280000000000004e-06 2024-09-16T14:49:23.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954016+0000", 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6839999999999999e-06 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954084+0000", 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8365e-05 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954094+0000", 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0147e-05 2024-09-16T14:49:23.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.638 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.638 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.638 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966228+0000", 2024-09-16T14:49:23.638 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012133961 2024-09-16T14:49:23.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.638 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.638 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.638 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966247+0000", 2024-09-16T14:49:23.638 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8386e-05 2024-09-16T14:49:23.638 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.638 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.638 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.638 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.639 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es6 61/27 ECSubWrite(tid=231, reqid=osd.11.0:231, at_version=61'37, trim_to=0'0, roll_forward_to=60'33))", 2024-09-16T14:49:23.639 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.954093+0000", 2024-09-16T14:49:23.639 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.583525064, 2024-09-16T14:49:23.639 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012660772000000001, 2024-09-16T14:49:23.639 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.639 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:23.639 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.639 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.639 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954093+0000", 2024-09-16T14:49:23.639 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.639 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.640 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954093+0000", 2024-09-16T14:49:23.640 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.640 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.640 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954092+0000", 2024-09-16T14:49:23.640 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.640 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.640 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954118+0000", 2024-09-16T14:49:23.640 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5442000000000001e-05 2024-09-16T14:49:23.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.641 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.641 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954127+0000", 2024-09-16T14:49:23.641 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9560000000000003e-06 2024-09-16T14:49:23.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.641 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.641 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954133+0000", 2024-09-16T14:49:23.641 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5810000000000002e-06 2024-09-16T14:49:23.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.641 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.641 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954527+0000", 2024-09-16T14:49:23.641 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000393161 2024-09-16T14:49:23.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.642 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:23.642 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954536+0000", 2024-09-16T14:49:23.642 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5699999999999999e-06 2024-09-16T14:49:23.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.642 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:23.642 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966735+0000", 2024-09-16T14:49:23.642 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012198413999999999 2024-09-16T14:49:23.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.642 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.642 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966754+0000", 2024-09-16T14:49:23.642 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9082000000000001e-05 2024-09-16T14:49:23.643 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.643 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.643 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.643 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.643 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:700 3.7s0 3:efaac682:::smithi04040151-34:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e76)", 2024-09-16T14:49:23.643 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:22.906313+0000", 2024-09-16T14:49:23.643 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.63130492900000001, 2024-09-16T14:49:23.643 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011847066999999999, 2024-09-16T14:49:23.643 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:23.643 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:23.643 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:23.643 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:23.643 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:23.644 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 700 2024-09-16T14:49:23.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.644 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:23.644 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.644 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:23.644 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:22.906313+0000", 2024-09-16T14:49:23.644 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.644 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.644 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:23.644 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:22.906313+0000", 2024-09-16T14:49:23.644 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:23.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.645 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:23.645 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:22.906312+0000", 2024-09-16T14:49:23.645 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:23.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.645 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:23.645 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:22.906315+0000", 2024-09-16T14:49:23.645 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9799999999999998e-06 2024-09-16T14:49:23.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.645 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:23.645 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:22.906322+0000", 2024-09-16T14:49:23.645 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8109999999999997e-06 2024-09-16T14:49:23.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.646 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:23.646 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:22.906328+0000", 2024-09-16T14:49:23.646 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.7989999999999999e-06 2024-09-16T14:49:23.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.646 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.646 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:22.911033+0000", 2024-09-16T14:49:23.646 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0047050089999999996 2024-09-16T14:49:23.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.646 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:23.646 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:22.911134+0000", 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010088400000000001 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:22.916988+0000", 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0058544549999999997 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:22.917083+0000", 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5005000000000003e-05 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:23.647 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:22.918160+0000", 2024-09-16T14:49:23.648 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001076661 2024-09-16T14:49:23.648 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.648 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.648 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.648 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:23.648 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:23.648 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:23.649 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:49:23.650 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:23.648+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:23.750 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:23.748+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:23.766 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:23.766 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:23.766 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:23.766 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:23.775 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:49:23.851 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:23.850+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:23.900 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T14:49:23.901 INFO:tasks.rados.rados.0.smithi040.stdout:290: done (1 left) 2024-09-16T14:49:23.901 INFO:tasks.rados.rados.0.smithi040.stdout:291: copy_from oid 20 from oid 48 current snap is 26 2024-09-16T14:49:23.901 INFO:tasks.rados.rados.0.smithi040.stdout:289: finishing write tid 1 to smithi04040151-34 2024-09-16T14:49:23.901 INFO:tasks.rados.rados.0.smithi040.stdout:289: finishing write tid 2 to smithi04040151-34 2024-09-16T14:49:23.901 INFO:tasks.rados.rados.0.smithi040.stdout:292: read oid 26 snap 24 2024-09-16T14:49:23.901 INFO:tasks.rados.rados.0.smithi040.stdout:292: expect (ObjNum 87 snap 14 seq_num 87) 2024-09-16T14:49:23.901 INFO:tasks.rados.rados.0.smithi040.stdout:293: read oid 29 snap -1 2024-09-16T14:49:23.901 INFO:tasks.rados.rados.0.smithi040.stdout:293: expect (ObjNum 28 snap 0 seq_num 28) 2024-09-16T14:49:23.901 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 11 current snap is 26 2024-09-16T14:49:23.901 INFO:tasks.rados.rados.0.smithi040.stdout:294: seq_num 114 ranges {0=753664,753664=688128,1441792=475136} 2024-09-16T14:49:23.905 INFO:tasks.rados.rados.0.smithi040.stdout:294: writing smithi04040151-11 from 0 to 753664 tid 1 2024-09-16T14:49:23.910 INFO:tasks.rados.rados.0.smithi040.stdout:294: writing smithi04040151-11 from 753664 to 1441792 tid 2 2024-09-16T14:49:23.917 INFO:tasks.rados.rados.0.smithi040.stdout:294: writing smithi04040151-11 from 1441792 to 1916928 tid 3 2024-09-16T14:49:23.918 INFO:tasks.rados.rados.0.smithi040.stdout:295: read oid 31 snap 18 2024-09-16T14:49:23.918 INFO:tasks.rados.rados.0.smithi040.stdout:295: expect (ObjNum 97 snap 18 seq_num 97) 2024-09-16T14:49:23.918 INFO:tasks.rados.rados.0.smithi040.stdout:296: snap_create 2024-09-16T14:49:23.918 INFO:tasks.rados.rados.0.smithi040.stdout:289: finishing write tid 3 to smithi04040151-34 2024-09-16T14:49:23.918 INFO:tasks.rados.rados.0.smithi040.stdout:289: finishing write tid 4 to smithi04040151-34 2024-09-16T14:49:23.918 INFO:tasks.rados.rados.0.smithi040.stdout:289: finishing write tid 5 to smithi04040151-34 2024-09-16T14:49:23.918 INFO:tasks.rados.rados.0.smithi040.stdout:289: oid 34 updating version 0 to 95 2024-09-16T14:49:23.918 INFO:tasks.rados.rados.0.smithi040.stdout:289: oid 34 updating version 95 to 96 2024-09-16T14:49:23.918 INFO:tasks.rados.rados.0.smithi040.stdout:289: oid 34 updating version 96 to 97 2024-09-16T14:49:23.918 INFO:tasks.rados.rados.0.smithi040.stdout:289: oid 34 updating version 97 to 98 2024-09-16T14:49:23.918 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 34 v 98 (ObjNum 113 snap 26 seq_num 113) dirty exists 2024-09-16T14:49:23.919 INFO:tasks.rados.rados.0.smithi040.stdout:289: left oid 34 (ObjNum 113 snap 26 seq_num 113) 2024-09-16T14:49:23.919 INFO:tasks.rados.rados.0.smithi040.stdout:293: expect (ObjNum 28 snap 0 seq_num 28) 2024-09-16T14:49:23.932 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-09-16T14:49:23.933 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:23.933 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:23.933 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:23.933 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:23.933 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:23.934 INFO:tasks.rados.rados.0.smithi040.stdout:291: finishing copy_from racing read to smithi04040151-20 2024-09-16T14:49:23.934 INFO:tasks.rados.rados.0.smithi040.stdout:292: expect (ObjNum 87 snap 14 seq_num 87) 2024-09-16T14:49:23.942 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:49:23.978 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:23.977+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:24.009 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:49:24.031 INFO:tasks.rados.rados.0.smithi040.stdout:294: finishing write tid 1 to smithi04040151-11 2024-09-16T14:49:24.031 INFO:tasks.rados.rados.0.smithi040.stdout:294: finishing write tid 2 to smithi04040151-11 2024-09-16T14:49:24.031 INFO:tasks.rados.rados.0.smithi040.stdout:294: finishing write tid 3 to smithi04040151-11 2024-09-16T14:49:24.031 INFO:tasks.rados.rados.0.smithi040.stdout:294: finishing write tid 4 to smithi04040151-11 2024-09-16T14:49:24.032 INFO:tasks.rados.rados.0.smithi040.stdout:291: finishing copy_from to smithi04040151-20 2024-09-16T14:49:24.032 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 20 v 4 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-16T14:49:24.032 INFO:tasks.rados.rados.0.smithi040.stdout:294: finishing write tid 5 to smithi04040151-11 2024-09-16T14:49:24.032 INFO:tasks.rados.rados.0.smithi040.stdout:294: oid 11 updating version 0 to 98 2024-09-16T14:49:24.032 INFO:tasks.rados.rados.0.smithi040.stdout:294: oid 11 updating version 98 to 100 2024-09-16T14:49:24.032 INFO:tasks.rados.rados.0.smithi040.stdout:294: oid 11 version 100 is already newer than 97 2024-09-16T14:49:24.032 INFO:tasks.rados.rados.0.smithi040.stdout:294: oid 11 version 100 is already newer than 99 2024-09-16T14:49:24.032 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 11 v 100 (ObjNum 114 snap 26 seq_num 114) dirty exists 2024-09-16T14:49:24.032 INFO:tasks.rados.rados.0.smithi040.stdout:294: left oid 11 (ObjNum 114 snap 26 seq_num 114) 2024-09-16T14:49:24.032 INFO:tasks.rados.rados.0.smithi040.stdout:295: expect (ObjNum 97 snap 18 seq_num 97) 2024-09-16T14:49:24.047 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:24.047 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:24.047 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:24.047 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:24.047 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.047 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs2 37/27 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:24.047 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:24.047 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.124159935000002, 2024-09-16T14:49:24.047 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010231874, 2024-09-16T14:49:24.048 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.048 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:24.048 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.048 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.048 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.048 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:24.048 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.048 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.048 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.049 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:24.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.049 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.049 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.049 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:24.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:24.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.049 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.049 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.049 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920725+0000", 2024-09-16T14:49:24.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2198999999999999e-05 2024-09-16T14:49:24.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.050 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.050 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.050 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920735+0000", 2024-09-16T14:49:24.050 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0122000000000001e-05 2024-09-16T14:49:24.050 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.050 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.050 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.050 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920742+0000", 2024-09-16T14:49:24.050 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7179999999999999e-06 2024-09-16T14:49:24.050 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.050 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.050 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.050 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921079+0000", 2024-09-16T14:49:24.050 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00033676000000000002 2024-09-16T14:49:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921088+0000", 2024-09-16T14:49:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9290000000000004e-06 2024-09-16T14:49:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930916+0000", 2024-09-16T14:49:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0098279749999999992 2024-09-16T14:49:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.051 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930935+0000", 2024-09-16T14:49:24.052 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9510000000000001e-05 2024-09-16T14:49:24.052 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.052 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.052 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.052 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.052 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.fs7 37/27 ECSubWrite(tid=189, reqid=osd.7.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'12))", 2024-09-16T14:49:24.052 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:24.052 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.123972236, 2024-09-16T14:49:24.052 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010007947999999999, 2024-09-16T14:49:24.052 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.052 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:24.053 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.053 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.053 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.054 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:24.054 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.054 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.054 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.054 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:24.054 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.054 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.054 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.054 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920890+0000", 2024-09-16T14:49:24.054 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:24.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.054 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.055 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.055 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920900+0000", 2024-09-16T14:49:24.055 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9839999999999996e-06 2024-09-16T14:49:24.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.055 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.055 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.055 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920910+0000", 2024-09-16T14:49:24.055 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4220000000000006e-06 2024-09-16T14:49:24.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.055 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.055 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.055 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920917+0000", 2024-09-16T14:49:24.055 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5349999999999999e-06 2024-09-16T14:49:24.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.056 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.056 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.056 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920967+0000", 2024-09-16T14:49:24.056 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9348000000000003e-05 2024-09-16T14:49:24.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.056 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.056 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.056 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920976+0000", 2024-09-16T14:49:24.056 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8810000000000002e-06 2024-09-16T14:49:24.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.056 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.056 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.056 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930871+0000", 2024-09-16T14:49:24.057 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0098950500000000007 2024-09-16T14:49:24.057 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.057 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.057 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.057 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930899+0000", 2024-09-16T14:49:24.057 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.847e-05 2024-09-16T14:49:24.057 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.057 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.057 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.057 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.057 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.057 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s3 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:24.057 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:24.058 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.122975193999999, 2024-09-16T14:49:24.058 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011669621999999999, 2024-09-16T14:49:24.058 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.058 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:24.058 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.058 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.058 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.058 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:24.058 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.058 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.058 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.058 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.058 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:24.058 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.059 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.059 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.059 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921887+0000", 2024-09-16T14:49:24.059 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:24.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.059 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.059 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.059 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921911+0000", 2024-09-16T14:49:24.059 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4012999999999999e-05 2024-09-16T14:49:24.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.059 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.059 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.059 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921923+0000", 2024-09-16T14:49:24.060 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1642e-05 2024-09-16T14:49:24.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.060 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.060 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.060 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921932+0000", 2024-09-16T14:49:24.060 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0210000000000005e-06 2024-09-16T14:49:24.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.060 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.060 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.060 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921988+0000", 2024-09-16T14:49:24.061 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5822999999999997e-05 2024-09-16T14:49:24.061 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.061 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.061 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.061 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921997+0000", 2024-09-16T14:49:24.061 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7900000000000005e-06 2024-09-16T14:49:24.061 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.061 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.061 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.062 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933539+0000", 2024-09-16T14:49:24.062 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011542248 2024-09-16T14:49:24.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.062 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.062 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.062 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933558+0000", 2024-09-16T14:49:24.062 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8964999999999999e-05 2024-09-16T14:49:24.062 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.062 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.062 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.062 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.062 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds6 37/27 ECSubWrite(tid=190, reqid=osd.7.0:188, at_version=0'0, trim_to=0'0, roll_forward_to=37'20))", 2024-09-16T14:49:24.062 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:24.063 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.122893593999997, 2024-09-16T14:49:24.063 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0090799120000000007, 2024-09-16T14:49:24.063 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.063 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:24.063 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.063 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.063 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.063 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:24.063 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.063 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.063 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.063 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.063 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:24.064 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.064 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.064 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.064 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921969+0000", 2024-09-16T14:49:24.064 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:24.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.064 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.064 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.064 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921983+0000", 2024-09-16T14:49:24.064 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4403000000000001e-05 2024-09-16T14:49:24.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.064 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.064 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.065 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921996+0000", 2024-09-16T14:49:24.065 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.259e-05 2024-09-16T14:49:24.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.065 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.065 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.065 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922005+0000", 2024-09-16T14:49:24.065 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9080000000000001e-06 2024-09-16T14:49:24.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.065 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.065 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.065 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922073+0000", 2024-09-16T14:49:24.065 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8127999999999996e-05 2024-09-16T14:49:24.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.065 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.066 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.066 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922085+0000", 2024-09-16T14:49:24.066 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1712999999999999e-05 2024-09-16T14:49:24.066 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.066 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.066 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.066 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931031+0000", 2024-09-16T14:49:24.066 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089464669999999996 2024-09-16T14:49:24.066 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.066 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.066 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.066 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931050+0000", 2024-09-16T14:49:24.066 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8471999999999999e-05 2024-09-16T14:49:24.066 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.067 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.067 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.067 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.067 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.067 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.6s2 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:49:24.067 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.923513+0000", 2024-09-16T14:49:24.067 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.121350638999999, 2024-09-16T14:49:24.067 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0075220499999999997, 2024-09-16T14:49:24.067 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.067 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:24.067 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.067 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.067 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.067 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923513+0000", 2024-09-16T14:49:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923513+0000", 2024-09-16T14:49:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923512+0000", 2024-09-16T14:49:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.069 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.069 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923534+0000", 2024-09-16T14:49:24.069 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1892999999999999e-05 2024-09-16T14:49:24.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.069 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.069 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.069 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923551+0000", 2024-09-16T14:49:24.069 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7127999999999999e-05 2024-09-16T14:49:24.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.069 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.069 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.069 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923559+0000", 2024-09-16T14:49:24.069 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6400000000000003e-06 2024-09-16T14:49:24.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.070 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.070 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.070 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923615+0000", 2024-09-16T14:49:24.070 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5560000000000003e-05 2024-09-16T14:49:24.070 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.070 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.070 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.070 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.923627+0000", 2024-09-16T14:49:24.070 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2296999999999999e-05 2024-09-16T14:49:24.070 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.070 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.070 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.070 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931010+0000", 2024-09-16T14:49:24.071 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.007382258 2024-09-16T14:49:24.071 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.071 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.071 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.071 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931035+0000", 2024-09-16T14:49:24.071 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5194e-05 2024-09-16T14:49:24.071 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.071 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.071 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.071 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.071 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.071 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as1 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:24.071 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:24.071 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.120798344999997, 2024-09-16T14:49:24.072 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0095052339999999996, 2024-09-16T14:49:24.072 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.072 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:24.072 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.072 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.072 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.072 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:24.072 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.072 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.072 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.072 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:24.072 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.073 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.073 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.073 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:24.073 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:24.073 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.073 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.073 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.073 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924089+0000", 2024-09-16T14:49:24.073 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3768999999999999e-05 2024-09-16T14:49:24.073 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.073 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.073 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.073 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924101+0000", 2024-09-16T14:49:24.073 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2294e-05 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924109+0000", 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7389999999999999e-06 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924163+0000", 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.4052999999999999e-05 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924172+0000", 2024-09-16T14:49:24.074 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7189999999999996e-06 2024-09-16T14:49:24.075 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.075 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.075 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.075 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933561+0000", 2024-09-16T14:49:24.075 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0093884659999999998 2024-09-16T14:49:24.075 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.075 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.075 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.075 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933570+0000", 2024-09-16T14:49:24.075 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4120000000000007e-06 2024-09-16T14:49:24.075 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.075 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.075 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.075 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.076 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.076 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.es3 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:24.076 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:24.076 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 40.120740910000002, 2024-09-16T14:49:24.076 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0094028089999999998, 2024-09-16T14:49:24.076 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.076 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:24.076 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.076 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.076 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.076 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:24.076 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.076 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.076 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.077 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.077 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:24.077 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.077 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.077 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.077 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.077 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:24.077 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:24.077 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.078 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.078 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.078 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924142+0000", 2024-09-16T14:49:24.078 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0162000000000001e-05 2024-09-16T14:49:24.078 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.078 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.078 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.078 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924150+0000", 2024-09-16T14:49:24.078 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0919999999999998e-06 2024-09-16T14:49:24.078 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.078 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.078 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.078 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924157+0000", 2024-09-16T14:49:24.078 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6069999999999996e-06 2024-09-16T14:49:24.079 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.079 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.079 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.079 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924215+0000", 2024-09-16T14:49:24.079 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8192000000000001e-05 2024-09-16T14:49:24.079 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.079 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.079 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.079 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924225+0000", 2024-09-16T14:49:24.079 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9760000000000007e-06 2024-09-16T14:49:24.079 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.079 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.079 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.079 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933494+0000", 2024-09-16T14:49:24.080 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0092692720000000003 2024-09-16T14:49:24.080 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.080 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.080 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.080 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933525+0000", 2024-09-16T14:49:24.080 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0976999999999998e-05 2024-09-16T14:49:24.080 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.080 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.080 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.080 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.080 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.080 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s3 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:24.080 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932044+0000", 2024-09-16T14:49:24.081 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.112819389999999, 2024-09-16T14:49:24.081 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0086059629999999995, 2024-09-16T14:49:24.081 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.081 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:24.081 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.081 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.081 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.081 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932044+0000", 2024-09-16T14:49:24.081 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.081 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.081 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.081 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932044+0000", 2024-09-16T14:49:24.081 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.082 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.082 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.082 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932043+0000", 2024-09-16T14:49:24.082 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:24.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.082 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.082 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.082 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932067+0000", 2024-09-16T14:49:24.082 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3501000000000001e-05 2024-09-16T14:49:24.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.082 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.082 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.083 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932078+0000", 2024-09-16T14:49:24.083 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1151999999999999e-05 2024-09-16T14:49:24.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.083 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.083 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.083 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932085+0000", 2024-09-16T14:49:24.083 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4019999999999996e-06 2024-09-16T14:49:24.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.083 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.083 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.083 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932134+0000", 2024-09-16T14:49:24.083 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8698e-05 2024-09-16T14:49:24.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.083 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.084 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.084 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932144+0000", 2024-09-16T14:49:24.084 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7440000000000002e-06 2024-09-16T14:49:24.084 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.084 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.084 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.084 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.940624+0000", 2024-09-16T14:49:24.084 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084803870000000007 2024-09-16T14:49:24.084 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.084 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.084 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.084 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.940650+0000", 2024-09-16T14:49:24.084 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5769999999999999e-05 2024-09-16T14:49:24.084 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.085 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.085 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.085 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.085 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.085 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds6 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:24.086 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932207+0000", 2024-09-16T14:49:24.086 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.112656329, 2024-09-16T14:49:24.086 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0085983610000000005, 2024-09-16T14:49:24.086 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.086 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:24.086 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.086 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.086 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.086 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932207+0000", 2024-09-16T14:49:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932207+0000", 2024-09-16T14:49:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932206+0000", 2024-09-16T14:49:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.088 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932229+0000", 2024-09-16T14:49:24.088 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3074e-05 2024-09-16T14:49:24.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.088 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.088 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.088 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932241+0000", 2024-09-16T14:49:24.088 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1505000000000001e-05 2024-09-16T14:49:24.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.088 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.088 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.088 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932249+0000", 2024-09-16T14:49:24.088 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7400000000000004e-06 2024-09-16T14:49:24.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.088 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.089 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.089 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932295+0000", 2024-09-16T14:49:24.089 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5908999999999997e-05 2024-09-16T14:49:24.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.089 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.089 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.089 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932304+0000", 2024-09-16T14:49:24.089 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1249999999999999e-06 2024-09-16T14:49:24.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.089 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.089 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.089 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.940790+0000", 2024-09-16T14:49:24.089 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084867380000000006 2024-09-16T14:49:24.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.090 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.090 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.090 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.940805+0000", 2024-09-16T14:49:24.090 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4934e-05 2024-09-16T14:49:24.090 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.090 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.090 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.090 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.090 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.090 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as1 61/53 ECSubWrite(tid=225, reqid=osd.11.0:225, at_version=61'26, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:24.090 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932623+0000", 2024-09-16T14:49:24.090 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.112240530000001, 2024-09-16T14:49:24.090 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089969119999999993, 2024-09-16T14:49:24.090 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.091 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:24.091 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.091 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.091 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.091 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932623+0000", 2024-09-16T14:49:24.091 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.091 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.091 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.091 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.091 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932623+0000", 2024-09-16T14:49:24.091 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.091 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.091 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.091 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.092 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932622+0000", 2024-09-16T14:49:24.092 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:24.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.092 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.092 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.092 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932643+0000", 2024-09-16T14:49:24.092 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0611e-05 2024-09-16T14:49:24.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.092 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.092 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.092 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932656+0000", 2024-09-16T14:49:24.092 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3613e-05 2024-09-16T14:49:24.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.093 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.093 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.093 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932663+0000", 2024-09-16T14:49:24.093 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0149999999999998e-06 2024-09-16T14:49:24.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.093 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.093 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.093 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932790+0000", 2024-09-16T14:49:24.093 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00012615099999999999 2024-09-16T14:49:24.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.093 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.093 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.093 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932803+0000", 2024-09-16T14:49:24.093 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3355e-05 2024-09-16T14:49:24.094 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.094 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.094 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.094 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941602+0000", 2024-09-16T14:49:24.094 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0087990859999999994 2024-09-16T14:49:24.094 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.094 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.094 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.094 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941620+0000", 2024-09-16T14:49:24.094 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7710000000000002e-05 2024-09-16T14:49:24.094 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.094 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.094 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.094 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.cs2 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932695+0000", 2024-09-16T14:49:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.112167900999999, 2024-09-16T14:49:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089530259999999993, 2024-09-16T14:49:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932695+0000", 2024-09-16T14:49:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.095 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.096 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.096 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932695+0000", 2024-09-16T14:49:24.096 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.096 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.096 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.096 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.096 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932695+0000", 2024-09-16T14:49:24.096 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:24.096 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.096 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.096 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.096 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932711+0000", 2024-09-16T14:49:24.096 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6248000000000001e-05 2024-09-16T14:49:24.097 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.097 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.097 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.097 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932721+0000", 2024-09-16T14:49:24.097 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7489999999999993e-06 2024-09-16T14:49:24.097 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.097 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.097 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.097 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932728+0000", 2024-09-16T14:49:24.097 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:49:24.097 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.097 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.097 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.097 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932815+0000", 2024-09-16T14:49:24.098 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6984999999999995e-05 2024-09-16T14:49:24.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.098 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.098 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.098 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932825+0000", 2024-09-16T14:49:24.098 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0459999999999994e-06 2024-09-16T14:49:24.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.098 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.098 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.098 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941636+0000", 2024-09-16T14:49:24.098 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0088109880000000005 2024-09-16T14:49:24.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.098 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.098 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.099 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941649+0000", 2024-09-16T14:49:24.099 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2989e-05 2024-09-16T14:49:24.099 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.099 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.099 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.099 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.099 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.5s6 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:24.099 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932903+0000", 2024-09-16T14:49:24.099 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.111960029999999, 2024-09-16T14:49:24.099 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0087662160000000003, 2024-09-16T14:49:24.099 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.099 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:24.099 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.100 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.100 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.100 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932903+0000", 2024-09-16T14:49:24.100 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.100 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.100 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.100 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932903+0000", 2024-09-16T14:49:24.100 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.100 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.100 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.100 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932903+0000", 2024-09-16T14:49:24.100 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:24.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.101 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.101 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.101 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932917+0000", 2024-09-16T14:49:24.101 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4443e-05 2024-09-16T14:49:24.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.101 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.101 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.101 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932927+0000", 2024-09-16T14:49:24.101 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8910000000000007e-06 2024-09-16T14:49:24.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.101 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.101 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932934+0000", 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.985e-06 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932978+0000", 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3730000000000003e-05 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932987+0000", 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3940000000000002e-06 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.102 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.103 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941659+0000", 2024-09-16T14:49:24.103 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0086717840000000001 2024-09-16T14:49:24.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.103 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.103 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.103 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941670+0000", 2024-09-16T14:49:24.103 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0654e-05 2024-09-16T14:49:24.103 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.103 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.103 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.103 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.103 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.cs2 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:24.103 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.933500+0000", 2024-09-16T14:49:24.104 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.111363179000001, 2024-09-16T14:49:24.104 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0087068370000000003, 2024-09-16T14:49:24.104 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.104 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:24.104 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.104 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.104 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.104 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933500+0000", 2024-09-16T14:49:24.104 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.104 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.104 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.104 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933500+0000", 2024-09-16T14:49:24.104 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933499+0000", 2024-09-16T14:49:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933519+0000", 2024-09-16T14:49:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9932000000000002e-05 2024-09-16T14:49:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.105 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933541+0000", 2024-09-16T14:49:24.106 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1698000000000001e-05 2024-09-16T14:49:24.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.106 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.106 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.106 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933554+0000", 2024-09-16T14:49:24.106 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2995e-05 2024-09-16T14:49:24.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.106 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.106 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.106 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935629+0000", 2024-09-16T14:49:24.106 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002075061 2024-09-16T14:49:24.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.106 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.106 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.107 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935637+0000", 2024-09-16T14:49:24.107 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4079999999999998e-06 2024-09-16T14:49:24.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.107 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.107 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.107 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942191+0000", 2024-09-16T14:49:24.107 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0065539630000000003 2024-09-16T14:49:24.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.107 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.107 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.107 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942207+0000", 2024-09-16T14:49:24.107 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5622999999999999e-05 2024-09-16T14:49:24.107 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.107 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.108 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.108 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.108 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:594 3.3s0 3:c9d06351:::smithi04040151-33:head [append 0~491520] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:24.108 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.981536+0000", 2024-09-16T14:49:24.108 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 12.063327138, 2024-09-16T14:49:24.108 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0295208730000001, 2024-09-16T14:49:24.108 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.108 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:24.108 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:24.108 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:24.108 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:24.108 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 594 2024-09-16T14:49:24.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.109 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.109 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.109 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.109 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.981536+0000", 2024-09-16T14:49:24.109 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.109 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.109 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.109 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.981536+0000", 2024-09-16T14:49:24.109 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.109 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.109 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.109 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.981535+0000", 2024-09-16T14:49:24.110 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:24.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.110 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.110 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.110 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982118+0000", 2024-09-16T14:49:24.110 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00058284500000000002 2024-09-16T14:49:24.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.110 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.110 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.110 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982129+0000", 2024-09-16T14:49:24.110 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1644999999999999e-05 2024-09-16T14:49:24.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.110 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.110 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.111 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982134+0000", 2024-09-16T14:49:24.111 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.152e-06 2024-09-16T14:49:24.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.111 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.111 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.111 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982188+0000", 2024-09-16T14:49:24.111 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3829999999999998e-05 2024-09-16T14:49:24.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.111 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.111 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:24.111 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982192+0000", 2024-09-16T14:49:24.111 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5980000000000001e-06 2024-09-16T14:49:24.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.111 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.112 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.112 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980648+0000", 2024-09-16T14:49:24.112 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99845620300000004 2024-09-16T14:49:24.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.112 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.112 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:24.112 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980918+0000", 2024-09-16T14:49:24.112 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00026981999999999999 2024-09-16T14:49:24.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.112 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.112 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.112 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.002491+0000", 2024-09-16T14:49:24.112 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.021573093000000002 2024-09-16T14:49:24.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.113 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.113 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:24.113 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.002715+0000", 2024-09-16T14:49:24.113 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00022431799999999999 2024-09-16T14:49:24.113 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.113 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.113 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.113 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004518+0000", 2024-09-16T14:49:24.113 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001802339 2024-09-16T14:49:24.113 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.113 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.113 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.113 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009950+0000", 2024-09-16T14:49:24.113 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0054329260000000002 2024-09-16T14:49:24.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.114 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.114 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:24.114 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.010633+0000", 2024-09-16T14:49:24.114 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00068230700000000005 2024-09-16T14:49:24.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.114 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.114 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.114 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.011057+0000", 2024-09-16T14:49:24.114 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000424313 2024-09-16T14:49:24.114 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.114 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.114 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.115 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.115 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:595 3.3s0 3:c9d06351:::smithi04040151-33:head [append 0~589824] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:24.115 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.985758+0000", 2024-09-16T14:49:24.115 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 12.05910521, 2024-09-16T14:49:24.115 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0310086730000001, 2024-09-16T14:49:24.115 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.115 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:24.115 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:24.115 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:24.115 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:24.115 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 595 2024-09-16T14:49:24.115 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.115 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.115 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.985758+0000", 2024-09-16T14:49:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.985758+0000", 2024-09-16T14:49:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.985757+0000", 2024-09-16T14:49:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:24.116 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986224+0000", 2024-09-16T14:49:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00046679099999999998 2024-09-16T14:49:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986234+0000", 2024-09-16T14:49:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0614999999999999e-05 2024-09-16T14:49:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986239+0000", 2024-09-16T14:49:24.117 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0000000000000004e-06 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986272+0000", 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2861999999999998e-05 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986274+0000", 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.013e-06 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.981881+0000", 2024-09-16T14:49:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99560661699999997 2024-09-16T14:49:24.119 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.119 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.119 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:24.119 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.981947+0000", 2024-09-16T14:49:24.119 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6390000000000006e-05 2024-09-16T14:49:24.119 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.119 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.119 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.119 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005200+0000", 2024-09-16T14:49:24.119 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.023253165999999999 2024-09-16T14:49:24.119 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.119 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.119 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:24.119 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005354+0000", 2024-09-16T14:49:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000153611 2024-09-16T14:49:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007291+0000", 2024-09-16T14:49:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001936716 2024-09-16T14:49:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.120 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009953+0000", 2024-09-16T14:49:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0026618190000000002 2024-09-16T14:49:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.016469+0000", 2024-09-16T14:49:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0065166629999999998 2024-09-16T14:49:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.016767+0000", 2024-09-16T14:49:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00029760600000000001 2024-09-16T14:49:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.121 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:596 3.3s0 3:c9d06351:::smithi04040151-33:head [append 0~557056] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.989693+0000", 2024-09-16T14:49:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 12.055170655, 2024-09-16T14:49:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.027510892, 2024-09-16T14:49:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 596 2024-09-16T14:49:24.122 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.989693+0000", 2024-09-16T14:49:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.989693+0000", 2024-09-16T14:49:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.123 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.989692+0000", 2024-09-16T14:49:24.124 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:24.124 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.124 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.124 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.124 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990180+0000", 2024-09-16T14:49:24.124 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00048803300000000002 2024-09-16T14:49:24.124 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.124 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.124 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.124 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990189+0000", 2024-09-16T14:49:24.124 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0440000000000001e-06 2024-09-16T14:49:24.124 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.124 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.124 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990194+0000", 2024-09-16T14:49:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8369999999999996e-06 2024-09-16T14:49:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990239+0000", 2024-09-16T14:49:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4898999999999998e-05 2024-09-16T14:49:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990241+0000", 2024-09-16T14:49:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:49:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.126 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.126 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.981982+0000", 2024-09-16T14:49:24.126 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99174135900000004 2024-09-16T14:49:24.126 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.126 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.126 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:24.126 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982045+0000", 2024-09-16T14:49:24.126 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2890000000000003e-05 2024-09-16T14:49:24.126 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.126 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.126 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.126 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007979+0000", 2024-09-16T14:49:24.126 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.025933905 2024-09-16T14:49:24.126 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.127 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.127 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:24.127 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.008137+0000", 2024-09-16T14:49:24.127 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015834399999999999 2024-09-16T14:49:24.127 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.127 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.127 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.127 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009359+0000", 2024-09-16T14:49:24.127 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0012212499999999999 2024-09-16T14:49:24.127 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.127 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.127 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.127 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014320+0000", 2024-09-16T14:49:24.127 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049609850000000002 2024-09-16T14:49:24.128 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.128 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.128 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:24.128 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.017003+0000", 2024-09-16T14:49:24.128 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002683389 2024-09-16T14:49:24.128 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.128 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.128 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.128 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.017204+0000", 2024-09-16T14:49:24.128 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00020068700000000001 2024-09-16T14:49:24.128 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.128 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.128 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.128 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:597 3.3s0 3:c9d06351:::smithi04040151-33:head [append 0~16384] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.990199+0000", 2024-09-16T14:49:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 12.05466431, 2024-09-16T14:49:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.029187326, 2024-09-16T14:49:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 597 2024-09-16T14:49:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.129 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990199+0000", 2024-09-16T14:49:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990199+0000", 2024-09-16T14:49:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990199+0000", 2024-09-16T14:49:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:24.130 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990216+0000", 2024-09-16T14:49:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7176999999999998e-05 2024-09-16T14:49:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990220+0000", 2024-09-16T14:49:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4739999999999999e-06 2024-09-16T14:49:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990224+0000", 2024-09-16T14:49:24.131 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3799999999999998e-06 2024-09-16T14:49:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990262+0000", 2024-09-16T14:49:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8461999999999999e-05 2024-09-16T14:49:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990264+0000", 2024-09-16T14:49:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6330000000000001e-06 2024-09-16T14:49:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.132 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982110+0000", 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99184635300000001 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982190+0000", 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0255999999999999e-05 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.011434+0000", 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.029243331000000001 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:24.133 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.011595+0000", 2024-09-16T14:49:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00016157699999999999 2024-09-16T14:49:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012441+0000", 2024-09-16T14:49:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00084541900000000003 2024-09-16T14:49:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.016197+0000", 2024-09-16T14:49:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003756073 2024-09-16T14:49:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.134 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.019019+0000", 2024-09-16T14:49:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00282168 2024-09-16T14:49:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.019386+0000", 2024-09-16T14:49:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00036785099999999999 2024-09-16T14:49:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.135 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:598 3.3s0 3:c9d06351:::smithi04040151-33:head [setxattr _header (57) in=7b] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.990235+0000", 2024-09-16T14:49:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 12.054628749999999, 2024-09-16T14:49:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.029738257, 2024-09-16T14:49:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 598 2024-09-16T14:49:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.136 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990235+0000", 2024-09-16T14:49:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990235+0000", 2024-09-16T14:49:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990234+0000", 2024-09-16T14:49:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.137 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990237+0000", 2024-09-16T14:49:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:49:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990240+0000", 2024-09-16T14:49:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.625e-06 2024-09-16T14:49:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.138 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990243+0000", 2024-09-16T14:49:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.9000000000000002e-06 2024-09-16T14:49:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990290+0000", 2024-09-16T14:49:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7154000000000003e-05 2024-09-16T14:49:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990292+0000", 2024-09-16T14:49:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.237e-06 2024-09-16T14:49:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.139 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982229+0000", 2024-09-16T14:49:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.991937069 2024-09-16T14:49:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982316+0000", 2024-09-16T14:49:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7495000000000002e-05 2024-09-16T14:49:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012980+0000", 2024-09-16T14:49:24.140 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.030663620999999999 2024-09-16T14:49:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013119+0000", 2024-09-16T14:49:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00013920800000000001 2024-09-16T14:49:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013756+0000", 2024-09-16T14:49:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00063710700000000004 2024-09-16T14:49:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:24.141 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.016198+0000", 2024-09-16T14:49:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002442222 2024-09-16T14:49:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.019618+0000", 2024-09-16T14:49:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003420083 2024-09-16T14:49:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.019973+0000", 2024-09-16T14:49:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00035448499999999998 2024-09-16T14:49:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.142 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.143 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.143 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.143 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:599 3.3s0 3:c9d06351:::smithi04040151-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:24.143 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.990261+0000", 2024-09-16T14:49:24.143 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 12.054602155, 2024-09-16T14:49:24.143 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.034782844, 2024-09-16T14:49:24.143 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.143 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:24.143 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:24.143 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:24.143 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:24.143 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 599 2024-09-16T14:49:24.143 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.143 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.144 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.144 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.144 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990261+0000", 2024-09-16T14:49:24.144 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.144 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.144 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.144 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.144 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990261+0000", 2024-09-16T14:49:24.144 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.144 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.144 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.144 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.144 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990261+0000", 2024-09-16T14:49:24.144 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:24.145 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.145 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.145 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.145 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990263+0000", 2024-09-16T14:49:24.145 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8059999999999999e-06 2024-09-16T14:49:24.145 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.145 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.145 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.145 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990267+0000", 2024-09-16T14:49:24.145 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8850000000000001e-06 2024-09-16T14:49:24.145 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.145 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.145 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.145 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990269+0000", 2024-09-16T14:49:24.146 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7839999999999999e-06 2024-09-16T14:49:24.146 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.146 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.146 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.146 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990311+0000", 2024-09-16T14:49:24.146 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1971000000000002e-05 2024-09-16T14:49:24.146 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.146 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.146 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:24.146 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990312+0000", 2024-09-16T14:49:24.146 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2e-07 2024-09-16T14:49:24.146 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.146 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.146 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.147 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982653+0000", 2024-09-16T14:49:24.147 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99234119499999995 2024-09-16T14:49:24.147 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.147 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.147 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:24.147 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982733+0000", 2024-09-16T14:49:24.147 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9537999999999996e-05 2024-09-16T14:49:24.147 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.147 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.147 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.147 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014037+0000", 2024-09-16T14:49:24.147 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.031304580999999998 2024-09-16T14:49:24.147 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.147 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.148 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:24.148 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014200+0000", 2024-09-16T14:49:24.148 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00016317999999999999 2024-09-16T14:49:24.148 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.148 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.148 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.148 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.020590+0000", 2024-09-16T14:49:24.148 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0063896650000000001 2024-09-16T14:49:24.148 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.148 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.148 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:24.148 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.020693+0000", 2024-09-16T14:49:24.148 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010270800000000001 2024-09-16T14:49:24.148 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.149 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.149 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.149 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.025044+0000", 2024-09-16T14:49:24.149 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.004351293 2024-09-16T14:49:24.149 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.149 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.149 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.149 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.149 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.149 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDPGRecoveryDeleteReply(3.3s0 e66,65 3:c09a69fe:::smithi04040151-5:2,35'7)", 2024-09-16T14:49:24.149 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.005793+0000", 2024-09-16T14:49:24.149 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 11.039070123, 2024-09-16T14:49:24.149 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0091592329999999993, 2024-09-16T14:49:24.149 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:24.150 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:24.150 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:24.150 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.150 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:24.150 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005793+0000", 2024-09-16T14:49:24.150 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.150 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.150 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.150 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:24.150 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005793+0000", 2024-09-16T14:49:24.150 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:24.150 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.150 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.150 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:24.151 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005792+0000", 2024-09-16T14:49:24.151 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:24.151 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.151 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.151 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:24.151 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005800+0000", 2024-09-16T14:49:24.151 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0309999999999998e-06 2024-09-16T14:49:24.151 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.151 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.151 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:24.151 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005813+0000", 2024-09-16T14:49:24.151 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.287e-05 2024-09-16T14:49:24.151 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.151 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.152 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:24.152 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005822+0000", 2024-09-16T14:49:24.152 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3920000000000003e-06 2024-09-16T14:49:24.152 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.152 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.152 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:24.152 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014351+0000", 2024-09-16T14:49:24.152 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0085288429999999995 2024-09-16T14:49:24.152 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:24.152 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:24.152 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:24.152 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014952+0000", 2024-09-16T14:49:24.152 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00060190500000000002 2024-09-16T14:49:24.152 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.153 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.153 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.153 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:24.153 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:24.153 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:24.154 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:24.154 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:24.155 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:24.155 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:24.155 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:49:24.156 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:24.153+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:24.169 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:24.170 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:24.187 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:24.187 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:24.203 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:24.203 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:24.219 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:24.220 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:24.236 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:24.236 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:24.252 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:24.252 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:24.255 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:24.253+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:24.260 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:24.260 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:24.269 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:49:24.270 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:24.270 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:24.286 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:24.286 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:24.303 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:24.303 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:24.320 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:24.320 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:24.356 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:24.355+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:24.410 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:24.411 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:24.419 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:49:24.458 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:24.457+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:24.558 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:24.557+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:24.561 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:24.561 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:24.569 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:49:24.659 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:24.657+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:24.671 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:24.671 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:24.679 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:49:24.759 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:24.758+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:24.782 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:24.782 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:24.790 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:49:24.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:24.876+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.8 to 1 (10000) 2024-09-16T14:49:24.903 INFO:tasks.rados.rados.0.smithi040.stdout:289: done (6 left) 2024-09-16T14:49:24.903 INFO:tasks.rados.rados.0.smithi040.stdout:291: done (5 left) 2024-09-16T14:49:24.904 INFO:tasks.rados.rados.0.smithi040.stdout:292: done (4 left) 2024-09-16T14:49:24.904 INFO:tasks.rados.rados.0.smithi040.stdout:293: done (3 left) 2024-09-16T14:49:24.904 INFO:tasks.rados.rados.0.smithi040.stdout:294: done (2 left) 2024-09-16T14:49:24.904 INFO:tasks.rados.rados.0.smithi040.stdout:295: done (1 left) 2024-09-16T14:49:24.904 INFO:tasks.rados.rados.0.smithi040.stdout:296: done (0 left) 2024-09-16T14:49:24.904 INFO:tasks.rados.rados.0.smithi040.stdout:297: snap_remove snap 18 2024-09-16T14:49:24.909 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:49:24.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-09-16T14:49:24.934 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:24.934 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:24.942 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:49:24.984 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:24.982+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:25.059 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:25.059 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:25.062 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:25.061+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:25.068 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:49:25.164 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:25.162+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:25.171 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:25.171 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:25.180 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:49:25.265 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:25.263+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:25.283 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:25.283 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:25.291 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:49:25.365 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:25.364+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:25.393 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:25.393 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:25.401 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:49:25.466 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:25.465+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:25.503 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:25.504 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:25.512 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:49:25.567 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:25.566+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:25.654 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:25.654 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:25.663 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:49:25.706 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:25.705+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:25.765 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:25.765 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:25.769 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:25.767+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:25.773 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:49:25.870 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:25.868+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:25.904 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is unset 2024-09-16T14:49:25.904 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.9 to 1 (10000) 2024-09-16T14:49:25.905 INFO:tasks.rados.rados.0.smithi040.stdout:297: done (0 left) 2024-09-16T14:49:25.905 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 45 current snap is 27 2024-09-16T14:49:25.905 INFO:tasks.rados.rados.0.smithi040.stdout:298: seq_num 115 ranges {0=770048,770048=704512,1474560=458752} 2024-09-16T14:49:25.909 INFO:tasks.rados.rados.0.smithi040.stdout:298: writing smithi04040151-45 from 0 to 770048 tid 1 2024-09-16T14:49:25.917 INFO:tasks.rados.rados.0.smithi040.stdout:298: writing smithi04040151-45 from 770048 to 1474560 tid 2 2024-09-16T14:49:25.921 INFO:tasks.rados.rados.0.smithi040.stdout:298: writing smithi04040151-45 from 1474560 to 1933312 tid 3 2024-09-16T14:49:25.922 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 22 current snap is 27 2024-09-16T14:49:25.922 INFO:tasks.rados.rados.0.smithi040.stdout:299: seq_num 116 ranges {1359872=786432,2146304=458752,2605056=442368,3047424=262144} 2024-09-16T14:49:25.927 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:25.927 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:25.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-09-16T14:49:25.934 INFO:tasks.rados.rados.0.smithi040.stdout:299: writing smithi04040151-22 from 1359872 to 2146304 tid 1 2024-09-16T14:49:25.937 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:49:25.938 INFO:tasks.rados.rados.0.smithi040.stdout:299: writing smithi04040151-22 from 2146304 to 2605056 tid 2 2024-09-16T14:49:25.940 INFO:tasks.rados.rados.0.smithi040.stdout:299: writing smithi04040151-22 from 2605056 to 3047424 tid 3 2024-09-16T14:49:25.942 INFO:tasks.rados.rados.0.smithi040.stdout:299: writing smithi04040151-22 from 3047424 to 3309568 tid 4 2024-09-16T14:49:25.942 INFO:tasks.rados.rados.0.smithi040.stdout:298: finishing write tid 1 to smithi04040151-45 2024-09-16T14:49:25.943 INFO:tasks.rados.rados.0.smithi040.stdout:298: finishing write tid 2 to smithi04040151-45 2024-09-16T14:49:25.943 INFO:tasks.rados.rados.0.smithi040.stdout:298: finishing write tid 3 to smithi04040151-45 2024-09-16T14:49:25.943 INFO:tasks.rados.rados.0.smithi040.stdout:298: finishing write tid 4 to smithi04040151-45 2024-09-16T14:49:25.943 INFO:tasks.rados.rados.0.smithi040.stdout:298: finishing write tid 5 to smithi04040151-45 2024-09-16T14:49:25.943 INFO:tasks.rados.rados.0.smithi040.stdout:298: oid 45 updating version 0 to 50 2024-09-16T14:49:25.943 INFO:tasks.rados.rados.0.smithi040.stdout:298: oid 45 updating version 50 to 52 2024-09-16T14:49:25.943 INFO:tasks.rados.rados.0.smithi040.stdout:298: oid 45 version 52 is already newer than 51 2024-09-16T14:49:25.943 INFO:tasks.rados.rados.0.smithi040.stdout:298: oid 45 updating version 52 to 53 2024-09-16T14:49:25.943 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 45 v 53 (ObjNum 115 snap 27 seq_num 115) dirty exists 2024-09-16T14:49:25.943 INFO:tasks.rados.rados.0.smithi040.stdout:298: left oid 45 (ObjNum 115 snap 27 seq_num 115) 2024-09-16T14:49:25.943 INFO:tasks.rados.rados.0.smithi040.stdout:299: finishing write tid 1 to smithi04040151-22 2024-09-16T14:49:25.943 INFO:tasks.rados.rados.0.smithi040.stdout:298: done (1 left) 2024-09-16T14:49:25.943 INFO:tasks.rados.rados.0.smithi040.stdout:300: read oid 15 snap -1 2024-09-16T14:49:25.943 INFO:tasks.rados.rados.0.smithi040.stdout:300: expect (ObjNum 94 snap 17 seq_num 94) 2024-09-16T14:49:25.944 INFO:tasks.rados.rados.0.smithi040.stdout:301: read oid 46 snap -1 2024-09-16T14:49:25.944 INFO:tasks.rados.rados.0.smithi040.stdout:301: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-16T14:49:25.944 INFO:tasks.rados.rados.0.smithi040.stdout:302: setattr oid 28 current snap is 27 2024-09-16T14:49:25.944 INFO:tasks.rados.rados.0.smithi040.stdout:299: finishing write tid 2 to smithi04040151-22 2024-09-16T14:49:25.944 INFO:tasks.rados.rados.0.smithi040.stdout:303: snap_remove snap 26 2024-09-16T14:49:25.982 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:25.981+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:26.041 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:26.041 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:26.050 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:49:26.091 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:26.090+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:26.172 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:26.171+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:26.193 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:26.193 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:26.202 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:49:26.273 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:26.272+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:26.331 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:49:26.344 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:26.344 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:26.352 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:49:26.375 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:26.373+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:26.454 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:26.454 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:26.454 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:26.455 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:26.462 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:49:26.470 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:26.471 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:26.475 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:26.473+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:26.486 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:26.486 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:26.501 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:26.502 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:26.519 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:26.520 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:26.536 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:26.536 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:26.552 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:26.553 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:26.568 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:26.569 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:26.572 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:26.572 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:26.572 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:26.572 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:26.575 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:26.573+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:26.581 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:49:26.585 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:26.585 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:26.601 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:26.602 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:26.618 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:26.618 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:26.633 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:26.634 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:26.682 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:26.682 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:26.683 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:26.683 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:26.683 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:26.687 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:26.684+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:26.691 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:49:26.778 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:26.776+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:26.795 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s6 66/65 ECSubWrite(tid=69, reqid=client.4500.0:588, at_version=66'56, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.992663+0000", 2024-09-16T14:49:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.80101902, 2024-09-16T14:49:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032913719999999999, 2024-09-16T14:49:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992663+0000", 2024-09-16T14:49:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992663+0000", 2024-09-16T14:49:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992662+0000", 2024-09-16T14:49:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992691+0000", 2024-09-16T14:49:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8535e-05 2024-09-16T14:49:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992702+0000", 2024-09-16T14:49:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1590000000000001e-05 2024-09-16T14:49:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992711+0000", 2024-09-16T14:49:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5310000000000001e-06 2024-09-16T14:49:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992772+0000", 2024-09-16T14:49:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.1235000000000006e-05 2024-09-16T14:49:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992783+0000", 2024-09-16T14:49:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1413999999999999e-05 2024-09-16T14:49:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995930+0000", 2024-09-16T14:49:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031467000000000001 2024-09-16T14:49:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995954+0000", 2024-09-16T14:49:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4243000000000002e-05 2024-09-16T14:49:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds3 66/65 ECSubWrite(tid=256, reqid=client.4500.0:601, at_version=66'53, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.993987+0000", 2024-09-16T14:49:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.799694630999999, 2024-09-16T14:49:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036421859999999999, 2024-09-16T14:49:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.993987+0000", 2024-09-16T14:49:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.993987+0000", 2024-09-16T14:49:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.993987+0000", 2024-09-16T14:49:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994130+0000", 2024-09-16T14:49:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014385799999999999 2024-09-16T14:49:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994146+0000", 2024-09-16T14:49:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.588e-05 2024-09-16T14:49:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.804 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.804 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.804 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994166+0000", 2024-09-16T14:49:26.804 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9582999999999999e-05 2024-09-16T14:49:26.804 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.804 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.804 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.804 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994229+0000", 2024-09-16T14:49:26.804 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3495000000000001e-05 2024-09-16T14:49:26.804 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.805 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.805 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.805 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994239+0000", 2024-09-16T14:49:26.805 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:49:26.805 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.805 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.805 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.805 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.997601+0000", 2024-09-16T14:49:26.805 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0033624900000000001 2024-09-16T14:49:26.805 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.805 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.806 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.806 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.997630+0000", 2024-09-16T14:49:26.806 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8530999999999998e-05 2024-09-16T14:49:26.806 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.806 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.806 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.806 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.806 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.806 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s6 66/65 ECSubWrite(tid=70, reqid=client.4500.0:589, at_version=66'57, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:26.806 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.994772+0000", 2024-09-16T14:49:26.806 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.798910389, 2024-09-16T14:49:26.806 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003079984, 2024-09-16T14:49:26.806 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.806 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.807 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.807 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.807 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.807 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994772+0000", 2024-09-16T14:49:26.807 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.807 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.807 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.807 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.807 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994772+0000", 2024-09-16T14:49:26.807 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.807 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.807 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.807 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.808 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994771+0000", 2024-09-16T14:49:26.808 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:26.808 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.808 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.808 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.808 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995009+0000", 2024-09-16T14:49:26.808 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000238309 2024-09-16T14:49:26.808 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.808 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.808 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.808 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995022+0000", 2024-09-16T14:49:26.808 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2892e-05 2024-09-16T14:49:26.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.809 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.809 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995030+0000", 2024-09-16T14:49:26.809 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0180000000000001e-06 2024-09-16T14:49:26.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.809 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.809 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995082+0000", 2024-09-16T14:49:26.809 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2361000000000002e-05 2024-09-16T14:49:26.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.809 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.809 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.810 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995093+0000", 2024-09-16T14:49:26.810 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0829000000000001e-05 2024-09-16T14:49:26.810 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.810 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.810 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.810 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.997835+0000", 2024-09-16T14:49:26.810 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0027418030000000001 2024-09-16T14:49:26.810 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.810 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.810 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.810 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.997852+0000", 2024-09-16T14:49:26.811 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6477000000000002e-05 2024-09-16T14:49:26.811 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.811 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.811 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.811 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.811 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.811 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s6 66/65 ECSubWrite(tid=71, reqid=client.4500.0:590, at_version=66'58, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:26.811 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.995298+0000", 2024-09-16T14:49:26.811 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.798383651, 2024-09-16T14:49:26.811 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025683059999999998, 2024-09-16T14:49:26.811 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.811 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.811 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.812 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.812 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.812 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995298+0000", 2024-09-16T14:49:26.812 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.812 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.812 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.812 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.812 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995298+0000", 2024-09-16T14:49:26.812 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.812 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.812 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.812 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.812 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995298+0000", 2024-09-16T14:49:26.813 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:26.813 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.813 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.813 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.813 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995320+0000", 2024-09-16T14:49:26.813 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2118000000000001e-05 2024-09-16T14:49:26.813 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.813 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.813 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.813 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995331+0000", 2024-09-16T14:49:26.813 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1181e-05 2024-09-16T14:49:26.813 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.813 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.813 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.814 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995339+0000", 2024-09-16T14:49:26.814 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.1349999999999992e-06 2024-09-16T14:49:26.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.814 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.814 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.814 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995600+0000", 2024-09-16T14:49:26.814 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00026129300000000001 2024-09-16T14:49:26.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.814 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.814 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.814 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995610+0000", 2024-09-16T14:49:26.814 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0139000000000001e-05 2024-09-16T14:49:26.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.815 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.815 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.815 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.997857+0000", 2024-09-16T14:49:26.815 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002246591 2024-09-16T14:49:26.815 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.815 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.815 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.815 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.997867+0000", 2024-09-16T14:49:26.815 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6160000000000008e-06 2024-09-16T14:49:26.816 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.816 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.816 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.816 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.816 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.816 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.0s2 66/65 ECSubWrite(tid=247, reqid=client.4500.0:591, at_version=66'21, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:26.816 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.000254+0000", 2024-09-16T14:49:26.816 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.793427769999999, 2024-09-16T14:49:26.816 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001790268, 2024-09-16T14:49:26.816 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.816 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.816 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.816 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.817 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.817 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000254+0000", 2024-09-16T14:49:26.817 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.817 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.817 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.817 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.817 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000254+0000", 2024-09-16T14:49:26.817 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.817 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.817 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.817 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.817 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000253+0000", 2024-09-16T14:49:26.817 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:26.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.818 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.818 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.818 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000432+0000", 2024-09-16T14:49:26.818 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017858800000000001 2024-09-16T14:49:26.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.818 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.818 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.818 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000448+0000", 2024-09-16T14:49:26.818 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5630000000000001e-05 2024-09-16T14:49:26.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.818 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.818 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.818 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000456+0000", 2024-09-16T14:49:26.819 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.653e-06 2024-09-16T14:49:26.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.819 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.819 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.819 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000516+0000", 2024-09-16T14:49:26.819 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.0050999999999998e-05 2024-09-16T14:49:26.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.819 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.819 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.819 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000536+0000", 2024-09-16T14:49:26.819 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0037000000000001e-05 2024-09-16T14:49:26.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.820 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.820 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.820 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.002018+0000", 2024-09-16T14:49:26.820 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0014817579999999999 2024-09-16T14:49:26.820 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.820 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.820 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.820 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.002044+0000", 2024-09-16T14:49:26.820 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6304000000000001e-05 2024-09-16T14:49:26.820 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.821 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.821 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.821 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.821 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDPGPush(3.as0 66/65 [PushOp(3:53e83d93:::smithi04040151-22:head, version: 49'20, data_included: [262144~77824], data_size: 77824, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:53e83d93:::smithi04040151-22:head@49'20, size: 1359872, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: 1), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1359872, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:false, omap_recovered_to:, omap_complete:true, error:false))])", 2024-09-16T14:49:26.821 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.000977+0000", 2024-09-16T14:49:26.821 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.792704614, 2024-09-16T14:49:26.821 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0022914279999999999, 2024-09-16T14:49:26.821 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.821 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.821 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.821 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.821 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.822 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000977+0000", 2024-09-16T14:49:26.822 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.822 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.822 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.822 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.822 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000977+0000", 2024-09-16T14:49:26.822 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.822 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.822 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.822 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.822 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000977+0000", 2024-09-16T14:49:26.822 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.822 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.822 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.823 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.823 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000977+0000", 2024-09-16T14:49:26.823 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.823 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.823 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.823 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.823 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.001002+0000", 2024-09-16T14:49:26.823 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4423e-05 2024-09-16T14:49:26.823 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.823 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.823 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.823 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.001016+0000", 2024-09-16T14:49:26.823 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3910000000000001e-05 2024-09-16T14:49:26.823 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.824 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.824 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.824 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.001086+0000", 2024-09-16T14:49:26.824 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0384999999999998e-05 2024-09-16T14:49:26.824 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.824 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.824 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.824 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.003269+0000", 2024-09-16T14:49:26.824 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00218271 2024-09-16T14:49:26.824 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.824 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.824 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.825 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.825 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.825 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s6 66/65 ECSubWrite(tid=72, reqid=client.4500.0:590, at_version=0'0, trim_to=0'0, roll_forward_to=66'58))", 2024-09-16T14:49:26.825 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.005902+0000", 2024-09-16T14:49:26.825 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.787780011000001, 2024-09-16T14:49:26.825 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032803229999999999, 2024-09-16T14:49:26.825 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.825 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.825 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.825 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.826 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.826 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005902+0000", 2024-09-16T14:49:26.826 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.826 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.826 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.826 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.826 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005902+0000", 2024-09-16T14:49:26.826 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.826 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.826 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.826 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.826 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005901+0000", 2024-09-16T14:49:26.826 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005961+0000", 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.0185000000000001e-05 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005976+0000", 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4489e-05 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005986+0000", 2024-09-16T14:49:26.827 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9119999999999993e-06 2024-09-16T14:49:26.828 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.828 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.828 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.828 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006031+0000", 2024-09-16T14:49:26.828 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5556000000000002e-05 2024-09-16T14:49:26.828 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.828 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.828 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.828 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006041+0000", 2024-09-16T14:49:26.828 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0161e-05 2024-09-16T14:49:26.828 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.828 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.828 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.829 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009154+0000", 2024-09-16T14:49:26.829 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031131010000000001 2024-09-16T14:49:26.829 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.829 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.829 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.829 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009182+0000", 2024-09-16T14:49:26.829 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7813999999999998e-05 2024-09-16T14:49:26.829 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.829 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.829 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.829 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.829 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.829 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds3 66/65 ECSubWrite(tid=259, reqid=client.4500.0:603, at_version=0'0, trim_to=0'0, roll_forward_to=66'55))", 2024-09-16T14:49:26.830 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.006460+0000", 2024-09-16T14:49:26.830 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.787221829, 2024-09-16T14:49:26.830 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0029491069999999999, 2024-09-16T14:49:26.830 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.831 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.831 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.831 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.831 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.831 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006460+0000", 2024-09-16T14:49:26.831 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.831 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.831 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.831 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006460+0000", 2024-09-16T14:49:26.831 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.831 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.832 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.832 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006459+0000", 2024-09-16T14:49:26.832 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:26.832 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.832 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.832 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.832 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006475+0000", 2024-09-16T14:49:26.832 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5421999999999999e-05 2024-09-16T14:49:26.832 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.832 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.832 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.832 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006488+0000", 2024-09-16T14:49:26.832 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2955999999999999e-05 2024-09-16T14:49:26.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.833 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.833 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.833 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006496+0000", 2024-09-16T14:49:26.833 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.1170000000000004e-06 2024-09-16T14:49:26.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.833 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.833 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.833 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006558+0000", 2024-09-16T14:49:26.833 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.1648000000000003e-05 2024-09-16T14:49:26.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.833 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.833 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.833 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006568+0000", 2024-09-16T14:49:26.834 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0356e-05 2024-09-16T14:49:26.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.834 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.834 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.834 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009384+0000", 2024-09-16T14:49:26.834 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0028158329999999998 2024-09-16T14:49:26.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.834 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.834 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.834 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009409+0000", 2024-09-16T14:49:26.834 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5508999999999999e-05 2024-09-16T14:49:26.834 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.834 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 66/65 ECSubWrite(tid=37, reqid=client.4500.0:595, at_version=66'24, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.007555+0000", 2024-09-16T14:49:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.78612721, 2024-09-16T14:49:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0019511909999999999, 2024-09-16T14:49:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007555+0000", 2024-09-16T14:49:26.835 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.836 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.836 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.836 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.836 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007555+0000", 2024-09-16T14:49:26.836 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.836 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.836 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.836 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.836 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007554+0000", 2024-09-16T14:49:26.836 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:26.836 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.836 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.836 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.837 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007708+0000", 2024-09-16T14:49:26.837 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015390600000000001 2024-09-16T14:49:26.837 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.837 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.837 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.837 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007720+0000", 2024-09-16T14:49:26.837 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2119999999999999e-05 2024-09-16T14:49:26.837 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.837 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.837 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.837 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007728+0000", 2024-09-16T14:49:26.837 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.977e-06 2024-09-16T14:49:26.837 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.837 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.838 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.838 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007833+0000", 2024-09-16T14:49:26.838 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000104934 2024-09-16T14:49:26.838 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.838 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.838 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.838 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007848+0000", 2024-09-16T14:49:26.838 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5534000000000001e-05 2024-09-16T14:49:26.838 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.838 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.838 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.838 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009487+0000", 2024-09-16T14:49:26.838 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001638818 2024-09-16T14:49:26.838 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.839 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.839 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.839 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009506+0000", 2024-09-16T14:49:26.839 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8749999999999998e-05 2024-09-16T14:49:26.839 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.839 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.839 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.839 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.839 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.839 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 66/65 ECSubWrite(tid=38, reqid=client.4500.0:596, at_version=66'25, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:26.839 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.009720+0000", 2024-09-16T14:49:26.839 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.783962023000001, 2024-09-16T14:49:26.839 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053370680000000004, 2024-09-16T14:49:26.839 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.840 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.840 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.840 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.840 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.840 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009720+0000", 2024-09-16T14:49:26.840 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.840 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.840 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.840 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009720+0000", 2024-09-16T14:49:26.840 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.841 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.841 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.841 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009719+0000", 2024-09-16T14:49:26.841 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:26.841 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.841 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.841 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.841 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009878+0000", 2024-09-16T14:49:26.841 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015914100000000001 2024-09-16T14:49:26.841 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.841 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.841 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.841 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009902+0000", 2024-09-16T14:49:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3164999999999999e-05 2024-09-16T14:49:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009911+0000", 2024-09-16T14:49:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9429999999999995e-06 2024-09-16T14:49:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009974+0000", 2024-09-16T14:49:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2117000000000002e-05 2024-09-16T14:49:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.842 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.843 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009985+0000", 2024-09-16T14:49:26.843 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1539000000000001e-05 2024-09-16T14:49:26.843 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.843 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.843 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.843 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.015029+0000", 2024-09-16T14:49:26.843 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050442999999999998 2024-09-16T14:49:26.843 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.843 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.843 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.843 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.015057+0000", 2024-09-16T14:49:26.843 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7606999999999998e-05 2024-09-16T14:49:26.843 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.844 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.844 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.844 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.844 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 66/65 ECSubWrite(tid=39, reqid=client.4500.0:597, at_version=66'26, trim_to=0'0, roll_forward_to=66'23))", 2024-09-16T14:49:26.844 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:26.844 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.781292716999999, 2024-09-16T14:49:26.844 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0060634030000000002, 2024-09-16T14:49:26.844 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.844 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.844 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.844 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.844 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.844 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:26.845 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.845 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.845 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.845 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.845 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:26.845 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.845 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.845 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.845 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.845 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012388+0000", 2024-09-16T14:49:26.845 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:26.845 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.845 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.846 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.846 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012433+0000", 2024-09-16T14:49:26.846 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4588000000000001e-05 2024-09-16T14:49:26.846 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.846 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.846 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.846 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012604+0000", 2024-09-16T14:49:26.846 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017141300000000001 2024-09-16T14:49:26.846 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.846 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.846 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.846 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012617+0000", 2024-09-16T14:49:26.846 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2792999999999999e-05 2024-09-16T14:49:26.846 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.847 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.847 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.847 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012666+0000", 2024-09-16T14:49:26.847 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9079000000000002e-05 2024-09-16T14:49:26.847 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.847 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.847 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.847 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012678+0000", 2024-09-16T14:49:26.847 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1405e-05 2024-09-16T14:49:26.847 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.847 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.847 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.847 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018433+0000", 2024-09-16T14:49:26.848 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005755146 2024-09-16T14:49:26.848 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.848 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.848 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.848 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018453+0000", 2024-09-16T14:49:26.848 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9780999999999999e-05 2024-09-16T14:49:26.848 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.848 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.848 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.848 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.848 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.848 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 66/65 ECSubWrite(tid=40, reqid=client.4500.0:598, at_version=66'27, trim_to=0'0, roll_forward_to=66'23))", 2024-09-16T14:49:26.848 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.013745+0000", 2024-09-16T14:49:26.848 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.779937088000001, 2024-09-16T14:49:26.849 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0047198850000000001, 2024-09-16T14:49:26.849 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.849 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.849 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.849 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.849 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.849 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013745+0000", 2024-09-16T14:49:26.849 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.849 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.849 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.849 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.849 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013745+0000", 2024-09-16T14:49:26.849 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.850 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.850 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.850 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.850 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013744+0000", 2024-09-16T14:49:26.850 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:26.850 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.850 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.850 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.850 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013766+0000", 2024-09-16T14:49:26.850 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1721000000000001e-05 2024-09-16T14:49:26.850 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.850 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.850 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.850 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013788+0000", 2024-09-16T14:49:26.851 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1708e-05 2024-09-16T14:49:26.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.851 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.851 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.851 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013798+0000", 2024-09-16T14:49:26.851 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.026e-05 2024-09-16T14:49:26.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.851 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.851 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.851 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013847+0000", 2024-09-16T14:49:26.851 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9579000000000001e-05 2024-09-16T14:49:26.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.851 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.851 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.852 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013857+0000", 2024-09-16T14:49:26.852 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9939999999999995e-06 2024-09-16T14:49:26.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.852 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.852 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.852 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018456+0000", 2024-09-16T14:49:26.852 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.004598141 2024-09-16T14:49:26.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.852 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.852 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.852 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018465+0000", 2024-09-16T14:49:26.852 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2439999999999999e-06 2024-09-16T14:49:26.852 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.853 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.853 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.853 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.853 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.853 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs7 70/65 ECSubWrite(tid=284, reqid=client.4500.0:632, at_version=70'44, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:26.853 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:16.869977+0000", 2024-09-16T14:49:26.853 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 9.9237046479999993, 2024-09-16T14:49:26.853 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0017620590000000001, 2024-09-16T14:49:26.853 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.853 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.853 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.853 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.853 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.853 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.869977+0000", 2024-09-16T14:49:26.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.854 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.854 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.869977+0000", 2024-09-16T14:49:26.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.854 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.854 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.869977+0000", 2024-09-16T14:49:26.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:26.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.854 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.855 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.870281+0000", 2024-09-16T14:49:26.855 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00030440000000000003 2024-09-16T14:49:26.855 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.855 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.855 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.855 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.870295+0000", 2024-09-16T14:49:26.855 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3397999999999999e-05 2024-09-16T14:49:26.855 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.855 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.855 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.855 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.870303+0000", 2024-09-16T14:49:26.855 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2320000000000001e-06 2024-09-16T14:49:26.855 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.855 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.870361+0000", 2024-09-16T14:49:26.856 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.7689000000000003e-05 2024-09-16T14:49:26.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.870374+0000", 2024-09-16T14:49:26.856 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3247e-05 2024-09-16T14:49:26.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.871718+0000", 2024-09-16T14:49:26.856 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0013438479999999999 2024-09-16T14:49:26.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.857 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.857 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:16.871739+0000", 2024-09-16T14:49:26.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1506e-05 2024-09-16T14:49:26.857 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.857 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.857 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.857 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:702 3.as0 3:572aff57:::smithi04040151-20:head [copy-from ver 4] snapc 2f=[2f,2e,29,28,25] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:26.857 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:26.857 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 2.8936044349999999, 2024-09-16T14:49:26.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.032342711000000003, 2024-09-16T14:49:26.858 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.858 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:26.858 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:26.858 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:26.858 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:26.858 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 702 2024-09-16T14:49:26.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.858 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.858 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.858 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.858 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:26.858 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.858 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.859 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.859 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:26.859 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.859 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.859 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.859 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900076+0000", 2024-09-16T14:49:26.859 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:26.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.859 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.859 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.859 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900082+0000", 2024-09-16T14:49:26.859 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3640000000000004e-06 2024-09-16T14:49:26.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.860 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.860 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.860 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900094+0000", 2024-09-16T14:49:26.860 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1208e-05 2024-09-16T14:49:26.860 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.860 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.860 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.860 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900105+0000", 2024-09-16T14:49:26.860 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1782e-05 2024-09-16T14:49:26.860 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.860 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.860 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.860 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900161+0000", 2024-09-16T14:49:26.861 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5244999999999997e-05 2024-09-16T14:49:26.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.861 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.861 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:26.861 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900170+0000", 2024-09-16T14:49:26.861 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4409999999999999e-06 2024-09-16T14:49:26.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.861 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.861 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.861 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.901930+0000", 2024-09-16T14:49:26.861 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001760246 2024-09-16T14:49:26.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.861 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.861 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902046+0000", 2024-09-16T14:49:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000115882 2024-09-16T14:49:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.917553+0000", 2024-09-16T14:49:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015506967999999999 2024-09-16T14:49:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918732+0000", 2024-09-16T14:49:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001178733 2024-09-16T14:49:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.863 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:26.863 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932379+0000", 2024-09-16T14:49:26.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013647276999999999 2024-09-16T14:49:26.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.863 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.863 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.863 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932420+0000", 2024-09-16T14:49:26.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1066999999999998e-05 2024-09-16T14:49:26.863 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.863 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.863 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.863 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.863 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:703 3.as0 3:572aff57:::smithi04040151-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:26.864 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:26.864 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 2.8935630369999998, 2024-09-16T14:49:26.864 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.018566787000000001, 2024-09-16T14:49:26.864 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.864 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:26.864 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:26.864 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:26.864 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:26.864 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 703 2024-09-16T14:49:26.864 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.864 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.864 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.864 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:26.865 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:26.865 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900118+0000", 2024-09-16T14:49:26.865 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:26.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900122+0000", 2024-09-16T14:49:26.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5669999999999999e-06 2024-09-16T14:49:26.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.866 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900129+0000", 2024-09-16T14:49:26.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0360000000000001e-06 2024-09-16T14:49:26.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.866 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900134+0000", 2024-09-16T14:49:26.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.916e-06 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900217+0000", 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3189000000000006e-05 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900222+0000", 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7430000000000002e-06 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902209+0000", 2024-09-16T14:49:26.867 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0019871440000000001 2024-09-16T14:49:26.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.868 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.868 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:26.868 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902254+0000", 2024-09-16T14:49:26.868 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4969000000000003e-05 2024-09-16T14:49:26.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.868 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.868 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.868 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918411+0000", 2024-09-16T14:49:26.868 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.016157207999999999 2024-09-16T14:49:26.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.868 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.868 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918534+0000", 2024-09-16T14:49:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000123146 2024-09-16T14:49:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918686+0000", 2024-09-16T14:49:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015134799999999999 2024-09-16T14:49:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs7 77/65 ECSubWrite(tid=308, reqid=osd.3.0:308, at_version=77'53, trim_to=0'0, roll_forward_to=74'51))", 2024-09-16T14:49:26.869 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.905099+0000", 2024-09-16T14:49:26.870 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 2.888582939, 2024-09-16T14:49:26.870 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0017398559999999999, 2024-09-16T14:49:26.870 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.870 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.870 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.870 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.870 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.870 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.905099+0000", 2024-09-16T14:49:26.870 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.871 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.871 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.871 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.871 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.905099+0000", 2024-09-16T14:49:26.871 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.871 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.871 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.871 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.871 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.905099+0000", 2024-09-16T14:49:26.871 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:26.871 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.871 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.871 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.872 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.905122+0000", 2024-09-16T14:49:26.872 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3116e-05 2024-09-16T14:49:26.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.872 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.872 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.872 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.905131+0000", 2024-09-16T14:49:26.872 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4550000000000001e-06 2024-09-16T14:49:26.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.872 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.872 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.872 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.905139+0000", 2024-09-16T14:49:26.872 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7770000000000002e-06 2024-09-16T14:49:26.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.905182+0000", 2024-09-16T14:49:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3387999999999999e-05 2024-09-16T14:49:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.905191+0000", 2024-09-16T14:49:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4619999999999996e-06 2024-09-16T14:49:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.906814+0000", 2024-09-16T14:49:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0016228530000000001 2024-09-16T14:49:26.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.874 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.874 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.906839+0000", 2024-09-16T14:49:26.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5131e-05 2024-09-16T14:49:26.874 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.874 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.874 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.874 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s2 77/65 ECSubWrite(tid=312, reqid=client.4500.0:711, at_version=77'98, trim_to=0'0, roll_forward_to=73'95))", 2024-09-16T14:49:26.874 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.915065+0000", 2024-09-16T14:49:26.874 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 2.8786165759999998, 2024-09-16T14:49:26.875 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003580101, 2024-09-16T14:49:26.875 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.875 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.875 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.875 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.875 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.875 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915065+0000", 2024-09-16T14:49:26.875 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.875 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.875 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.875 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915065+0000", 2024-09-16T14:49:26.875 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.876 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915065+0000", 2024-09-16T14:49:26.876 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:26.876 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.876 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915202+0000", 2024-09-16T14:49:26.876 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00013698000000000001 2024-09-16T14:49:26.876 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.876 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915211+0000", 2024-09-16T14:49:26.876 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1940000000000004e-06 2024-09-16T14:49:26.877 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.877 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.877 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.877 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915218+0000", 2024-09-16T14:49:26.877 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2230000000000001e-06 2024-09-16T14:49:26.877 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.880 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.880 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.880 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915387+0000", 2024-09-16T14:49:26.881 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00016841100000000001 2024-09-16T14:49:26.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.881 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.881 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.881 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915395+0000", 2024-09-16T14:49:26.881 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4530000000000002e-06 2024-09-16T14:49:26.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.881 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.881 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.881 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918628+0000", 2024-09-16T14:49:26.881 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032327760000000001 2024-09-16T14:49:26.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.881 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.882 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.882 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918646+0000", 2024-09-16T14:49:26.882 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7750000000000001e-05 2024-09-16T14:49:26.882 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.882 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.882 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.882 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.882 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.882 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s2 77/65 ECSubWrite(tid=313, reqid=client.4500.0:712, at_version=77'99, trim_to=0'0, roll_forward_to=73'95))", 2024-09-16T14:49:26.882 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.921327+0000", 2024-09-16T14:49:26.882 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 2.872355174, 2024-09-16T14:49:26.882 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0026816610000000001, 2024-09-16T14:49:26.882 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.882 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.883 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.883 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921327+0000", 2024-09-16T14:49:26.883 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.883 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.883 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921327+0000", 2024-09-16T14:49:26.883 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.883 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.883 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921326+0000", 2024-09-16T14:49:26.884 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:26.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.884 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.884 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.884 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921590+0000", 2024-09-16T14:49:26.884 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00026393599999999999 2024-09-16T14:49:26.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.884 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.884 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.884 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921617+0000", 2024-09-16T14:49:26.884 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6815999999999999e-05 2024-09-16T14:49:26.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.884 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.884 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.885 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921636+0000", 2024-09-16T14:49:26.885 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9371000000000001e-05 2024-09-16T14:49:26.885 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.885 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.885 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.885 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921675+0000", 2024-09-16T14:49:26.885 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8794000000000002e-05 2024-09-16T14:49:26.885 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.885 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.885 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.885 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921683+0000", 2024-09-16T14:49:26.885 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.1280000000000008e-06 2024-09-16T14:49:26.885 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.885 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.886 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.886 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.923988+0000", 2024-09-16T14:49:26.886 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0023044340000000002 2024-09-16T14:49:26.886 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.886 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.886 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.886 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.924008+0000", 2024-09-16T14:49:26.886 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0868000000000001e-05 2024-09-16T14:49:26.886 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.886 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.886 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.886 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.886 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.886 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs7 77/65 ECSubWrite(tid=314, reqid=osd.3.0:308, at_version=0'0, trim_to=0'0, roll_forward_to=77'53))", 2024-09-16T14:49:26.887 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.921644+0000", 2024-09-16T14:49:26.887 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 2.8720379440000001, 2024-09-16T14:49:26.887 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021148930000000001, 2024-09-16T14:49:26.887 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.887 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.887 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.887 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.887 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.887 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921644+0000", 2024-09-16T14:49:26.887 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.887 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.887 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.887 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.887 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921644+0000", 2024-09-16T14:49:26.888 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.888 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.888 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.888 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.888 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921644+0000", 2024-09-16T14:49:26.888 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:26.888 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.888 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.888 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.888 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921661+0000", 2024-09-16T14:49:26.888 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7371e-05 2024-09-16T14:49:26.888 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.888 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921671+0000", 2024-09-16T14:49:26.889 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9720000000000004e-06 2024-09-16T14:49:26.889 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.889 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921676+0000", 2024-09-16T14:49:26.889 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5199999999999997e-06 2024-09-16T14:49:26.889 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.889 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921725+0000", 2024-09-16T14:49:26.889 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8273999999999999e-05 2024-09-16T14:49:26.889 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.890 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.890 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.890 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.921735+0000", 2024-09-16T14:49:26.890 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0155e-05 2024-09-16T14:49:26.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.890 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.890 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.890 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.923738+0000", 2024-09-16T14:49:26.890 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0020028189999999999 2024-09-16T14:49:26.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.890 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.890 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.890 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.923759+0000", 2024-09-16T14:49:26.890 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1195999999999998e-05 2024-09-16T14:49:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s2 77/65 ECSubWrite(tid=320, reqid=client.4500.0:713, at_version=0'0, trim_to=0'0, roll_forward_to=77'100))", 2024-09-16T14:49:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.932063+0000", 2024-09-16T14:49:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 2.8616191190000002, 2024-09-16T14:49:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002371387, 2024-09-16T14:49:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.891 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:26.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932063+0000", 2024-09-16T14:49:26.892 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.892 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.892 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:26.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932063+0000", 2024-09-16T14:49:26.892 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:26.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.892 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.892 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:26.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932062+0000", 2024-09-16T14:49:26.892 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:26.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.892 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.893 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:26.893 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932073+0000", 2024-09-16T14:49:26.893 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0817e-05 2024-09-16T14:49:26.893 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.893 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.893 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:26.893 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932088+0000", 2024-09-16T14:49:26.893 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5046e-05 2024-09-16T14:49:26.893 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.893 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.893 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:26.893 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932096+0000", 2024-09-16T14:49:26.893 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7479999999999993e-06 2024-09-16T14:49:26.893 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.894 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.894 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:26.894 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932138+0000", 2024-09-16T14:49:26.894 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2651999999999998e-05 2024-09-16T14:49:26.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.894 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.894 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:26.894 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932148+0000", 2024-09-16T14:49:26.894 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2590000000000006e-06 2024-09-16T14:49:26.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.894 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.894 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:26.894 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.934406+0000", 2024-09-16T14:49:26.894 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0022588220000000002 2024-09-16T14:49:26.895 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:26.895 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:26.895 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:26.895 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.934434+0000", 2024-09-16T14:49:26.895 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7813e-05 2024-09-16T14:49:26.895 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.895 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.895 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.895 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:26.895 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:26.895 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:26.897 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:49:26.897 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:26.894+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:26.913 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.10 to 1 (10000) 2024-09-16T14:49:26.914 INFO:tasks.rados.rados.0.smithi040.stdout:303: done (4 left) 2024-09-16T14:49:26.914 INFO:tasks.rados.rados.0.smithi040.stdout:304: read oid 10 snap -1 2024-09-16T14:49:26.914 INFO:tasks.rados.rados.0.smithi040.stdout:304: expect deleted 2024-09-16T14:49:26.914 INFO:tasks.rados.rados.0.smithi040.stdout:299: finishing write tid 3 to smithi04040151-22 2024-09-16T14:49:26.914 INFO:tasks.rados.rados.0.smithi040.stdout:299: finishing write tid 4 to smithi04040151-22 2024-09-16T14:49:26.915 INFO:tasks.rados.rados.0.smithi040.stdout:299: finishing write tid 5 to smithi04040151-22 2024-09-16T14:49:26.915 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 28 v 62 (ObjNum 68 snap 4 seq_num 68) dirty exists 2024-09-16T14:49:26.915 INFO:tasks.rados.rados.0.smithi040.stdout:299: finishing write tid 6 to smithi04040151-22 2024-09-16T14:49:26.915 INFO:tasks.rados.rados.0.smithi040.stdout:299: oid 22 updating version 0 to 39 2024-09-16T14:49:26.915 INFO:tasks.rados.rados.0.smithi040.stdout:299: oid 22 version 39 is already newer than 37 2024-09-16T14:49:26.915 INFO:tasks.rados.rados.0.smithi040.stdout:299: oid 22 updating version 39 to 40 2024-09-16T14:49:26.915 INFO:tasks.rados.rados.0.smithi040.stdout:299: oid 22 version 40 is already newer than 38 2024-09-16T14:49:26.915 INFO:tasks.rados.rados.0.smithi040.stdout:299: oid 22 updating version 40 to 41 2024-09-16T14:49:26.915 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 22 v 41 (ObjNum 116 snap 27 seq_num 116) dirty exists 2024-09-16T14:49:26.915 INFO:tasks.rados.rados.0.smithi040.stdout:299: left oid 22 (ObjNum 116 snap 27 seq_num 116) 2024-09-16T14:49:26.915 INFO:tasks.rados.rados.0.smithi040.stdout:301: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-16T14:49:26.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-09-16T14:49:26.939 INFO:tasks.rados.rados.0.smithi040.stdout:300: expect (ObjNum 94 snap 17 seq_num 94) 2024-09-16T14:49:26.990 INFO:tasks.rados.rados.0.smithi040.stdout:299: done (4 left) 2024-09-16T14:49:26.990 INFO:tasks.rados.rados.0.smithi040.stdout:300: done (3 left) 2024-09-16T14:49:26.990 INFO:tasks.rados.rados.0.smithi040.stdout:301: done (2 left) 2024-09-16T14:49:26.990 INFO:tasks.rados.rados.0.smithi040.stdout:302: done (1 left) 2024-09-16T14:49:26.990 INFO:tasks.rados.rados.0.smithi040.stdout:304: done (0 left) 2024-09-16T14:49:26.990 INFO:tasks.rados.rados.0.smithi040.stdout:305: rmattr oid 50 current snap is 27 2024-09-16T14:49:26.990 INFO:tasks.rados.rados.0.smithi040.stdout:306: rollback oid 33 current snap is 27 2024-09-16T14:49:26.991 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 33 to 25 2024-09-16T14:49:26.991 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 1 current snap is 27 2024-09-16T14:49:26.991 INFO:tasks.rados.rados.0.smithi040.stdout:307: seq_num 117 ranges {1703936=425984,2129920=606208,2736128=540672,3276800=409600} 2024-09-16T14:49:26.997 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:26.995+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:27.002 INFO:tasks.rados.rados.0.smithi040.stdout:307: writing smithi04040151-1 from 1703936 to 2129920 tid 1 2024-09-16T14:49:27.007 INFO:tasks.rados.rados.0.smithi040.stdout:307: writing smithi04040151-1 from 2129920 to 2736128 tid 2 2024-09-16T14:49:27.011 INFO:tasks.rados.rados.0.smithi040.stdout:307: writing smithi04040151-1 from 2736128 to 3276800 tid 3 2024-09-16T14:49:27.015 INFO:tasks.rados.rados.0.smithi040.stdout:307: writing smithi04040151-1 from 3276800 to 3686400 tid 4 2024-09-16T14:49:27.015 INFO:tasks.rados.rados.0.smithi040.stdout:308: copy_from oid 49 from oid 18 current snap is 27 2024-09-16T14:49:27.015 INFO:tasks.rados.rados.0.smithi040.stdout:306: finishing rollback tid 0 to smithi04040151-33 2024-09-16T14:49:27.015 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 50 v 59 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-16T14:49:27.015 INFO:tasks.rados.rados.0.smithi040.stdout:306: finishing rollback tid 1 to smithi04040151-33 2024-09-16T14:49:27.015 INFO:tasks.rados.rados.0.smithi040.stdout:306: finishing rollback tid 2 to smithi04040151-33 2024-09-16T14:49:27.016 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 33 v 31 (ObjNum 98 snap 18 seq_num 98) dirty exists 2024-09-16T14:49:27.016 INFO:tasks.rados.rados.0.smithi040.stdout:307: finishing write tid 1 to smithi04040151-1 2024-09-16T14:49:27.016 INFO:tasks.rados.rados.0.smithi040.stdout:305: done (3 left) 2024-09-16T14:49:27.016 INFO:tasks.rados.rados.0.smithi040.stdout:306: done (2 left) 2024-09-16T14:49:27.016 INFO:tasks.rados.rados.0.smithi040.stdout:309: setattr oid 9 current snap is 27 2024-09-16T14:49:27.016 INFO:tasks.rados.rados.0.smithi040.stdout:307: finishing write tid 2 to smithi04040151-1 2024-09-16T14:49:27.016 INFO:tasks.rados.rados.0.smithi040.stdout:310: read oid 41 snap 24 2024-09-16T14:49:27.017 INFO:tasks.rados.rados.0.smithi040.stdout:310: expect (ObjNum 104 snap 22 seq_num 104) 2024-09-16T14:49:27.017 INFO:tasks.rados.rados.0.smithi040.stdout:311: snap_remove snap 25 2024-09-16T14:49:27.027 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:27.028 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:27.028 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:27.028 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:27.038 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:49:27.098 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:27.096+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:27.147 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:27.147 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:27.147 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:27.147 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:27.147 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:27.156 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:49:27.199 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:27.198+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:27.264 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:27.264 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:27.265 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:27.265 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:27.265 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.265 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:454 3.6s0 3:608d2d89:::smithi04040151-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:27.265 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:27.265 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.333143388, 2024-09-16T14:49:27.265 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013776338000000001, 2024-09-16T14:49:27.265 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.265 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:27.265 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:27.265 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:27.265 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 454 2024-09-16T14:49:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.266 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929436+0000", 2024-09-16T14:49:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7939999999999998e-06 2024-09-16T14:49:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929444+0000", 2024-09-16T14:49:27.267 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4749999999999993e-06 2024-09-16T14:49:27.268 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.268 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.268 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.268 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929449+0000", 2024-09-16T14:49:27.268 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2750000000000003e-06 2024-09-16T14:49:27.268 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.268 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.268 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.268 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.933117+0000", 2024-09-16T14:49:27.268 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0036674250000000002 2024-09-16T14:49:27.268 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.268 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.268 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:27.269 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.933207+0000", 2024-09-16T14:49:27.269 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0358000000000005e-05 2024-09-16T14:49:27.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.269 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.269 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.269 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.937436+0000", 2024-09-16T14:49:27.269 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0042287089999999998 2024-09-16T14:49:27.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.269 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.269 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:27.269 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.937535+0000", 2024-09-16T14:49:27.269 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9501999999999997e-05 2024-09-16T14:49:27.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.269 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.270 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.270 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.943210+0000", 2024-09-16T14:49:27.270 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056742429999999998 2024-09-16T14:49:27.270 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.270 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.270 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.270 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.270 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 52/27 ECSubWrite(tid=182, reqid=osd.11.0:182, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:27.270 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:27.270 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.375331641999999, 2024-09-16T14:49:27.270 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014202568, 2024-09-16T14:49:27.270 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.270 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.271 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.271 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.271 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.271 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:27.271 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.271 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.271 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.271 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:27.271 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.271 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.271 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.272 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:27.272 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.272 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.272 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.272 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887302+0000", 2024-09-16T14:49:27.272 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.7673999999999997e-05 2024-09-16T14:49:27.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.272 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.272 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.272 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887313+0000", 2024-09-16T14:49:27.272 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0712e-05 2024-09-16T14:49:27.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.272 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.273 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.273 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887321+0000", 2024-09-16T14:49:27.273 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1329999999999999e-06 2024-09-16T14:49:27.273 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.273 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.273 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.273 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887397+0000", 2024-09-16T14:49:27.273 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5564999999999999e-05 2024-09-16T14:49:27.273 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.273 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.273 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.273 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887407+0000", 2024-09-16T14:49:27.273 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9690000000000006e-06 2024-09-16T14:49:27.273 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.274 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.274 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.274 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901422+0000", 2024-09-16T14:49:27.274 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01401468 2024-09-16T14:49:27.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.274 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.274 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.274 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901448+0000", 2024-09-16T14:49:27.274 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6191999999999999e-05 2024-09-16T14:49:27.274 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.274 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.274 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.274 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.275 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 52/27 ECSubWrite(tid=214, reqid=osd.7.0:214, at_version=52'35, trim_to=0'0, roll_forward_to=51'31))", 2024-09-16T14:49:27.275 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:27.275 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.375288442999999, 2024-09-16T14:49:27.275 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014116002000000001, 2024-09-16T14:49:27.275 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.275 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.275 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.275 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.275 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.275 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:27.275 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.275 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.275 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.276 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:27.276 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.276 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.276 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.276 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.276 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:27.276 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.276 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.276 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.276 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.276 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887318+0000", 2024-09-16T14:49:27.276 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0477999999999999e-05 2024-09-16T14:49:27.276 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.276 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.277 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.277 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887329+0000", 2024-09-16T14:49:27.277 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0845999999999999e-05 2024-09-16T14:49:27.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.277 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.277 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.277 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887337+0000", 2024-09-16T14:49:27.277 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7880000000000007e-06 2024-09-16T14:49:27.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.277 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.277 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.277 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887631+0000", 2024-09-16T14:49:27.277 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029412399999999999 2024-09-16T14:49:27.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.278 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.278 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.278 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887640+0000", 2024-09-16T14:49:27.278 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0180000000000007e-06 2024-09-16T14:49:27.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.278 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.278 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.278 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901368+0000", 2024-09-16T14:49:27.278 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013728346000000001 2024-09-16T14:49:27.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.278 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.278 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.278 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901404+0000", 2024-09-16T14:49:27.278 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5964000000000003e-05 2024-09-16T14:49:27.279 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.279 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.279 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.279 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.279 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 52/27 ECSubWrite(tid=51, reqid=osd.1.0:50, at_version=0'0, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T14:49:27.279 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:27.279 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.373415475000002, 2024-09-16T14:49:27.279 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013036324, 2024-09-16T14:49:27.279 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.279 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.279 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.279 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.280 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.280 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:27.280 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.280 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.280 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.280 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:27.280 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.280 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.280 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.280 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889160+0000", 2024-09-16T14:49:27.280 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:27.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.281 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.281 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889174+0000", 2024-09-16T14:49:27.281 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3416e-05 2024-09-16T14:49:27.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.281 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.281 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889186+0000", 2024-09-16T14:49:27.281 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2369e-05 2024-09-16T14:49:27.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.281 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.281 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889195+0000", 2024-09-16T14:49:27.281 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6540000000000005e-06 2024-09-16T14:49:27.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.282 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.282 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889243+0000", 2024-09-16T14:49:27.282 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.8455000000000003e-05 2024-09-16T14:49:27.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.282 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.282 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889253+0000", 2024-09-16T14:49:27.282 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3700000000000001e-06 2024-09-16T14:49:27.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.282 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.282 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902168+0000", 2024-09-16T14:49:27.283 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012915311 2024-09-16T14:49:27.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.283 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.283 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902198+0000", 2024-09-16T14:49:27.283 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9641e-05 2024-09-16T14:49:27.283 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.283 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.283 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.283 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s2 52/27 ECSubWrite(tid=29, reqid=osd.4.0:29, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:27.283 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:27.284 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 28.3726123, 2024-09-16T14:49:27.284 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012371216000000001, 2024-09-16T14:49:27.284 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.284 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.284 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.284 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.284 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.284 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:27.284 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.284 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.284 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.284 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:27.285 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.285 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.288 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.288 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.288 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889990+0000", 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5548999999999999e-05 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890001+0000", 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.098e-05 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.289 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890009+0000", 2024-09-16T14:49:27.290 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5930000000000006e-06 2024-09-16T14:49:27.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.290 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.290 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.290 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890069+0000", 2024-09-16T14:49:27.290 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.9855999999999997e-05 2024-09-16T14:49:27.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.290 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.290 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.290 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890082+0000", 2024-09-16T14:49:27.290 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3312999999999999e-05 2024-09-16T14:49:27.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.290 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.290 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.291 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902310+0000", 2024-09-16T14:49:27.291 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012227780000000001 2024-09-16T14:49:27.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.291 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.291 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.291 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902336+0000", 2024-09-16T14:49:27.291 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5652000000000001e-05 2024-09-16T14:49:27.291 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.291 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.291 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.291 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.291 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s2 61/27 ECSubWrite(tid=35, reqid=osd.4.0:34, at_version=0'0, trim_to=0'0, roll_forward_to=61'22))", 2024-09-16T14:49:27.291 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:27.292 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.333166798000001, 2024-09-16T14:49:27.292 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012712925, 2024-09-16T14:49:27.292 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.292 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.292 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.292 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.292 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.292 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:27.292 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.292 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.292 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.292 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.292 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:27.293 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.293 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.293 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.293 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929409+0000", 2024-09-16T14:49:27.293 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:27.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.293 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.293 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.293 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929422+0000", 2024-09-16T14:49:27.293 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3497e-05 2024-09-16T14:49:27.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.293 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.293 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.294 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929434+0000", 2024-09-16T14:49:27.294 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.205e-05 2024-09-16T14:49:27.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.294 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.294 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929443+0000", 2024-09-16T14:49:27.294 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8419999999999994e-06 2024-09-16T14:49:27.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.294 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.294 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929490+0000", 2024-09-16T14:49:27.294 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6825e-05 2024-09-16T14:49:27.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.295 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.295 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929500+0000", 2024-09-16T14:49:27.295 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8810000000000008e-06 2024-09-16T14:49:27.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.295 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.295 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942092+0000", 2024-09-16T14:49:27.295 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012591955 2024-09-16T14:49:27.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.295 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.295 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942123+0000", 2024-09-16T14:49:27.295 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1015e-05 2024-09-16T14:49:27.295 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.296 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.296 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.296 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs6 61/27 ECSubWrite(tid=235, reqid=osd.7.0:235, at_version=61'21, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:27.296 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:27.296 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.331739887000001, 2024-09-16T14:49:27.296 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011696734, 2024-09-16T14:49:27.296 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.296 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.296 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.296 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.296 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:27.297 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.297 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.297 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:27.297 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.297 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.297 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:27.297 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:27.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.298 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.298 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930866+0000", 2024-09-16T14:49:27.298 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9138000000000001e-05 2024-09-16T14:49:27.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.298 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.298 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930874+0000", 2024-09-16T14:49:27.298 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5890000000000003e-06 2024-09-16T14:49:27.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.298 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.298 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930882+0000", 2024-09-16T14:49:27.298 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.25e-06 2024-09-16T14:49:27.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930933+0000", 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1536999999999998e-05 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930941+0000", 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7510000000000008e-06 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942507+0000", 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011565765 2024-09-16T14:49:27.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.300 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.300 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942534+0000", 2024-09-16T14:49:27.300 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6900999999999999e-05 2024-09-16T14:49:27.300 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.300 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.300 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.300 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 61/27 ECSubWrite(tid=63, reqid=osd.1.0:63, at_version=61'50, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:27.300 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:27.300 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.330203450999999, 2024-09-16T14:49:27.300 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014509462000000001, 2024-09-16T14:49:27.301 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.301 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.301 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.301 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.301 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:27.301 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.301 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.301 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:27.301 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.302 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.302 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:27.302 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.302 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.302 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932395+0000", 2024-09-16T14:49:27.302 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2039999999999999e-05 2024-09-16T14:49:27.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.302 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.302 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932404+0000", 2024-09-16T14:49:27.302 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4879999999999996e-06 2024-09-16T14:49:27.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.303 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.303 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932412+0000", 2024-09-16T14:49:27.303 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.61e-06 2024-09-16T14:49:27.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.303 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.303 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932577+0000", 2024-09-16T14:49:27.303 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00016469799999999999 2024-09-16T14:49:27.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.303 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.303 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932605+0000", 2024-09-16T14:49:27.303 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7963e-05 2024-09-16T14:49:27.304 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.304 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.304 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946863+0000", 2024-09-16T14:49:27.304 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014258600999999999 2024-09-16T14:49:27.304 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.304 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.304 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946883+0000", 2024-09-16T14:49:27.304 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9494e-05 2024-09-16T14:49:27.304 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.304 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.304 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.304 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.305 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.305 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs6 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:27.305 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:27.305 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.330120046000001, 2024-09-16T14:49:27.305 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014400098, 2024-09-16T14:49:27.305 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.305 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.305 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.305 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.305 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.305 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:27.305 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.306 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.306 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.306 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:27.306 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.306 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.306 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.306 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.306 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932456+0000", 2024-09-16T14:49:27.306 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.306 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.306 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.306 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.306 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932475+0000", 2024-09-16T14:49:27.306 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8695999999999999e-05 2024-09-16T14:49:27.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.307 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.307 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932485+0000", 2024-09-16T14:49:27.307 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7799999999999995e-06 2024-09-16T14:49:27.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.307 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.307 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932492+0000", 2024-09-16T14:49:27.307 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5209999999999997e-06 2024-09-16T14:49:27.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.307 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.307 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932705+0000", 2024-09-16T14:49:27.308 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00021277700000000001 2024-09-16T14:49:27.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.308 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.308 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.308 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932721+0000", 2024-09-16T14:49:27.308 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6095000000000001e-05 2024-09-16T14:49:27.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.308 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.308 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.308 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946825+0000", 2024-09-16T14:49:27.308 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014103269999999999 2024-09-16T14:49:27.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.308 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.308 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.309 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946857+0000", 2024-09-16T14:49:27.309 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2271999999999999e-05 2024-09-16T14:49:27.309 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.309 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.309 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.309 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.309 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.309 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:27.309 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:27.309 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.329923196999999, 2024-09-16T14:49:27.309 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015300085999999999, 2024-09-16T14:49:27.309 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.309 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.309 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932653+0000", 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.311 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.311 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932674+0000", 2024-09-16T14:49:27.311 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1047e-05 2024-09-16T14:49:27.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.311 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.311 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932685+0000", 2024-09-16T14:49:27.311 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0733999999999999e-05 2024-09-16T14:49:27.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.311 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.311 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932693+0000", 2024-09-16T14:49:27.311 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0819999999999999e-06 2024-09-16T14:49:27.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.312 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.312 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945297+0000", 2024-09-16T14:49:27.312 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012604198 2024-09-16T14:49:27.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.312 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.312 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945306+0000", 2024-09-16T14:49:27.312 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1549999999999996e-06 2024-09-16T14:49:27.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.312 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.313 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947927+0000", 2024-09-16T14:49:27.313 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.002621295 2024-09-16T14:49:27.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.313 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.313 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947954+0000", 2024-09-16T14:49:27.313 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6219000000000001e-05 2024-09-16T14:49:27.313 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.313 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.313 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.313 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s3 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:27.314 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:27.314 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.329835468999999, 2024-09-16T14:49:27.314 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013712046, 2024-09-16T14:49:27.314 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.314 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.314 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.314 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.314 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.314 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:27.314 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.314 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.314 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.314 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.314 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:27.315 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.315 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.315 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.315 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:27.315 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.315 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.315 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.315 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932760+0000", 2024-09-16T14:49:27.315 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9247999999999999e-05 2024-09-16T14:49:27.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.315 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.315 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.316 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932773+0000", 2024-09-16T14:49:27.316 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3081000000000001e-05 2024-09-16T14:49:27.316 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.316 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.316 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932783+0000", 2024-09-16T14:49:27.316 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7850000000000003e-06 2024-09-16T14:49:27.316 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.316 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.316 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932830+0000", 2024-09-16T14:49:27.316 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7608999999999998e-05 2024-09-16T14:49:27.316 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.317 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.317 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932842+0000", 2024-09-16T14:49:27.317 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1082e-05 2024-09-16T14:49:27.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.317 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.317 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946430+0000", 2024-09-16T14:49:27.317 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013588141999999999 2024-09-16T14:49:27.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.317 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.317 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946453+0000", 2024-09-16T14:49:27.317 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3714999999999999e-05 2024-09-16T14:49:27.317 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.318 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.318 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.318 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.318 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.318 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:27.318 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:27.318 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.329420527, 2024-09-16T14:49:27.318 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013268299000000001, 2024-09-16T14:49:27.318 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.318 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.318 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.318 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.318 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.318 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933155+0000", 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.319 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933182+0000", 2024-09-16T14:49:27.320 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7090999999999999e-05 2024-09-16T14:49:27.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.320 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.320 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933202+0000", 2024-09-16T14:49:27.320 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9459000000000001e-05 2024-09-16T14:49:27.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.320 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.320 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933211+0000", 2024-09-16T14:49:27.320 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.397e-06 2024-09-16T14:49:27.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.320 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.321 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933284+0000", 2024-09-16T14:49:27.321 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2711000000000002e-05 2024-09-16T14:49:27.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.321 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.321 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.321 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933294+0000", 2024-09-16T14:49:27.321 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0079e-05 2024-09-16T14:49:27.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.321 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.321 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.321 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946408+0000", 2024-09-16T14:49:27.321 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013114097999999999 2024-09-16T14:49:27.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.322 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.322 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.322 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946425+0000", 2024-09-16T14:49:27.322 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6568000000000001e-05 2024-09-16T14:49:27.322 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.322 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.322 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.322 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.322 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.322 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs6 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:27.322 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:27.322 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.329203626000002, 2024-09-16T14:49:27.322 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014598939999999999, 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.323 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.324 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933372+0000", 2024-09-16T14:49:27.324 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:27.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.324 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.324 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.324 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933398+0000", 2024-09-16T14:49:27.324 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6129999999999999e-05 2024-09-16T14:49:27.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.324 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.324 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.324 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933414+0000", 2024-09-16T14:49:27.324 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5549999999999999e-05 2024-09-16T14:49:27.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.325 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.325 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933422+0000", 2024-09-16T14:49:27.325 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8620000000000003e-06 2024-09-16T14:49:27.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.325 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.325 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945189+0000", 2024-09-16T14:49:27.325 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011767552000000001 2024-09-16T14:49:27.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.325 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.325 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945200+0000", 2024-09-16T14:49:27.326 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0766e-05 2024-09-16T14:49:27.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.326 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.326 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.326 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947957+0000", 2024-09-16T14:49:27.326 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0027565319999999999 2024-09-16T14:49:27.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.326 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.326 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.326 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947972+0000", 2024-09-16T14:49:27.326 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5359999999999999e-05 2024-09-16T14:49:27.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.326 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.327 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.327 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs6 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:27.327 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:27.327 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.328197360000001, 2024-09-16T14:49:27.327 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010566212, 2024-09-16T14:49:27.327 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.327 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.327 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.327 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.327 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:27.327 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.328 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.328 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:27.328 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.328 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.328 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:27.328 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:27.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.328 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.328 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934728+0000", 2024-09-16T14:49:27.329 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034932000000000002 2024-09-16T14:49:27.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.329 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.329 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.329 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934739+0000", 2024-09-16T14:49:27.329 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1318000000000001e-05 2024-09-16T14:49:27.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.329 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.329 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.329 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934752+0000", 2024-09-16T14:49:27.329 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2901999999999999e-05 2024-09-16T14:49:27.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.329 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.330 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.330 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934823+0000", 2024-09-16T14:49:27.330 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1118000000000006e-05 2024-09-16T14:49:27.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.330 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.330 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.330 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934834+0000", 2024-09-16T14:49:27.330 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0591000000000001e-05 2024-09-16T14:49:27.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.330 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.330 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.330 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.944917+0000", 2024-09-16T14:49:27.330 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010082906000000001 2024-09-16T14:49:27.331 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.331 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.331 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.944946+0000", 2024-09-16T14:49:27.331 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8895999999999999e-05 2024-09-16T14:49:27.331 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.331 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.331 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.331 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.331 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 61/27 ECSubWrite(tid=237, reqid=osd.7.0:237, at_version=61'50, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:27.331 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:27.331 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.326997241000001, 2024-09-16T14:49:27.331 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011297481, 2024-09-16T14:49:27.332 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.332 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.332 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.332 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.332 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:27.332 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.332 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.332 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:27.332 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.333 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.333 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935579+0000", 2024-09-16T14:49:27.333 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:27.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.333 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.333 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935608+0000", 2024-09-16T14:49:27.333 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9578999999999999e-05 2024-09-16T14:49:27.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.333 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935625+0000", 2024-09-16T14:49:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7075000000000002e-05 2024-09-16T14:49:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935642+0000", 2024-09-16T14:49:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6480000000000001e-05 2024-09-16T14:49:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935874+0000", 2024-09-16T14:49:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00023162399999999999 2024-09-16T14:49:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.335 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.335 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935887+0000", 2024-09-16T14:49:27.335 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3811e-05 2024-09-16T14:49:27.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.335 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.335 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946860+0000", 2024-09-16T14:49:27.335 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010972552999999999 2024-09-16T14:49:27.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.335 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.335 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946877+0000", 2024-09-16T14:49:27.335 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7132000000000001e-05 2024-09-16T14:49:27.335 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.336 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.336 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.336 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 61/27 ECSubWrite(tid=231, reqid=osd.11.0:231, at_version=61'37, trim_to=0'0, roll_forward_to=60'33))", 2024-09-16T14:49:27.336 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:27.336 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.308748696999999, 2024-09-16T14:49:27.336 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012494261, 2024-09-16T14:49:27.336 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.336 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.336 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.336 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.336 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:27.337 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.337 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.337 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:27.337 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.337 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.337 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:27.337 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.337 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953840+0000", 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1983e-05 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953846+0000", 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.319e-06 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953852+0000", 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.823e-06 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.338 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.339 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954193+0000", 2024-09-16T14:49:27.339 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034066199999999999 2024-09-16T14:49:27.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.339 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.339 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954203+0000", 2024-09-16T14:49:27.339 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0421999999999999e-05 2024-09-16T14:49:27.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.339 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.339 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966300+0000", 2024-09-16T14:49:27.339 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012096519 2024-09-16T14:49:27.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.340 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.340 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966322+0000", 2024-09-16T14:49:27.340 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2829000000000001e-05 2024-09-16T14:49:27.340 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.340 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.340 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.340 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 61/27 ECSubWrite(tid=244, reqid=osd.3.0:243, at_version=0'0, trim_to=0'0, roll_forward_to=61'81))", 2024-09-16T14:49:27.340 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:27.340 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 19.308298427, 2024-09-16T14:49:27.340 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012038332000000001, 2024-09-16T14:49:27.340 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.341 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.341 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.341 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.341 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:27.341 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.341 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.341 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:27.341 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.341 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.342 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954277+0000", 2024-09-16T14:49:27.342 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:27.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.342 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.342 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954290+0000", 2024-09-16T14:49:27.342 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2364000000000001e-05 2024-09-16T14:49:27.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.342 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.342 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954302+0000", 2024-09-16T14:49:27.342 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2665e-05 2024-09-16T14:49:27.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.343 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.343 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954322+0000", 2024-09-16T14:49:27.343 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9429e-05 2024-09-16T14:49:27.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.343 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.343 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954374+0000", 2024-09-16T14:49:27.343 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2182000000000003e-05 2024-09-16T14:49:27.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.343 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.343 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954384+0000", 2024-09-16T14:49:27.343 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0022e-05 2024-09-16T14:49:27.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.344 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.344 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966295+0000", 2024-09-16T14:49:27.344 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011910399 2024-09-16T14:49:27.344 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.344 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.344 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966317+0000", 2024-09-16T14:49:27.344 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2169999999999999e-05 2024-09-16T14:49:27.344 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.344 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.344 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.344 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.345 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:688 3.6s0 3:608d2d89:::smithi04040151-48:head [copy-from ver 4] snapc 2e=[2e,2b,29,28,25] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T14:49:27.345 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:21.913077+0000", 2024-09-16T14:49:27.345 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.3494998149999997, 2024-09-16T14:49:27.345 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.023450634000000001, 2024-09-16T14:49:27.345 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.345 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:27.345 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:27.345 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:27.345 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:27.345 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 688 2024-09-16T14:49:27.345 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.345 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.345 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.345 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913077+0000", 2024-09-16T14:49:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913077+0000", 2024-09-16T14:49:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913075+0000", 2024-09-16T14:49:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.347 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.347 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913085+0000", 2024-09-16T14:49:27.347 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8810000000000008e-06 2024-09-16T14:49:27.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.347 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.347 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913099+0000", 2024-09-16T14:49:27.347 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.382e-05 2024-09-16T14:49:27.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.347 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.347 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913108+0000", 2024-09-16T14:49:27.347 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7390000000000005e-06 2024-09-16T14:49:27.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913174+0000", 2024-09-16T14:49:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6598000000000002e-05 2024-09-16T14:49:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913288+0000", 2024-09-16T14:49:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011353000000000001 2024-09-16T14:49:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.348 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.929027+0000", 2024-09-16T14:49:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015739758999999999 2024-09-16T14:49:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.931900+0000", 2024-09-16T14:49:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0028725310000000002 2024-09-16T14:49:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.936419+0000", 2024-09-16T14:49:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0045194340000000001 2024-09-16T14:49:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.349 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.936528+0000", 2024-09-16T14:49:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010822199999999999 2024-09-16T14:49:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:689 3.6s0 3:608d2d89:::smithi04040151-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-09-16T14:49:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 5.3494516829999998, 2024-09-16T14:49:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.018229170999999999, 2024-09-16T14:49:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:27.350 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:27.351 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:27.351 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:27.351 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 689 2024-09-16T14:49:27.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.351 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.351 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.351 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:27.351 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.351 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.351 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:27.352 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.352 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.352 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:27.352 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.352 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.352 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913128+0000", 2024-09-16T14:49:27.352 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.4580000000000002e-06 2024-09-16T14:49:27.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.352 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913137+0000", 2024-09-16T14:49:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6039999999999993e-06 2024-09-16T14:49:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913145+0000", 2024-09-16T14:49:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6200000000000005e-06 2024-09-16T14:49:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913491+0000", 2024-09-16T14:49:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034584200000000001 2024-09-16T14:49:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.354 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:27.354 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913536+0000", 2024-09-16T14:49:27.354 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.5269000000000003e-05 2024-09-16T14:49:27.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.354 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.354 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.930844+0000", 2024-09-16T14:49:27.354 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.017307802000000001 2024-09-16T14:49:27.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.355 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:27.355 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.930954+0000", 2024-09-16T14:49:27.355 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011033500000000001 2024-09-16T14:49:27.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.355 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.355 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.931354+0000", 2024-09-16T14:49:27.355 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00039983500000000002 2024-09-16T14:49:27.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.355 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.356 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 77/27 ECSubWrite(tid=311, reqid=osd.3.0:311, at_version=77'100, trim_to=0'0, roll_forward_to=76'98))", 2024-09-16T14:49:27.356 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.915984+0000", 2024-09-16T14:49:27.356 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 3.3465926110000002, 2024-09-16T14:49:27.356 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010184956, 2024-09-16T14:49:27.356 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.356 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.356 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.356 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.356 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.915984+0000", 2024-09-16T14:49:27.356 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.357 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.357 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.915984+0000", 2024-09-16T14:49:27.357 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.357 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.357 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.915984+0000", 2024-09-16T14:49:27.357 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.357 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.357 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916003+0000", 2024-09-16T14:49:27.357 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9877999999999998e-05 2024-09-16T14:49:27.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.358 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.358 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916014+0000", 2024-09-16T14:49:27.358 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:49:27.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.358 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.358 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916022+0000", 2024-09-16T14:49:27.358 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8930000000000005e-06 2024-09-16T14:49:27.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.358 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.358 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916084+0000", 2024-09-16T14:49:27.359 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1385999999999998e-05 2024-09-16T14:49:27.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.359 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.359 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916095+0000", 2024-09-16T14:49:27.359 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0852999999999999e-05 2024-09-16T14:49:27.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.359 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.359 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.926132+0000", 2024-09-16T14:49:27.359 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010037842999999999 2024-09-16T14:49:27.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.360 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.360 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.926169+0000", 2024-09-16T14:49:27.360 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6777999999999999e-05 2024-09-16T14:49:27.360 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.360 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.360 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.360 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.360 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.360 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:27.361 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:27.359+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:27.363 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:49:27.463 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:27.461+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:27.470 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:27.470 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:27.470 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:27.470 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:27.479 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:49:27.564 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:27.562+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:27.585 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:27.585 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:27.585 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:27.585 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:27.585 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:27.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:49:27.665 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:27.662+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:27.702 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:27.702 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:27.702 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:27.702 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:27.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.703 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.2s5 37/27 ECSubWrite(tid=123, reqid=osd.3.0:123, at_version=37'38, trim_to=0'0, roll_forward_to=37'34))", 2024-09-16T14:49:27.703 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:27.703 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 43.764039971999999, 2024-09-16T14:49:27.703 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0094962529999999996, 2024-09-16T14:49:27.703 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.703 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.703 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.703 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.703 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:27.703 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.704 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.704 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:27.704 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.704 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.704 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935902+0000", 2024-09-16T14:49:27.704 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:27.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.704 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.704 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935938+0000", 2024-09-16T14:49:27.705 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5731000000000001e-05 2024-09-16T14:49:27.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.705 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.705 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935949+0000", 2024-09-16T14:49:27.705 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1229999999999999e-05 2024-09-16T14:49:27.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.705 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.705 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935957+0000", 2024-09-16T14:49:27.705 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.473e-06 2024-09-16T14:49:27.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.706 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.706 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.936913+0000", 2024-09-16T14:49:27.706 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00095599900000000002 2024-09-16T14:49:27.706 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.706 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.706 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.936923+0000", 2024-09-16T14:49:27.706 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4059999999999994e-06 2024-09-16T14:49:27.706 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.706 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.707 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945371+0000", 2024-09-16T14:49:27.707 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084484279999999991 2024-09-16T14:49:27.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.707 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.707 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.707 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945399+0000", 2024-09-16T14:49:27.707 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7716e-05 2024-09-16T14:49:27.707 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.707 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.707 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.708 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.708 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.708 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.0s3 37/27 ECSubWrite(tid=109, reqid=osd.11.0:109, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:27.708 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:27.708 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 43.759493352, 2024-09-16T14:49:27.708 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090738399999999997, 2024-09-16T14:49:27.708 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.708 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.708 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.708 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.708 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.709 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:27.709 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.709 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.709 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.709 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.709 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:27.709 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.709 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.709 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.709 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940449+0000", 2024-09-16T14:49:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940467+0000", 2024-09-16T14:49:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8113000000000001e-05 2024-09-16T14:49:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940475+0000", 2024-09-16T14:49:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8499999999999994e-06 2024-09-16T14:49:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.710 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940481+0000", 2024-09-16T14:49:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0979999999999999e-06 2024-09-16T14:49:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940569+0000", 2024-09-16T14:49:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8395999999999999e-05 2024-09-16T14:49:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.940577+0000", 2024-09-16T14:49:27.711 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7500000000000003e-06 2024-09-16T14:49:27.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.712 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.712 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.712 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949500+0000", 2024-09-16T14:49:27.712 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089230319999999991 2024-09-16T14:49:27.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.712 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.712 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.712 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.949523+0000", 2024-09-16T14:49:27.713 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3082e-05 2024-09-16T14:49:27.713 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.713 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.713 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.713 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.713 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.713 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:302 3.4s0 3:255172f3:::smithi04040151-3:head [create,setxattr qPd\\P_iff[nbYodrbXe[dQ_okmVWqe[RceaqQRNm (630) in=40b,setxattr NWYlZi[dPXnppNo]`fatPqfSaTlsQVetOmrQm^ap (138) in=40b,setxattr iVfcaeRT]TQppeYbq_`lRobpUsaVcWXphplasbgY (1507) in=40b,setxattr YeaeOf[jt`g\\k^]VfXTjOONb`jT[WYUgdiglVjYl (8841) in=40b,setxattr [iqZhSWl_sclXkajVYTXlOX]eeeWkPi\\dOYYt_Wb (1928) in=40b,setxattr \\[gaSPWYO`NQXpTccfbk`SZPaU`lbpTk]a] (1262) in=35b,setxattr ]\\]U^niYqQmjcgO[dsO`YpRZeRV]`X]bU[X^_Qis (1086) in=40b,setxattr VafomYY]aa`a^pbWebSRaithSn_dQjQYq]aTarm\\ (1402) in=40b,setxattr acj^VZhfaaikXrXq]_haYqkl[^_atWUTaloiVj`a (105) in=40b,setxattr VNUlaeZOglfodan^Ycb`dqVeZhSTmO_rabViYVch (70) in=40b,setxattr Qka\\PoUoghRc\\Qcnl`kVbj[rs\\hSdVXmlNqha`ta (594) in=40b,setxattr qsSQhb_apjdTlikdpj\\maaVdS[mVfsh_Vqi[ogWo (771) in=40b,setxattr Tm]oi_Rf_V`pSY[VhYSXSQN^bqjtgmm^fUo^f\\ZV (1087) in=40b,setxattr \\dTq (1262) in=4b,setxattr nettWfThbXj]cUpsYrTjZ^qk^\\[ZoZ[qb]Sf[ZbN (1684) in=40b,setxattr Sa\\hrdN^YombnT\\torZ`N]aRqicS_lP^XgP`e`Oc (1963) in=40b,setxattr dWhoTgYPSTh]jfWhPTs^iVdYbtfhi_WjOdes]NXh (1051) in=40b,setxattr Qbp_QrhXQf`O]blcqeqgRchO]RrhbshoV\\cW[_es (10033) in=40b,setxattr WYbQqgdRjmdbeoaP]U]n]PTWOYkP^lWOanNTnaOr (1998) in=40b,setxattr Rfbdg`[^ZYrN]cl][Ypr^tpXmb_mbh[gf\\idocW] (10804) in=40b] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:27.713 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:27.713 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 42.767527575000003, 2024-09-16T14:49:27.713 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.009468509, 2024-09-16T14:49:27.713 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.713 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:27.714 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:27.714 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:27.714 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:27.714 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 302 2024-09-16T14:49:27.714 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.714 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.714 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.714 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.714 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:27.714 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.714 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.714 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.715 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.715 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:27.715 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.715 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.715 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.715 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.715 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932413+0000", 2024-09-16T14:49:27.715 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:27.715 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.715 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.715 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.715 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932459+0000", 2024-09-16T14:49:27.715 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6414e-05 2024-09-16T14:49:27.715 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.716 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.716 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.716 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932491+0000", 2024-09-16T14:49:27.716 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1656000000000003e-05 2024-09-16T14:49:27.716 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.716 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.716 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.716 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932499+0000", 2024-09-16T14:49:27.716 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4940000000000003e-06 2024-09-16T14:49:27.716 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.716 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.716 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.716 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932557+0000", 2024-09-16T14:49:27.717 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8044000000000002e-05 2024-09-16T14:49:27.717 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.717 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.717 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:27.717 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932704+0000", 2024-09-16T14:49:27.717 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00014615 2024-09-16T14:49:27.717 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.717 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.717 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.717 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.934021+0000", 2024-09-16T14:49:27.717 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001317052 2024-09-16T14:49:27.717 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.717 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.718 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.718 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941512+0000", 2024-09-16T14:49:27.718 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074917789999999996 2024-09-16T14:49:27.718 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.718 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.718 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:27.718 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941804+0000", 2024-09-16T14:49:27.718 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029206099999999998 2024-09-16T14:49:27.718 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.718 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.718 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.718 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941884+0000", 2024-09-16T14:49:27.718 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9231000000000001e-05 2024-09-16T14:49:27.718 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.719 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.719 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.719 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.719 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.719 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:365 3.4s0 3:2b1464a3:::smithi04040151-28:head [copy-from ver 15] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:27.719 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:27.719 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 38.760674178000002, 2024-09-16T14:49:27.719 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.019409261000000001, 2024-09-16T14:49:27.719 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.719 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:27.719 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:27.719 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:27.719 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:27.720 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 365 2024-09-16T14:49:27.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.720 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.720 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.720 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.720 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:27.720 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.720 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.720 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.720 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:27.720 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.721 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.721 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.721 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939267+0000", 2024-09-16T14:49:27.721 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:27.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.721 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.721 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.721 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939278+0000", 2024-09-16T14:49:27.721 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.093e-05 2024-09-16T14:49:27.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.721 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.721 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.721 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939295+0000", 2024-09-16T14:49:27.721 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7161000000000001e-05 2024-09-16T14:49:27.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.722 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.722 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939304+0000", 2024-09-16T14:49:27.722 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.329e-06 2024-09-16T14:49:27.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.722 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.722 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939378+0000", 2024-09-16T14:49:27.722 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3456999999999998e-05 2024-09-16T14:49:27.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.722 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:27.723 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939387+0000", 2024-09-16T14:49:27.723 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1449999999999997e-06 2024-09-16T14:49:27.723 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.723 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.723 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.723 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.941885+0000", 2024-09-16T14:49:27.723 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0024978349999999999 2024-09-16T14:49:27.723 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.723 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.723 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:27.723 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942001+0000", 2024-09-16T14:49:27.723 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011674499999999999 2024-09-16T14:49:27.723 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.724 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.724 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.724 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.953426+0000", 2024-09-16T14:49:27.724 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011424808999999999 2024-09-16T14:49:27.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.724 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.724 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.724 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956855+0000", 2024-09-16T14:49:27.724 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0034292160000000001 2024-09-16T14:49:27.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.724 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.724 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:27.724 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.958596+0000", 2024-09-16T14:49:27.724 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001740969 2024-09-16T14:49:27.725 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.725 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.725 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.725 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.958678+0000", 2024-09-16T14:49:27.725 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1592000000000001e-05 2024-09-16T14:49:27.725 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.725 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.725 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.725 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.725 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.725 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:366 3.4s0 3:2b1464a3:::smithi04040151-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:27.725 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:27.726 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 38.760624608000001, 2024-09-16T14:49:27.726 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.017101175, 2024-09-16T14:49:27.726 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.726 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:27.726 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:27.726 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:27.726 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:27.726 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 366 2024-09-16T14:49:27.726 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.726 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.726 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.726 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.726 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:27.726 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.727 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.727 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.727 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.727 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:27.727 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.727 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.727 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.727 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.727 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:27.727 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.727 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.727 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.727 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.727 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939321+0000", 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.49e-06 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939331+0000", 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.006e-05 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939336+0000", 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2959999999999998e-06 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.728 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939454+0000", 2024-09-16T14:49:27.729 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000117726 2024-09-16T14:49:27.729 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.729 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.729 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:27.729 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939461+0000", 2024-09-16T14:49:27.729 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1940000000000001e-06 2024-09-16T14:49:27.729 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.729 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.729 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.729 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942256+0000", 2024-09-16T14:49:27.729 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0027949350000000001 2024-09-16T14:49:27.730 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.730 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.730 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:27.730 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942325+0000", 2024-09-16T14:49:27.730 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.9119e-05 2024-09-16T14:49:27.730 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.730 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.730 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.730 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956179+0000", 2024-09-16T14:49:27.730 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013853343000000001 2024-09-16T14:49:27.731 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.731 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.731 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:27.731 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956295+0000", 2024-09-16T14:49:27.731 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000115899 2024-09-16T14:49:27.731 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.731 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.731 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.731 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956419+0000", 2024-09-16T14:49:27.731 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012467399999999999 2024-09-16T14:49:27.731 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.732 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.732 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.732 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.732 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.732 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:391 3.4s0 3:20ea1aa7:::smithi04040151-17:head [append 0~0] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:27.732 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:27.732 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.835412562000002, 2024-09-16T14:49:27.732 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011589693999999999, 2024-09-16T14:49:27.732 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.732 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:27.733 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:27.733 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:27.733 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:27.733 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 391 2024-09-16T14:49:27.733 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.733 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.733 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.733 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.733 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:27.734 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.734 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.734 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.734 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.734 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:27.734 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.734 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.734 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.734 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.734 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864528+0000", 2024-09-16T14:49:27.735 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:27.735 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.735 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.735 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.735 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864540+0000", 2024-09-16T14:49:27.735 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2514000000000001e-05 2024-09-16T14:49:27.735 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.735 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.735 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.735 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864569+0000", 2024-09-16T14:49:27.735 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8716000000000001e-05 2024-09-16T14:49:27.736 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.736 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.736 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.736 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864580+0000", 2024-09-16T14:49:27.736 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1564e-05 2024-09-16T14:49:27.736 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.736 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.736 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.736 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864681+0000", 2024-09-16T14:49:27.736 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010063100000000001 2024-09-16T14:49:27.736 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.736 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.736 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:27.737 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864690+0000", 2024-09-16T14:49:27.737 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6300000000000004e-06 2024-09-16T14:49:27.737 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.737 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.737 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.737 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.869283+0000", 2024-09-16T14:49:27.737 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.004593186 2024-09-16T14:49:27.737 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.737 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.738 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:27.738 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.869365+0000", 2024-09-16T14:49:27.738 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.174e-05 2024-09-16T14:49:27.738 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.738 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.738 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.738 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870206+0000", 2024-09-16T14:49:27.738 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00084176100000000001 2024-09-16T14:49:27.738 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.738 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.739 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.739 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.873261+0000", 2024-09-16T14:49:27.739 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0030546549999999999 2024-09-16T14:49:27.739 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.739 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.739 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:27.739 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.876069+0000", 2024-09-16T14:49:27.739 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0028082210000000001 2024-09-16T14:49:27.739 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.739 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.740 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.740 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.876120+0000", 2024-09-16T14:49:27.740 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.0581999999999998e-05 2024-09-16T14:49:27.740 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.740 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.740 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.740 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.740 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.740 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:392 3.4s0 3:20ea1aa7:::smithi04040151-17:head [rollback c] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:27.740 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:27.740 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.835321981999996, 2024-09-16T14:49:27.741 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013845265000000001, 2024-09-16T14:49:27.741 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.741 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:27.741 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:27.741 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:27.741 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:27.741 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 392 2024-09-16T14:49:27.741 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.741 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.741 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.742 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.742 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:27.742 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.742 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.742 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.742 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.742 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:27.742 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.742 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.742 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.742 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.743 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864620+0000", 2024-09-16T14:49:27.743 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:27.743 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.743 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.743 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.743 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864626+0000", 2024-09-16T14:49:27.743 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6699999999999999e-06 2024-09-16T14:49:27.743 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.743 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.743 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.743 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864643+0000", 2024-09-16T14:49:27.743 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7404000000000001e-05 2024-09-16T14:49:27.743 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.744 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.744 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.744 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864651+0000", 2024-09-16T14:49:27.744 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0860000000000002e-06 2024-09-16T14:49:27.744 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.744 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.744 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.744 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864759+0000", 2024-09-16T14:49:27.744 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010751000000000001 2024-09-16T14:49:27.744 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.744 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.744 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:27.744 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864765+0000", 2024-09-16T14:49:27.745 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.737e-06 2024-09-16T14:49:27.745 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.745 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.745 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.745 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870832+0000", 2024-09-16T14:49:27.745 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060664930000000001 2024-09-16T14:49:27.745 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.745 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.745 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:27.745 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870972+0000", 2024-09-16T14:49:27.745 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00014033599999999999 2024-09-16T14:49:27.745 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.745 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.745 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.871637+0000", 2024-09-16T14:49:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00066502800000000002 2024-09-16T14:49:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.873263+0000", 2024-09-16T14:49:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0016256409999999999 2024-09-16T14:49:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.878403+0000", 2024-09-16T14:49:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0051401600000000004 2024-09-16T14:49:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.746 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.747 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.747 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.878466+0000", 2024-09-16T14:49:27.747 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.2964000000000002e-05 2024-09-16T14:49:27.747 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.747 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.747 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.747 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.747 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.747 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:393 3.4s0 3:20ea1aa7:::smithi04040151-17:head [append 0~0] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:27.747 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:27.747 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.835262387999997, 2024-09-16T14:49:27.747 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.016150277000000001, 2024-09-16T14:49:27.747 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.748 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:27.748 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:27.748 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:27.748 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:27.748 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 393 2024-09-16T14:49:27.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.748 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.748 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.748 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:27.748 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.749 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.749 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:27.749 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.749 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.749 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:27.749 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.749 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.749 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864688+0000", 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5259999999999993e-06 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864705+0000", 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6368999999999999e-05 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864712+0000", 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4749999999999996e-06 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.750 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864791+0000", 2024-09-16T14:49:27.751 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9159000000000006e-05 2024-09-16T14:49:27.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.751 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:27.751 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864797+0000", 2024-09-16T14:49:27.751 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5400000000000003e-06 2024-09-16T14:49:27.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.751 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.751 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872109+0000", 2024-09-16T14:49:27.751 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073124569999999996 2024-09-16T14:49:27.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.751 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:27.752 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872237+0000", 2024-09-16T14:49:27.752 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012735099999999999 2024-09-16T14:49:27.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.752 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.752 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872865+0000", 2024-09-16T14:49:27.752 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00062808000000000004 2024-09-16T14:49:27.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.752 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.752 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.874081+0000", 2024-09-16T14:49:27.752 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001216186 2024-09-16T14:49:27.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.753 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:27.753 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.880752+0000", 2024-09-16T14:49:27.753 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0066711339999999996 2024-09-16T14:49:27.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.753 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.753 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.880831+0000", 2024-09-16T14:49:27.753 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8681000000000001e-05 2024-09-16T14:49:27.753 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.753 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.753 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.754 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.9s5 51/27 ECSubWrite(tid=35, reqid=osd.8.0:35, at_version=51'21, trim_to=0'0, roll_forward_to=47'19))", 2024-09-16T14:49:27.754 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:27.754 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.810817135000001, 2024-09-16T14:49:27.754 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010449557999999999, 2024-09-16T14:49:27.754 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.754 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.754 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.754 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.754 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:27.754 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889125+0000", 2024-09-16T14:49:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.755 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889142+0000", 2024-09-16T14:49:27.756 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7186000000000001e-05 2024-09-16T14:49:27.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.756 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.756 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889152+0000", 2024-09-16T14:49:27.756 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7920000000000004e-06 2024-09-16T14:49:27.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.756 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.756 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889159+0000", 2024-09-16T14:49:27.756 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.0319999999999998e-06 2024-09-16T14:49:27.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.757 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.757 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889203+0000", 2024-09-16T14:49:27.757 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.4264000000000002e-05 2024-09-16T14:49:27.757 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.757 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.757 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.757 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889212+0000", 2024-09-16T14:49:27.757 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6940000000000001e-06 2024-09-16T14:49:27.757 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.757 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.757 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.758 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.899537+0000", 2024-09-16T14:49:27.758 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010325328 2024-09-16T14:49:27.758 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.758 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.759 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.759 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.899575+0000", 2024-09-16T14:49:27.759 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.7741999999999999e-05 2024-09-16T14:49:27.759 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.759 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.759 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.759 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.759 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.759 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s5 51/27 ECSubWrite(tid=60, reqid=client.4500.0:467, at_version=51'19, trim_to=0'0, roll_forward_to=51'16))", 2024-09-16T14:49:27.759 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:27.759 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.70201793, 2024-09-16T14:49:27.759 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0099497439999999999, 2024-09-16T14:49:27.760 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.760 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.760 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.760 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.760 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.760 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:27.760 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.760 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.760 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.760 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.760 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:27.760 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.760 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.761 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.761 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.761 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997924+0000", 2024-09-16T14:49:27.761 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:27.761 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.761 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.761 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.761 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997944+0000", 2024-09-16T14:49:27.761 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0404e-05 2024-09-16T14:49:27.761 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.761 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.761 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.761 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997957+0000", 2024-09-16T14:49:27.762 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2123000000000001e-05 2024-09-16T14:49:27.762 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.762 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.762 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.762 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997966+0000", 2024-09-16T14:49:27.762 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6549999999999999e-06 2024-09-16T14:49:27.762 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.762 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.762 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.762 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002158+0000", 2024-09-16T14:49:27.762 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0041915930000000004 2024-09-16T14:49:27.762 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.762 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.763 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.763 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002168+0000", 2024-09-16T14:49:27.763 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0071000000000001e-05 2024-09-16T14:49:27.763 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.763 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.764 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.764 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007845+0000", 2024-09-16T14:49:27.764 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056765690000000002 2024-09-16T14:49:27.764 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.764 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.764 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.764 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007875+0000", 2024-09-16T14:49:27.764 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0077e-05 2024-09-16T14:49:27.764 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.764 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.764 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.765 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.765 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.765 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:506 3.4s0 3:255172f3:::smithi04040151-3:head [rollback 13] snapc 1b=[1b,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-16T14:49:27.765 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:27.765 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.805580299999999, 2024-09-16T14:49:27.765 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0092517710000000006, 2024-09-16T14:49:27.765 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.766 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:27.766 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:27.766 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:27.766 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:27.766 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 506 2024-09-16T14:49:27.766 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.766 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.766 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.767 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.767 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:27.767 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.767 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.767 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.767 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.767 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:27.767 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.768 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.768 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.768 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.768 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:27.768 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.768 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.768 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.769 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.769 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894365+0000", 2024-09-16T14:49:27.769 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.571e-06 2024-09-16T14:49:27.769 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.769 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.769 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.769 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894370+0000", 2024-09-16T14:49:27.770 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8279999999999999e-06 2024-09-16T14:49:27.770 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.770 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.770 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.770 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894374+0000", 2024-09-16T14:49:27.770 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5920000000000001e-06 2024-09-16T14:49:27.770 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.771 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.771 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.771 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894483+0000", 2024-09-16T14:49:27.771 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010895699999999999 2024-09-16T14:49:27.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.772 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:27.772 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894491+0000", 2024-09-16T14:49:27.772 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9389999999999997e-06 2024-09-16T14:49:27.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.773 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.773 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.773 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.897713+0000", 2024-09-16T14:49:27.773 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0032221060000000002 2024-09-16T14:49:27.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.773 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.773 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:27.774 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.897829+0000", 2024-09-16T14:49:27.774 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000116082 2024-09-16T14:49:27.774 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.774 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.774 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.898559+0000", 2024-09-16T14:49:27.775 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00072999800000000004 2024-09-16T14:49:27.775 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.775 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.775 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.775 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.901829+0000", 2024-09-16T14:49:27.775 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0032702159999999998 2024-09-16T14:49:27.775 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.775 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.775 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903546+0000", 2024-09-16T14:49:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001716827 2024-09-16T14:49:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903614+0000", 2024-09-16T14:49:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8058999999999993e-05 2024-09-16T14:49:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.776 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:507 3.4s0 3:255172f3:::smithi04040151-3:head [append 0~0] snapc 1b=[1b,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-16T14:49:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.805548325, 2024-09-16T14:49:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0095393700000000001, 2024-09-16T14:49:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 507 2024-09-16T14:49:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.777 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.778 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894397+0000", 2024-09-16T14:49:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.774e-06 2024-09-16T14:49:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894403+0000", 2024-09-16T14:49:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1079999999999998e-06 2024-09-16T14:49:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894406+0000", 2024-09-16T14:49:27.779 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.337e-06 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894536+0000", 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012969199999999999 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894545+0000", 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7499999999999992e-06 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899040+0000", 2024-09-16T14:49:27.780 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044951289999999996 2024-09-16T14:49:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899164+0000", 2024-09-16T14:49:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012377699999999999 2024-09-16T14:49:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899721+0000", 2024-09-16T14:49:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00055683800000000004 2024-09-16T14:49:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.781 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.901911+0000", 2024-09-16T14:49:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0021908650000000002 2024-09-16T14:49:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903859+0000", 2024-09-16T14:49:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001947383 2024-09-16T14:49:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903934+0000", 2024-09-16T14:49:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5056999999999996e-05 2024-09-16T14:49:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.782 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:588 3.4s0 3:255172f3:::smithi04040151-3:head [append 0~0] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:11.975961+0000", 2024-09-16T14:49:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.723981835, 2024-09-16T14:49:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0258510810000001, 2024-09-16T14:49:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 588 2024-09-16T14:49:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.783 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975961+0000", 2024-09-16T14:49:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975961+0000", 2024-09-16T14:49:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.784 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975959+0000", 2024-09-16T14:49:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975968+0000", 2024-09-16T14:49:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1020000000000003e-06 2024-09-16T14:49:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975982+0000", 2024-09-16T14:49:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.33e-05 2024-09-16T14:49:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.785 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.786 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975990+0000", 2024-09-16T14:49:27.786 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4679999999999992e-06 2024-09-16T14:49:27.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.786 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.786 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976080+0000", 2024-09-16T14:49:27.786 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9642999999999995e-05 2024-09-16T14:49:27.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.786 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for peered", 2024-09-16T14:49:27.786 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976083+0000", 2024-09-16T14:49:27.786 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.92e-06 2024-09-16T14:49:27.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.787 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.787 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.983734+0000", 2024-09-16T14:49:27.787 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.007651056 2024-09-16T14:49:27.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.787 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:27.787 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.984659+0000", 2024-09-16T14:49:27.787 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000925493 2024-09-16T14:49:27.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.787 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.787 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.991407+0000", 2024-09-16T14:49:27.787 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0067480969999999998 2024-09-16T14:49:27.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.788 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:27.788 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.991550+0000", 2024-09-16T14:49:27.788 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00014299700000000001 2024-09-16T14:49:27.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.788 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.788 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.992634+0000", 2024-09-16T14:49:27.788 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001083894 2024-09-16T14:49:27.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.788 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.788 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.998056+0000", 2024-09-16T14:49:27.789 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0054220070000000004 2024-09-16T14:49:27.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.789 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:27.789 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.001429+0000", 2024-09-16T14:49:27.789 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0033727480000000001 2024-09-16T14:49:27.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.789 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.789 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.001812+0000", 2024-09-16T14:49:27.789 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00038303600000000001 2024-09-16T14:49:27.789 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.789 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.789 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.790 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.790 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:589 3.4s0 3:255172f3:::smithi04040151-3:head [rollback 24] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:27.790 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:11.976008+0000", 2024-09-16T14:49:27.790 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.723934890000001, 2024-09-16T14:49:27.790 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.027673222, 2024-09-16T14:49:27.790 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.790 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:27.790 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:27.790 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:27.790 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:27.790 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 589 2024-09-16T14:49:27.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.791 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.791 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.791 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976008+0000", 2024-09-16T14:49:27.791 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.791 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.791 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976008+0000", 2024-09-16T14:49:27.791 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.792 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.792 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976007+0000", 2024-09-16T14:49:27.792 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.792 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.792 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976012+0000", 2024-09-16T14:49:27.792 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9760000000000003e-06 2024-09-16T14:49:27.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.792 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.793 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976025+0000", 2024-09-16T14:49:27.793 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2884000000000001e-05 2024-09-16T14:49:27.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.793 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.793 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976031+0000", 2024-09-16T14:49:27.793 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0079999999999999e-06 2024-09-16T14:49:27.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.793 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.793 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976122+0000", 2024-09-16T14:49:27.793 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0946000000000006e-05 2024-09-16T14:49:27.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for peered", 2024-09-16T14:49:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976123+0000", 2024-09-16T14:49:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1489999999999999e-06 2024-09-16T14:49:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.985902+0000", 2024-09-16T14:49:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.009778528 2024-09-16T14:49:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.986100+0000", 2024-09-16T14:49:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00019825499999999999 2024-09-16T14:49:27.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.993202+0000", 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071018089999999997 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.993327+0000", 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000125031 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.994109+0000", 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00078209799999999995 2024-09-16T14:49:27.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.796 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.796 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.796 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.999045+0000", 2024-09-16T14:49:27.796 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0049362879999999996 2024-09-16T14:49:27.796 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.796 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.796 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:27.796 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.003322+0000", 2024-09-16T14:49:27.796 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0042765879999999996 2024-09-16T14:49:27.796 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.796 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.796 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.796 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.003681+0000", 2024-09-16T14:49:27.796 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00035919199999999999 2024-09-16T14:49:27.797 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.797 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.797 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.797 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.797 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.797 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:590 3.4s0 3:255172f3:::smithi04040151-3:head [append 0~0] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:27.797 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:11.976073+0000", 2024-09-16T14:49:27.797 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.723870012000001, 2024-09-16T14:49:27.797 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0282184670000001, 2024-09-16T14:49:27.797 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.797 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:27.797 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:27.797 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:27.797 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:27.798 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 590 2024-09-16T14:49:27.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.798 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.798 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.798 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.798 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976073+0000", 2024-09-16T14:49:27.798 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.798 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.798 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.798 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976073+0000", 2024-09-16T14:49:27.798 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.798 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.799 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.799 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976072+0000", 2024-09-16T14:49:27.799 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.799 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.799 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.799 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.799 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976076+0000", 2024-09-16T14:49:27.799 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.985e-06 2024-09-16T14:49:27.799 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.799 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.799 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.799 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976084+0000", 2024-09-16T14:49:27.799 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0080000000000002e-06 2024-09-16T14:49:27.799 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.800 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.800 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.800 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976095+0000", 2024-09-16T14:49:27.800 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0413e-05 2024-09-16T14:49:27.800 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.800 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.800 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.800 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976150+0000", 2024-09-16T14:49:27.800 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5588999999999999e-05 2024-09-16T14:49:27.800 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.800 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.800 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for peered", 2024-09-16T14:49:27.800 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976151+0000", 2024-09-16T14:49:27.801 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8700000000000004e-07 2024-09-16T14:49:27.801 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.801 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.801 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.801 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.986165+0000", 2024-09-16T14:49:27.801 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.010013982 2024-09-16T14:49:27.801 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.801 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.801 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:27.801 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.986260+0000", 2024-09-16T14:49:27.801 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4545999999999999e-05 2024-09-16T14:49:27.801 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.801 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.801 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.802 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.994573+0000", 2024-09-16T14:49:27.802 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0083134779999999991 2024-09-16T14:49:27.802 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.802 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.802 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:27.802 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.994693+0000", 2024-09-16T14:49:27.802 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000119707 2024-09-16T14:49:27.802 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.802 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.802 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.802 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.995284+0000", 2024-09-16T14:49:27.802 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00059083400000000004 2024-09-16T14:49:27.802 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.802 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.999048+0000", 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.003763878 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.003931+0000", 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0048833339999999996 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.004291+0000", 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00036028999999999998 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.803 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.804 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.804 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.804 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.804 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.as2 77/72 ECSubWrite(tid=12, reqid=osd.6.0:12, at_version=77'32, trim_to=0'0, roll_forward_to=74'30))", 2024-09-16T14:49:27.804 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.903509+0000", 2024-09-16T14:49:27.804 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 3.7964339969999998, 2024-09-16T14:49:27.804 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010698673000000001, 2024-09-16T14:49:27.804 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.804 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.804 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.804 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.804 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.804 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903509+0000", 2024-09-16T14:49:27.805 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.805 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.805 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.805 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903509+0000", 2024-09-16T14:49:27.805 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.805 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.805 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.805 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903508+0000", 2024-09-16T14:49:27.805 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.805 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.805 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.806 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903532+0000", 2024-09-16T14:49:27.806 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3799999999999999e-05 2024-09-16T14:49:27.806 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.806 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.806 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.806 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903542+0000", 2024-09-16T14:49:27.806 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9459999999999993e-06 2024-09-16T14:49:27.806 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.806 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.806 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.806 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903550+0000", 2024-09-16T14:49:27.806 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7989999999999994e-06 2024-09-16T14:49:27.806 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.806 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903601+0000", 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1479e-05 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903610+0000", 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0669999999999998e-06 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.914176+0000", 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010566048999999999 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.807 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.808 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.808 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.914207+0000", 2024-09-16T14:49:27.808 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0973000000000003e-05 2024-09-16T14:49:27.808 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.808 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.808 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.808 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.808 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.808 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.bs6 77/65 ECSubWrite(tid=308, reqid=osd.3.0:308, at_version=77'53, trim_to=0'0, roll_forward_to=74'51))", 2024-09-16T14:49:27.808 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.904842+0000", 2024-09-16T14:49:27.808 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 3.795100331, 2024-09-16T14:49:27.808 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011875419, 2024-09-16T14:49:27.808 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.808 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.809 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.809 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.809 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.809 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904842+0000", 2024-09-16T14:49:27.809 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.809 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.809 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.809 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904842+0000", 2024-09-16T14:49:27.809 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.809 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.809 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.809 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904842+0000", 2024-09-16T14:49:27.810 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:27.810 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.810 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.810 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904866+0000", 2024-09-16T14:49:27.810 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3557000000000002e-05 2024-09-16T14:49:27.810 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.810 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.810 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904878+0000", 2024-09-16T14:49:27.810 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2320999999999999e-05 2024-09-16T14:49:27.810 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.810 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.811 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904892+0000", 2024-09-16T14:49:27.811 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4042e-05 2024-09-16T14:49:27.811 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.811 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.811 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904956+0000", 2024-09-16T14:49:27.811 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4026e-05 2024-09-16T14:49:27.811 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.811 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.811 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904966+0000", 2024-09-16T14:49:27.811 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0128e-05 2024-09-16T14:49:27.811 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.812 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.812 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916682+0000", 2024-09-16T14:49:27.812 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011715316999999999 2024-09-16T14:49:27.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.812 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.812 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.812 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916718+0000", 2024-09-16T14:49:27.812 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6233999999999999e-05 2024-09-16T14:49:27.812 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.812 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.812 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.812 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.812 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.6s1 77/27 ECSubWrite(tid=60, reqid=osd.0.0:59, at_version=0'0, trim_to=0'0, roll_forward_to=77'40))", 2024-09-16T14:49:27.813 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.905380+0000", 2024-09-16T14:49:27.813 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 3.7945623419999999, 2024-09-16T14:49:27.813 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011394315, 2024-09-16T14:49:27.813 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.813 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.813 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.813 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.813 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.813 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905380+0000", 2024-09-16T14:49:27.813 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.813 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.813 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.813 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905380+0000", 2024-09-16T14:49:27.814 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.814 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.814 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905380+0000", 2024-09-16T14:49:27.814 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:27.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.814 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.814 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905391+0000", 2024-09-16T14:49:27.814 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1868999999999999e-05 2024-09-16T14:49:27.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.814 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.815 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905404+0000", 2024-09-16T14:49:27.815 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2495e-05 2024-09-16T14:49:27.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.815 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.815 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905413+0000", 2024-09-16T14:49:27.815 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.02e-06 2024-09-16T14:49:27.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.815 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.815 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905463+0000", 2024-09-16T14:49:27.815 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.0283e-05 2024-09-16T14:49:27.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.816 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.816 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905473+0000", 2024-09-16T14:49:27.816 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6250000000000002e-06 2024-09-16T14:49:27.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.816 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.816 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.816 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916749+0000", 2024-09-16T14:49:27.816 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011275821 2024-09-16T14:49:27.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.816 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.816 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.816 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916775+0000", 2024-09-16T14:49:27.816 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6055999999999999e-05 2024-09-16T14:49:27.816 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.817 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.817 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.817 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.817 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.817 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s5 77/72 ECSubWrite(tid=142, reqid=osd.5.0:142, at_version=77'47, trim_to=0'0, roll_forward_to=74'45))", 2024-09-16T14:49:27.817 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.907289+0000", 2024-09-16T14:49:27.817 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 3.7926536779999998, 2024-09-16T14:49:27.817 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012031185, 2024-09-16T14:49:27.817 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.817 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.817 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.817 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.817 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.817 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907289+0000", 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907289+0000", 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907289+0000", 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.818 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907322+0000", 2024-09-16T14:49:27.819 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3012999999999997e-05 2024-09-16T14:49:27.819 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.819 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.819 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.819 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907336+0000", 2024-09-16T14:49:27.819 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4603000000000001e-05 2024-09-16T14:49:27.819 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.819 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.819 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.819 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907346+0000", 2024-09-16T14:49:27.819 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4390000000000006e-06 2024-09-16T14:49:27.819 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.819 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.819 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.820 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907421+0000", 2024-09-16T14:49:27.820 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5488999999999995e-05 2024-09-16T14:49:27.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.820 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.820 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.820 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907432+0000", 2024-09-16T14:49:27.820 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1034e-05 2024-09-16T14:49:27.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.820 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.820 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.820 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.919295+0000", 2024-09-16T14:49:27.820 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011863146999999999 2024-09-16T14:49:27.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.820 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.821 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.821 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.919320+0000", 2024-09-16T14:49:27.821 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4791999999999999e-05 2024-09-16T14:49:27.821 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.821 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.821 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.821 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.821 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.9s5 77/27 ECSubWrite(tid=82, reqid=osd.8.0:82, at_version=77'59, trim_to=0'0, roll_forward_to=74'57))", 2024-09-16T14:49:27.821 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.910048+0000", 2024-09-16T14:49:27.821 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 3.789894511, 2024-09-16T14:49:27.821 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0093052840000000005, 2024-09-16T14:49:27.821 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:27.821 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:27.822 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:27.822 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.822 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:27.822 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910048+0000", 2024-09-16T14:49:27.822 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.822 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.822 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.822 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:27.822 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910048+0000", 2024-09-16T14:49:27.822 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:27.822 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.822 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.822 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:27.822 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910048+0000", 2024-09-16T14:49:27.823 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:27.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.823 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.823 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:27.823 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910076+0000", 2024-09-16T14:49:27.823 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7858e-05 2024-09-16T14:49:27.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.823 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.823 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:27.823 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910089+0000", 2024-09-16T14:49:27.823 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3129000000000001e-05 2024-09-16T14:49:27.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910096+0000", 2024-09-16T14:49:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8009999999999998e-06 2024-09-16T14:49:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910185+0000", 2024-09-16T14:49:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9568999999999996e-05 2024-09-16T14:49:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910194+0000", 2024-09-16T14:49:27.824 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7390000000000005e-06 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.919338+0000", 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0091444439999999998 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.919354+0000", 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5258e-05 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:27.825 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:27.826 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:27.827 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:49:27.828 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:27.825+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:27.924 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.11 to 1 (10000) 2024-09-16T14:49:27.925 INFO:tasks.rados.rados.0.smithi040.stdout:311: done (4 left) 2024-09-16T14:49:27.926 INFO:tasks.rados.rados.0.smithi040.stdout:312: snap_remove snap 21 2024-09-16T14:49:27.928 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:27.926+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:27.941 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:49:27.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T14:49:27.985 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:27.985 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:27.985 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:27.985 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:27.994 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:49:28.035 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:28.033+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:28.130 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:28.128+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:28.145 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:28.145 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:28.145 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:28.145 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:28.145 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:28.154 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:49:28.231 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:28.229+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:28.263 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:28.263 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:28.263 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:28.263 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:28.263 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.263 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=128, reqid=client.4500.0:369, at_version=42'38, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:28.263 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:28.263 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 39.301000377000001, 2024-09-16T14:49:28.263 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060794769999999998, 2024-09-16T14:49:28.263 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.264 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.264 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.264 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.264 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.264 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:28.264 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.264 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.264 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.264 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.264 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:28.264 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.264 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.264 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.265 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.265 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959976+0000", 2024-09-16T14:49:28.265 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:28.265 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.265 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.265 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.265 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960436+0000", 2024-09-16T14:49:28.265 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00046022400000000002 2024-09-16T14:49:28.265 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.265 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.265 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.265 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960455+0000", 2024-09-16T14:49:28.265 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8162999999999999e-05 2024-09-16T14:49:28.265 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.266 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.266 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.266 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960468+0000", 2024-09-16T14:49:28.266 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3135999999999999e-05 2024-09-16T14:49:28.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.266 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.267 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.267 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960527+0000", 2024-09-16T14:49:28.267 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8859e-05 2024-09-16T14:49:28.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.267 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.267 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.267 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960533+0000", 2024-09-16T14:49:28.267 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:49:28.267 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.268 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.268 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.268 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.966018+0000", 2024-09-16T14:49:28.268 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0054845069999999996 2024-09-16T14:49:28.268 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.268 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.268 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.268 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.966057+0000", 2024-09-16T14:49:28.268 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.8927999999999997e-05 2024-09-16T14:49:28.268 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.268 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.269 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.269 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.269 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=129, reqid=client.4500.0:370, at_version=42'39, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:28.269 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:28.269 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 39.299744619000002, 2024-09-16T14:49:28.269 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074279639999999996, 2024-09-16T14:49:28.269 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.269 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.270 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.270 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.270 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.270 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:28.270 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.270 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.270 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.270 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:28.270 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.271 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.271 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.271 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961232+0000", 2024-09-16T14:49:28.271 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:28.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.271 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.271 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.272 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962118+0000", 2024-09-16T14:49:28.272 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00088592399999999996 2024-09-16T14:49:28.272 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.272 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.273 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.273 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962132+0000", 2024-09-16T14:49:28.273 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4127e-05 2024-09-16T14:49:28.273 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.273 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.273 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.273 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962142+0000", 2024-09-16T14:49:28.273 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0181e-05 2024-09-16T14:49:28.273 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.273 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.273 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.273 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962192+0000", 2024-09-16T14:49:28.273 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9826e-05 2024-09-16T14:49:28.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.274 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.274 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.274 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962202+0000", 2024-09-16T14:49:28.274 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8400000000000007e-06 2024-09-16T14:49:28.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.274 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.274 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.274 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968622+0000", 2024-09-16T14:49:28.274 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00642021 2024-09-16T14:49:28.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.275 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.275 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.275 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968661+0000", 2024-09-16T14:49:28.275 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.8670000000000001e-05 2024-09-16T14:49:28.275 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.275 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.275 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.275 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.275 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.275 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=130, reqid=client.4500.0:371, at_version=42'40, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:28.275 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:28.275 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 39.298409364000001, 2024-09-16T14:49:28.275 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0061144900000000002, 2024-09-16T14:49:28.276 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.276 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.276 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.276 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.276 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.276 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:28.276 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.276 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.276 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.276 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.276 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:28.276 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.276 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.277 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.277 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.277 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962567+0000", 2024-09-16T14:49:28.277 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.277 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.277 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.277 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962595+0000", 2024-09-16T14:49:28.277 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7824999999999999e-05 2024-09-16T14:49:28.277 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.277 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.277 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.277 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962611+0000", 2024-09-16T14:49:28.277 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:49:28.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.278 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.278 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.278 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962620+0000", 2024-09-16T14:49:28.278 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8259999999999999e-06 2024-09-16T14:49:28.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.278 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.278 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.278 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.963285+0000", 2024-09-16T14:49:28.278 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00066560100000000004 2024-09-16T14:49:28.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.278 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.278 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.278 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.963295+0000", 2024-09-16T14:49:28.279 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0036e-05 2024-09-16T14:49:28.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.279 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.279 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.279 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968667+0000", 2024-09-16T14:49:28.279 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0053715960000000002 2024-09-16T14:49:28.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.279 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.279 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.279 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968683+0000", 2024-09-16T14:49:28.279 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5522000000000001e-05 2024-09-16T14:49:28.279 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.279 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.279 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.280 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.280 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 43/27 ECSubWrite(tid=133, reqid=osd.11.0:133, at_version=43'42, trim_to=0'0, roll_forward_to=42'40))", 2024-09-16T14:49:28.280 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:28.280 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 38.407201569999998, 2024-09-16T14:49:28.280 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073093740000000004, 2024-09-16T14:49:28.280 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.280 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.280 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.280 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.280 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.280 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:28.281 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.281 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.281 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:28.281 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.281 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.281 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:28.281 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:28.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.281 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.282 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854515+0000", 2024-09-16T14:49:28.282 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00073950999999999997 2024-09-16T14:49:28.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.282 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.282 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854527+0000", 2024-09-16T14:49:28.282 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2012e-05 2024-09-16T14:49:28.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.282 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.282 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854535+0000", 2024-09-16T14:49:28.282 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:49:28.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.283 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.283 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854952+0000", 2024-09-16T14:49:28.283 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00041779399999999998 2024-09-16T14:49:28.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.283 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.283 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854961+0000", 2024-09-16T14:49:28.283 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7749999999999998e-06 2024-09-16T14:49:28.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.283 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.283 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.861052+0000", 2024-09-16T14:49:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006090608 2024-09-16T14:49:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.861085+0000", 2024-09-16T14:49:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3442999999999998e-05 2024-09-16T14:49:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 51/27 ECSubWrite(tid=169, reqid=osd.11.0:169, at_version=51'52, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:28.284 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 30.379791649000001, 2024-09-16T14:49:28.285 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006781848, 2024-09-16T14:49:28.285 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.285 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.285 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.285 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.285 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.285 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:28.285 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.285 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.285 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.285 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.285 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:28.285 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.286 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.286 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.286 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881185+0000", 2024-09-16T14:49:28.286 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.286 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.286 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.286 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881208+0000", 2024-09-16T14:49:28.286 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2823000000000002e-05 2024-09-16T14:49:28.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.286 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.286 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.287 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881218+0000", 2024-09-16T14:49:28.287 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3929999999999997e-06 2024-09-16T14:49:28.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.287 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.287 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.287 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881225+0000", 2024-09-16T14:49:28.287 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1629999999999999e-06 2024-09-16T14:49:28.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.287 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.287 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.287 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881279+0000", 2024-09-16T14:49:28.287 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4147000000000002e-05 2024-09-16T14:49:28.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.287 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.288 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.288 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881288+0000", 2024-09-16T14:49:28.288 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4630000000000007e-06 2024-09-16T14:49:28.288 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.288 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.288 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.288 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.887941+0000", 2024-09-16T14:49:28.288 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0066530210000000003 2024-09-16T14:49:28.288 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.288 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.288 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.288 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.887968+0000", 2024-09-16T14:49:28.288 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6228999999999999e-05 2024-09-16T14:49:28.289 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.289 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.289 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.289 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 51/27 ECSubWrite(tid=173, reqid=osd.11.0:173, at_version=51'54, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:28.289 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:28.289 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 30.379640422000001, 2024-09-16T14:49:28.289 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0080949799999999999, 2024-09-16T14:49:28.289 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.289 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.289 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.289 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.290 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:28.290 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.290 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.290 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.290 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:28.290 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.290 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.290 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.290 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881336+0000", 2024-09-16T14:49:28.290 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.290 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.291 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.291 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881356+0000", 2024-09-16T14:49:28.291 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9334000000000002e-05 2024-09-16T14:49:28.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.291 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.291 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.291 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881366+0000", 2024-09-16T14:49:28.291 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0742e-05 2024-09-16T14:49:28.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.291 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.291 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.291 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881374+0000", 2024-09-16T14:49:28.291 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5020000000000003e-06 2024-09-16T14:49:28.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.292 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.292 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.292 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.882087+0000", 2024-09-16T14:49:28.292 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00071271300000000002 2024-09-16T14:49:28.292 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.293 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.293 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.293 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.882096+0000", 2024-09-16T14:49:28.293 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4550000000000001e-06 2024-09-16T14:49:28.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.293 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.293 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.294 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889403+0000", 2024-09-16T14:49:28.294 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073067749999999997 2024-09-16T14:49:28.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.294 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.294 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889432+0000", 2024-09-16T14:49:28.294 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9065999999999999e-05 2024-09-16T14:49:28.294 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.294 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.295 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.295 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 52/27 ECSubWrite(tid=50, reqid=osd.1.0:50, at_version=52'40, trim_to=0'0, roll_forward_to=51'36))", 2024-09-16T14:49:28.295 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:28.295 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.378437553000001, 2024-09-16T14:49:28.295 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060730380000000002, 2024-09-16T14:49:28.295 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.295 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.295 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.295 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.295 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:28.295 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.296 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.296 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882540+0000", 2024-09-16T14:49:28.296 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.296 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.296 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882539+0000", 2024-09-16T14:49:28.296 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:28.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.296 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.297 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882566+0000", 2024-09-16T14:49:28.297 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7101999999999999e-05 2024-09-16T14:49:28.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.297 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.297 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882580+0000", 2024-09-16T14:49:28.297 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4605e-05 2024-09-16T14:49:28.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.297 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.297 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.882589+0000", 2024-09-16T14:49:28.297 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1020000000000003e-06 2024-09-16T14:49:28.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.298 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.298 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.884101+0000", 2024-09-16T14:49:28.298 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001511601 2024-09-16T14:49:28.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.298 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.298 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.884112+0000", 2024-09-16T14:49:28.298 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1055e-05 2024-09-16T14:49:28.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.298 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.298 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888581+0000", 2024-09-16T14:49:28.298 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044684240000000004 2024-09-16T14:49:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.888613+0000", 2024-09-16T14:49:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2354000000000002e-05 2024-09-16T14:49:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=235, reqid=osd.7.0:235, at_version=61'21, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.329451546000001, 2024-09-16T14:49:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0080251469999999998, 2024-09-16T14:49:28.300 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.300 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.300 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.300 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.300 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:28.300 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.300 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.300 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:28.300 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.301 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.301 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931525+0000", 2024-09-16T14:49:28.301 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.301 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.301 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931556+0000", 2024-09-16T14:49:28.301 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0713000000000002e-05 2024-09-16T14:49:28.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.301 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.301 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931577+0000", 2024-09-16T14:49:28.301 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0879000000000001e-05 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931599+0000", 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2107999999999999e-05 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932715+0000", 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0011159119999999999 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932725+0000", 2024-09-16T14:49:28.302 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8239999999999995e-06 2024-09-16T14:49:28.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.303 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.303 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939520+0000", 2024-09-16T14:49:28.303 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00679556 2024-09-16T14:49:28.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.303 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.303 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939551+0000", 2024-09-16T14:49:28.303 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0582e-05 2024-09-16T14:49:28.303 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.303 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.303 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.304 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 61/27 ECSubWrite(tid=63, reqid=osd.1.0:63, at_version=61'50, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:28.304 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:28.304 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.329330756000001, 2024-09-16T14:49:28.304 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090916260000000002, 2024-09-16T14:49:28.304 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.304 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.304 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.305 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.305 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:28.305 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.305 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.305 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.305 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:28.305 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.305 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.305 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.305 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931646+0000", 2024-09-16T14:49:28.305 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:28.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931665+0000", 2024-09-16T14:49:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8459999999999999e-05 2024-09-16T14:49:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931677+0000", 2024-09-16T14:49:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2150000000000001e-05 2024-09-16T14:49:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931684+0000", 2024-09-16T14:49:28.306 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3599999999999998e-06 2024-09-16T14:49:28.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.307 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.307 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931754+0000", 2024-09-16T14:49:28.307 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.9733999999999993e-05 2024-09-16T14:49:28.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.307 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.307 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931766+0000", 2024-09-16T14:49:28.307 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1873e-05 2024-09-16T14:49:28.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.307 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.308 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940712+0000", 2024-09-16T14:49:28.308 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089461669999999997 2024-09-16T14:49:28.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.308 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.308 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.308 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940738+0000", 2024-09-16T14:49:28.308 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6092999999999999e-05 2024-09-16T14:49:28.308 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.309 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.309 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.309 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.309 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.309 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:28.309 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:28.309 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.328930743000001, 2024-09-16T14:49:28.309 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010994122, 2024-09-16T14:49:28.309 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.309 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.309 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.309 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.309 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.309 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932066+0000", 2024-09-16T14:49:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9802000000000001e-05 2024-09-16T14:49:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932076+0000", 2024-09-16T14:49:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8470000000000008e-06 2024-09-16T14:49:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932086+0000", 2024-09-16T14:49:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7010000000000008e-06 2024-09-16T14:49:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933307+0000", 2024-09-16T14:49:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0012207450000000001 2024-09-16T14:49:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933319+0000", 2024-09-16T14:49:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2473000000000001e-05 2024-09-16T14:49:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943013+0000", 2024-09-16T14:49:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0096943700000000008 2024-09-16T14:49:28.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.313 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.316 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943041+0000", 2024-09-16T14:49:28.316 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7376999999999999e-05 2024-09-16T14:49:28.317 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.317 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.317 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.317 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:28.317 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:28.317 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.327805914999999, 2024-09-16T14:49:28.317 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0099360779999999992, 2024-09-16T14:49:28.317 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.317 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.317 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.318 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.318 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:28.318 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.318 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.318 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.318 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.318 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:28.318 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.318 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.318 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.318 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.318 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933171+0000", 2024-09-16T14:49:28.318 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933205+0000", 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3630999999999997e-05 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933214+0000", 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0620000000000007e-06 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933222+0000", 2024-09-16T14:49:28.319 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:28.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.320 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.320 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933271+0000", 2024-09-16T14:49:28.320 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9251e-05 2024-09-16T14:49:28.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.320 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.320 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933280+0000", 2024-09-16T14:49:28.320 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5419999999999994e-06 2024-09-16T14:49:28.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.320 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.321 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943089+0000", 2024-09-16T14:49:28.321 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0098082529999999994 2024-09-16T14:49:28.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.321 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.321 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.321 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943108+0000", 2024-09-16T14:49:28.321 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8916e-05 2024-09-16T14:49:28.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.321 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.322 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.322 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s2 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:28.322 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:28.322 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.327667386000002, 2024-09-16T14:49:28.322 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01234993, 2024-09-16T14:49:28.322 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.322 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.322 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.322 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.322 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.322 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:28.322 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.322 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.322 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.323 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.323 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:28.323 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.323 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.323 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.323 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:28.323 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.323 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.323 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.324 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.324 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933339+0000", 2024-09-16T14:49:28.324 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9107000000000001e-05 2024-09-16T14:49:28.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.324 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.324 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.324 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933347+0000", 2024-09-16T14:49:28.324 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6219999999999998e-06 2024-09-16T14:49:28.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.324 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.324 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.324 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933354+0000", 2024-09-16T14:49:28.324 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.72e-06 2024-09-16T14:49:28.324 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.325 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.325 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933438+0000", 2024-09-16T14:49:28.325 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.3666999999999997e-05 2024-09-16T14:49:28.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.325 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.325 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933453+0000", 2024-09-16T14:49:28.325 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4814000000000001e-05 2024-09-16T14:49:28.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.325 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.325 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945638+0000", 2024-09-16T14:49:28.325 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012185367000000001 2024-09-16T14:49:28.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.326 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.326 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.326 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945660+0000", 2024-09-16T14:49:28.326 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2033999999999999e-05 2024-09-16T14:49:28.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.326 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.326 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.326 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:28.326 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:28.326 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.327622264999999, 2024-09-16T14:49:28.326 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012249365, 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.327 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933354+0000", 2024-09-16T14:49:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933377+0000", 2024-09-16T14:49:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3087000000000001e-05 2024-09-16T14:49:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933393+0000", 2024-09-16T14:49:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5231e-05 2024-09-16T14:49:28.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.329 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.329 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.329 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933400+0000", 2024-09-16T14:49:28.329 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.6190000000000002e-06 2024-09-16T14:49:28.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.329 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.329 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.329 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933470+0000", 2024-09-16T14:49:28.329 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.936e-05 2024-09-16T14:49:28.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.329 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.329 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.329 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933479+0000", 2024-09-16T14:49:28.329 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2609999999999999e-06 2024-09-16T14:49:28.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.330 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.330 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.330 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945575+0000", 2024-09-16T14:49:28.330 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012095896 2024-09-16T14:49:28.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.330 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.330 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.330 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945605+0000", 2024-09-16T14:49:28.330 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9765e-05 2024-09-16T14:49:28.330 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.330 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.330 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.331 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.331 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:28.331 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:28.331 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.327579970999999, 2024-09-16T14:49:28.331 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013525466, 2024-09-16T14:49:28.331 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.331 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.331 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.331 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.331 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:28.331 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.331 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.332 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.332 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:28.332 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.332 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.332 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:28.332 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.333 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.333 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933451+0000", 2024-09-16T14:49:28.333 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4162000000000001e-05 2024-09-16T14:49:28.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.333 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.333 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933459+0000", 2024-09-16T14:49:28.333 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1110000000000008e-06 2024-09-16T14:49:28.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.333 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.333 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:28.334 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5050000000000004e-06 2024-09-16T14:49:28.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.334 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.334 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.936478+0000", 2024-09-16T14:49:28.334 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0030122790000000001 2024-09-16T14:49:28.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.334 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.334 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.936488+0000", 2024-09-16T14:49:28.334 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0197000000000001e-05 2024-09-16T14:49:28.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.334 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.335 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946886+0000", 2024-09-16T14:49:28.335 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010397912 2024-09-16T14:49:28.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.335 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.335 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946923+0000", 2024-09-16T14:49:28.335 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6699999999999998e-05 2024-09-16T14:49:28.335 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.335 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.335 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.335 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:28.335 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:28.336 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.32642251, 2024-09-16T14:49:28.336 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0083397039999999999, 2024-09-16T14:49:28.336 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.336 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.336 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.336 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.336 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:28.336 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.336 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.336 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:28.336 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.337 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.337 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934554+0000", 2024-09-16T14:49:28.337 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:28.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.337 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.337 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935547+0000", 2024-09-16T14:49:28.337 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00099309300000000001 2024-09-16T14:49:28.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.337 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.338 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935565+0000", 2024-09-16T14:49:28.338 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8335999999999999e-05 2024-09-16T14:49:28.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.338 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.338 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935578+0000", 2024-09-16T14:49:28.338 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2831e-05 2024-09-16T14:49:28.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.338 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.338 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935653+0000", 2024-09-16T14:49:28.338 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4411000000000003e-05 2024-09-16T14:49:28.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.339 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.339 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935663+0000", 2024-09-16T14:49:28.339 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0869999999999999e-05 2024-09-16T14:49:28.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.339 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.339 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942652+0000", 2024-09-16T14:49:28.339 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006988129 2024-09-16T14:49:28.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.339 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.339 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942895+0000", 2024-09-16T14:49:28.339 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00024308899999999999 2024-09-16T14:49:28.339 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.340 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.340 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.340 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s2 61/53 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=61'38, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:28.340 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:28.340 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.323083629999999, 2024-09-16T14:49:28.340 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090892060000000007, 2024-09-16T14:49:28.340 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.340 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.340 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.340 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.340 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:28.341 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.341 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.341 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:28.341 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.341 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.341 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937893+0000", 2024-09-16T14:49:28.341 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:28.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.341 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.342 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937915+0000", 2024-09-16T14:49:28.342 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2566e-05 2024-09-16T14:49:28.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.342 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.343 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937929+0000", 2024-09-16T14:49:28.343 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3314e-05 2024-09-16T14:49:28.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.343 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.343 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937938+0000", 2024-09-16T14:49:28.343 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2050000000000008e-06 2024-09-16T14:49:28.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.343 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.343 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940150+0000", 2024-09-16T14:49:28.343 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0022119040000000002 2024-09-16T14:49:28.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.344 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.344 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940158+0000", 2024-09-16T14:49:28.344 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2930000000000001e-06 2024-09-16T14:49:28.344 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.344 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.344 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946972+0000", 2024-09-16T14:49:28.344 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0068139150000000002 2024-09-16T14:49:28.344 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.344 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.344 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946983+0000", 2024-09-16T14:49:28.344 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0907000000000001e-05 2024-09-16T14:49:28.345 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.345 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.345 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.345 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.345 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.345 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 61/27 ECSubWrite(tid=226, reqid=osd.11.0:226, at_version=61'82, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:28.345 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:28.345 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.322561015000002, 2024-09-16T14:49:28.345 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085452879999999998, 2024-09-16T14:49:28.345 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.345 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.345 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.345 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.345 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.346 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:28.346 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.346 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.346 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:28.346 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.346 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.346 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:28.346 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:28.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.347 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.347 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938432+0000", 2024-09-16T14:49:28.347 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5931e-05 2024-09-16T14:49:28.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.347 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.347 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938444+0000", 2024-09-16T14:49:28.347 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1933000000000001e-05 2024-09-16T14:49:28.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.347 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.347 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938449+0000", 2024-09-16T14:49:28.347 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2070000000000003e-06 2024-09-16T14:49:28.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.348 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.348 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938492+0000", 2024-09-16T14:49:28.348 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2827999999999997e-05 2024-09-16T14:49:28.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.348 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.348 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938500+0000", 2024-09-16T14:49:28.348 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7600000000000008e-06 2024-09-16T14:49:28.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.348 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.348 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946932+0000", 2024-09-16T14:49:28.348 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084318099999999997 2024-09-16T14:49:28.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.349 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.349 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946962+0000", 2024-09-16T14:49:28.349 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9603000000000001e-05 2024-09-16T14:49:28.349 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.349 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.349 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.349 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=238, reqid=client.4500.0:536, at_version=61'23, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:28.349 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:28.349 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.321738043, 2024-09-16T14:49:28.349 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0064046049999999998, 2024-09-16T14:49:28.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.350 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.350 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.350 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.350 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:28.350 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.350 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.350 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:28.350 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.351 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.351 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939238+0000", 2024-09-16T14:49:28.351 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:28.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.351 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.351 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939537+0000", 2024-09-16T14:49:28.351 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029911299999999999 2024-09-16T14:49:28.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.351 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.351 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939551+0000", 2024-09-16T14:49:28.351 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4003e-05 2024-09-16T14:49:28.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.352 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.352 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939561+0000", 2024-09-16T14:49:28.352 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4269999999999997e-06 2024-09-16T14:49:28.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.352 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.352 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939777+0000", 2024-09-16T14:49:28.352 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00021644599999999999 2024-09-16T14:49:28.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.352 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.352 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939790+0000", 2024-09-16T14:49:28.352 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2381999999999999e-05 2024-09-16T14:49:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945601+0000", 2024-09-16T14:49:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0058117139999999999 2024-09-16T14:49:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945644+0000", 2024-09-16T14:49:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.265e-05 2024-09-16T14:49:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 75/65 ECSubWrite(tid=135, reqid=client.4500.0:690, at_version=75'49, trim_to=0'0, roll_forward_to=73'47))", 2024-09-16T14:49:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:21.922611+0000", 2024-09-16T14:49:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 6.3383661690000004, 2024-09-16T14:49:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090307089999999996, 2024-09-16T14:49:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922611+0000", 2024-09-16T14:49:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.355 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.355 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922611+0000", 2024-09-16T14:49:28.355 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.355 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.355 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922611+0000", 2024-09-16T14:49:28.355 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.355 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.355 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922966+0000", 2024-09-16T14:49:28.355 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00035514900000000002 2024-09-16T14:49:28.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.356 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.356 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922984+0000", 2024-09-16T14:49:28.356 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7747000000000001e-05 2024-09-16T14:49:28.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.356 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.356 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922993+0000", 2024-09-16T14:49:28.356 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8850000000000005e-06 2024-09-16T14:49:28.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.356 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.356 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.923096+0000", 2024-09-16T14:49:28.356 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000103626 2024-09-16T14:49:28.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.357 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.357 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.923107+0000", 2024-09-16T14:49:28.357 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0370999999999999e-05 2024-09-16T14:49:28.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.357 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.357 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.931610+0000", 2024-09-16T14:49:28.357 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085030420000000006 2024-09-16T14:49:28.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.357 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.357 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.931642+0000", 2024-09-16T14:49:28.358 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.235e-05 2024-09-16T14:49:28.358 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.358 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.358 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.358 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 75/65 ECSubWrite(tid=136, reqid=client.4500.0:691, at_version=75'50, trim_to=0'0, roll_forward_to=73'47))", 2024-09-16T14:49:28.358 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:21.926141+0000", 2024-09-16T14:49:28.358 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 6.3348363890000003, 2024-09-16T14:49:28.358 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0063006299999999998, 2024-09-16T14:49:28.358 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.358 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.358 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.359 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.359 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926141+0000", 2024-09-16T14:49:28.359 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.359 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.359 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926141+0000", 2024-09-16T14:49:28.359 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.359 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.359 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926140+0000", 2024-09-16T14:49:28.360 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:28.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.360 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.360 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926912+0000", 2024-09-16T14:49:28.360 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00077166799999999999 2024-09-16T14:49:28.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.360 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.360 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926932+0000", 2024-09-16T14:49:28.361 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0273000000000001e-05 2024-09-16T14:49:28.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.361 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.361 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926943+0000", 2024-09-16T14:49:28.361 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0934000000000001e-05 2024-09-16T14:49:28.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.361 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.361 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.927015+0000", 2024-09-16T14:49:28.361 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2012000000000001e-05 2024-09-16T14:49:28.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.362 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.362 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.927027+0000", 2024-09-16T14:49:28.362 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2490000000000001e-05 2024-09-16T14:49:28.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.362 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.362 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.362 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.932418+0000", 2024-09-16T14:49:28.362 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0053903349999999996 2024-09-16T14:49:28.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.362 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.362 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.362 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.932442+0000", 2024-09-16T14:49:28.362 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4043e-05 2024-09-16T14:49:28.362 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.363 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.363 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.363 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.363 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.363 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:28.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:49:28.365 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:28.362+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:28.465 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:28.464+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:28.511 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:28.511 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:28.511 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:28.511 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:28.521 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:49:28.567 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:28.565+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:28.628 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:28.628 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:28.628 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:28.628 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:28.629 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:28.637 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:49:28.644 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:49:28.669 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:28.667+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:28.746 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:28.746 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:28.746 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:28.746 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:28.747 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.747 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:303 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:28.747 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:28.747 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 43.811600665, 2024-09-16T14:49:28.747 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014763235, 2024-09-16T14:49:28.747 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.747 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:28.747 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.747 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.747 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.747 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 303 2024-09-16T14:49:28.747 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.747 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.747 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.748 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.748 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:28.748 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.748 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.748 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932468+0000", 2024-09-16T14:49:28.748 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.748 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.748 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932467+0000", 2024-09-16T14:49:28.748 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:28.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.749 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.749 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932474+0000", 2024-09-16T14:49:28.749 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3189999999999997e-06 2024-09-16T14:49:28.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.749 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.749 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932490+0000", 2024-09-16T14:49:28.749 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5681000000000001e-05 2024-09-16T14:49:28.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.749 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.749 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932498+0000", 2024-09-16T14:49:28.750 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7789999999999996e-06 2024-09-16T14:49:28.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.750 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.750 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932544+0000", 2024-09-16T14:49:28.750 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6486000000000002e-05 2024-09-16T14:49:28.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.750 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:28.750 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932550+0000", 2024-09-16T14:49:28.750 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.3149999999999999e-06 2024-09-16T14:49:28.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.750 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.751 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.939655+0000", 2024-09-16T14:49:28.751 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071050039999999998 2024-09-16T14:49:28.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.751 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.751 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.939762+0000", 2024-09-16T14:49:28.751 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000106885 2024-09-16T14:49:28.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.751 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.751 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.947232+0000", 2024-09-16T14:49:28.751 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074701619999999998 2024-09-16T14:49:28.751 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.751 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.752 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.752 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:304 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:28.752 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:28.752 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 43.811562924, 2024-09-16T14:49:28.752 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015391726, 2024-09-16T14:49:28.752 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.752 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:28.752 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.752 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.752 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.752 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 304 2024-09-16T14:49:28.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.753 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.753 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.753 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:28.753 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.753 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.753 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:28.753 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.753 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.753 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932506+0000", 2024-09-16T14:49:28.754 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.754 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.754 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932510+0000", 2024-09-16T14:49:28.754 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.1069999999999998e-06 2024-09-16T14:49:28.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.754 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.754 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932516+0000", 2024-09-16T14:49:28.754 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4699999999999999e-06 2024-09-16T14:49:28.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.754 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.755 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932522+0000", 2024-09-16T14:49:28.755 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4750000000000001e-06 2024-09-16T14:49:28.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.755 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.755 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.755 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932610+0000", 2024-09-16T14:49:28.755 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8623999999999997e-05 2024-09-16T14:49:28.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.755 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.755 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:28.755 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932616+0000", 2024-09-16T14:49:28.755 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.9649999999999996e-06 2024-09-16T14:49:28.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.755 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940205+0000", 2024-09-16T14:49:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0075882570000000002 2024-09-16T14:49:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.940285+0000", 2024-09-16T14:49:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0783000000000002e-05 2024-09-16T14:49:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.947898+0000", 2024-09-16T14:49:28.756 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0076124820000000003 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:305 3.2s0 3:4bb56677:::smithi04040151-21:head [read 0~344064,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 43.811482134000002, 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015660833999999998, 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.757 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 305 2024-09-16T14:49:28.758 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.758 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.758 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.758 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.758 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:28.758 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.758 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.758 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.758 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.758 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932587+0000", 2024-09-16T14:49:28.758 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.758 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.758 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.758 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.759 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932586+0000", 2024-09-16T14:49:28.759 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.759 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.759 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.759 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.759 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932591+0000", 2024-09-16T14:49:28.759 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2470000000000002e-06 2024-09-16T14:49:28.759 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.759 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.760 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.760 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932602+0000", 2024-09-16T14:49:28.760 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1396000000000001e-05 2024-09-16T14:49:28.760 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.760 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.760 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.760 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932608+0000", 2024-09-16T14:49:28.761 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.7169999999999996e-06 2024-09-16T14:49:28.761 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.761 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.761 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.761 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932646+0000", 2024-09-16T14:49:28.761 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.7978999999999997e-05 2024-09-16T14:49:28.761 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.762 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.762 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:28.762 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932652+0000", 2024-09-16T14:49:28.762 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0310000000000004e-06 2024-09-16T14:49:28.762 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.762 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.762 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.762 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941247+0000", 2024-09-16T14:49:28.762 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085954180000000005 2024-09-16T14:49:28.762 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.762 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.763 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.763 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941327+0000", 2024-09-16T14:49:28.763 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9354e-05 2024-09-16T14:49:28.763 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.763 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.763 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.763 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.948248+0000", 2024-09-16T14:49:28.763 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0069212400000000004 2024-09-16T14:49:28.763 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.763 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.763 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.763 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.764 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.764 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:403 3.2s0 3:421e4b92:::smithi04040151-11:head [copy-from ver 2] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:28.764 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:28.764 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 34.860504237000001, 2024-09-16T14:49:28.764 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015904270000000002, 2024-09-16T14:49:28.764 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.764 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:28.764 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.764 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.764 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.764 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 403 2024-09-16T14:49:28.764 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.765 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.765 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.765 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.765 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:28.765 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.765 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.765 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.765 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.765 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883565+0000", 2024-09-16T14:49:28.765 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.765 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.765 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.765 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.766 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883563+0000", 2024-09-16T14:49:28.766 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:28.766 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.766 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.766 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.766 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883573+0000", 2024-09-16T14:49:28.766 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0583e-05 2024-09-16T14:49:28.766 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.766 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.766 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.766 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883593+0000", 2024-09-16T14:49:28.766 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0052e-05 2024-09-16T14:49:28.766 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.766 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.767 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.767 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883603+0000", 2024-09-16T14:49:28.767 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4409999999999999e-06 2024-09-16T14:49:28.767 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.767 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.767 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.767 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883667+0000", 2024-09-16T14:49:28.767 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4671000000000005e-05 2024-09-16T14:49:28.767 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.767 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.767 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.767 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883775+0000", 2024-09-16T14:49:28.767 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000108068 2024-09-16T14:49:28.767 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.768 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.768 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.768 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.896809+0000", 2024-09-16T14:49:28.768 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01303355 2024-09-16T14:49:28.768 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.768 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.768 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.768 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897717+0000", 2024-09-16T14:49:28.768 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00090801599999999999 2024-09-16T14:49:28.768 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.769 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.769 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:28.769 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.899408+0000", 2024-09-16T14:49:28.769 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0016911459999999999 2024-09-16T14:49:28.769 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.769 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.770 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.770 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.899469+0000", 2024-09-16T14:49:28.770 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0968999999999999e-05 2024-09-16T14:49:28.770 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.770 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.770 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.770 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.770 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.770 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:404 3.2s0 3:421e4b92:::smithi04040151-11:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:28.770 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:28.770 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 34.86045094, 2024-09-16T14:49:28.770 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0139831, 2024-09-16T14:49:28.771 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.771 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:28.771 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.771 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.771 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.771 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 404 2024-09-16T14:49:28.771 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.772 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.772 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.772 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:28.772 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.772 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.772 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883618+0000", 2024-09-16T14:49:28.772 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.772 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.772 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883617+0000", 2024-09-16T14:49:28.773 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:28.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.773 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.773 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.773 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883622+0000", 2024-09-16T14:49:28.773 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7489999999999998e-06 2024-09-16T14:49:28.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.773 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.773 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.773 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883631+0000", 2024-09-16T14:49:28.773 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6489999999999997e-06 2024-09-16T14:49:28.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.773 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.773 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.774 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883639+0000", 2024-09-16T14:49:28.774 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2120000000000003e-06 2024-09-16T14:49:28.774 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.774 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.774 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883930+0000", 2024-09-16T14:49:28.774 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029138200000000002 2024-09-16T14:49:28.774 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.774 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:28.775 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.883961+0000", 2024-09-16T14:49:28.775 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0190999999999999e-05 2024-09-16T14:49:28.775 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.775 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.775 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.775 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897411+0000", 2024-09-16T14:49:28.775 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013450037999999999 2024-09-16T14:49:28.775 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.775 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.775 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.775 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897506+0000", 2024-09-16T14:49:28.775 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5785000000000005e-05 2024-09-16T14:49:28.775 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.775 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.776 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.776 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.897601+0000", 2024-09-16T14:49:28.776 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4868e-05 2024-09-16T14:49:28.776 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.776 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.776 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.776 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.776 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.776 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s4 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:28.776 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.931930+0000", 2024-09-16T14:49:28.776 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.812139039000002, 2024-09-16T14:49:28.776 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014515299000000001, 2024-09-16T14:49:28.776 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.776 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.777 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.777 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.777 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.777 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931930+0000", 2024-09-16T14:49:28.777 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.777 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.777 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.777 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.777 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931930+0000", 2024-09-16T14:49:28.777 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.777 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.777 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.777 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.777 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931929+0000", 2024-09-16T14:49:28.778 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:28.778 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.778 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.778 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.778 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931950+0000", 2024-09-16T14:49:28.778 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0661000000000001e-05 2024-09-16T14:49:28.778 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.778 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.778 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.778 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931962+0000", 2024-09-16T14:49:28.778 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1949e-05 2024-09-16T14:49:28.778 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.778 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.779 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.779 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931970+0000", 2024-09-16T14:49:28.779 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9510000000000006e-06 2024-09-16T14:49:28.779 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.779 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.779 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.779 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932022+0000", 2024-09-16T14:49:28.779 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1990000000000002e-05 2024-09-16T14:49:28.779 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.779 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.779 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.779 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932032+0000", 2024-09-16T14:49:28.779 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0474e-05 2024-09-16T14:49:28.780 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.780 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.780 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.780 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946420+0000", 2024-09-16T14:49:28.780 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0143874 2024-09-16T14:49:28.780 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.780 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.780 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.780 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946445+0000", 2024-09-16T14:49:28.780 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5627e-05 2024-09-16T14:49:28.780 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.780 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.780 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.781 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.781 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.781 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.as6 61/53 ECSubWrite(tid=225, reqid=osd.11.0:225, at_version=61'26, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:28.781 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932886+0000", 2024-09-16T14:49:28.781 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.811183545999999, 2024-09-16T14:49:28.781 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014177248, 2024-09-16T14:49:28.781 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.781 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.781 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.781 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.781 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.781 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932886+0000", 2024-09-16T14:49:28.781 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.782 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.782 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.782 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932886+0000", 2024-09-16T14:49:28.782 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.782 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.782 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.782 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932885+0000", 2024-09-16T14:49:28.782 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:28.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.782 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.782 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.782 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932916+0000", 2024-09-16T14:49:28.783 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1257999999999998e-05 2024-09-16T14:49:28.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.783 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.783 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.783 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932934+0000", 2024-09-16T14:49:28.783 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7958999999999999e-05 2024-09-16T14:49:28.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.783 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.783 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.784 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932942+0000", 2024-09-16T14:49:28.784 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9189999999999999e-06 2024-09-16T14:49:28.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.784 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.784 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933003+0000", 2024-09-16T14:49:28.784 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1301000000000001e-05 2024-09-16T14:49:28.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.784 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.784 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933023+0000", 2024-09-16T14:49:28.784 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0258000000000002e-05 2024-09-16T14:49:28.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.785 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.785 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946638+0000", 2024-09-16T14:49:28.785 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013614924 2024-09-16T14:49:28.785 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.785 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.785 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947063+0000", 2024-09-16T14:49:28.785 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00042440400000000002 2024-09-16T14:49:28.785 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.785 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.786 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.786 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s1 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:28.786 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933175+0000", 2024-09-16T14:49:28.786 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.810894217000001, 2024-09-16T14:49:28.786 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014250087999999999, 2024-09-16T14:49:28.786 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.786 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.786 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.786 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.786 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933175+0000", 2024-09-16T14:49:28.786 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.787 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.787 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933175+0000", 2024-09-16T14:49:28.787 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.787 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.787 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933174+0000", 2024-09-16T14:49:28.787 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.787 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.787 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933196+0000", 2024-09-16T14:49:28.788 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1577999999999999e-05 2024-09-16T14:49:28.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.788 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.788 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933210+0000", 2024-09-16T14:49:28.788 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3864e-05 2024-09-16T14:49:28.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.788 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.788 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933217+0000", 2024-09-16T14:49:28.788 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1999999999999997e-06 2024-09-16T14:49:28.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.789 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.789 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933306+0000", 2024-09-16T14:49:28.789 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8985000000000003e-05 2024-09-16T14:49:28.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.789 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.789 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933315+0000", 2024-09-16T14:49:28.789 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5730000000000008e-06 2024-09-16T14:49:28.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.789 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.789 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947397+0000", 2024-09-16T14:49:28.789 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014082643000000001 2024-09-16T14:49:28.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.790 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.790 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.790 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947425+0000", 2024-09-16T14:49:28.790 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7793e-05 2024-09-16T14:49:28.790 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.790 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.790 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.790 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.790 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs1 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:28.790 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933463+0000", 2024-09-16T14:49:28.790 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 20.810605939999999, 2024-09-16T14:49:28.790 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014016617, 2024-09-16T14:49:28.790 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.791 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.791 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.791 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.791 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933463+0000", 2024-09-16T14:49:28.791 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.791 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.791 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933463+0000", 2024-09-16T14:49:28.791 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.791 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.792 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933462+0000", 2024-09-16T14:49:28.792 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:28.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.792 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.792 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933487+0000", 2024-09-16T14:49:28.792 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4287e-05 2024-09-16T14:49:28.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.792 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.792 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933501+0000", 2024-09-16T14:49:28.792 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4202999999999999e-05 2024-09-16T14:49:28.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.793 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.793 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933509+0000", 2024-09-16T14:49:28.793 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5979999999999999e-06 2024-09-16T14:49:28.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.793 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.793 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940824+0000", 2024-09-16T14:49:28.793 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073150840000000003 2024-09-16T14:49:28.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.793 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.793 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940836+0000", 2024-09-16T14:49:28.793 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2002e-05 2024-09-16T14:49:28.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.794 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.794 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947393+0000", 2024-09-16T14:49:28.794 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006557223 2024-09-16T14:49:28.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.794 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.795 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947480+0000", 2024-09-16T14:49:28.795 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6948999999999997e-05 2024-09-16T14:49:28.795 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.795 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.795 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.795 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.795 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:704 3.7s0 3:e72b26aa:::smithi04040151-26:2e [read 0~2670592,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:28.795 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.900073+0000", 2024-09-16T14:49:28.795 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.8439959689999998, 2024-09-16T14:49:28.796 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.018635723, 2024-09-16T14:49:28.796 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.796 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:28.796 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.796 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.796 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.796 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 704 2024-09-16T14:49:28.796 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.797 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.797 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.797 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.797 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900073+0000", 2024-09-16T14:49:28.797 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.797 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.797 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.797 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.797 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900073+0000", 2024-09-16T14:49:28.797 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.797 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.797 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.797 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.798 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900071+0000", 2024-09-16T14:49:28.798 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:28.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.798 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.798 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.798 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900082+0000", 2024-09-16T14:49:28.798 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0372e-05 2024-09-16T14:49:28.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.798 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.798 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.798 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900101+0000", 2024-09-16T14:49:28.798 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.98e-05 2024-09-16T14:49:28.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.798 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.799 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.799 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900114+0000", 2024-09-16T14:49:28.799 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2125e-05 2024-09-16T14:49:28.799 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.799 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.799 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.799 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900179+0000", 2024-09-16T14:49:28.799 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5630999999999996e-05 2024-09-16T14:49:28.799 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.799 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.799 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:28.799 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900187+0000", 2024-09-16T14:49:28.799 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0930000000000003e-06 2024-09-16T14:49:28.799 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.800 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.800 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.800 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.902879+0000", 2024-09-16T14:49:28.800 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0026919880000000002 2024-09-16T14:49:28.800 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.800 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.800 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.800 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903019+0000", 2024-09-16T14:49:28.800 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00013949899999999999 2024-09-16T14:49:28.800 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.800 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.800 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.800 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918709+0000", 2024-09-16T14:49:28.800 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015690075000000001 2024-09-16T14:49:28.801 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.801 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.801 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.801 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.801 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.801 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:705 3.7s0 3:e72b26aa:::smithi04040151-26:2e [read 0~2670592,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:28.801 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.900149+0000", 2024-09-16T14:49:28.801 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.8439196960000004, 2024-09-16T14:49:28.801 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.022052176999999999, 2024-09-16T14:49:28.801 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.801 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:28.802 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.802 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.802 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.802 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 705 2024-09-16T14:49:28.802 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.802 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.802 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.802 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.802 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900149+0000", 2024-09-16T14:49:28.802 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.802 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.802 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.802 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.802 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900149+0000", 2024-09-16T14:49:28.803 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.803 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.803 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.803 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.803 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900149+0000", 2024-09-16T14:49:28.803 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:28.803 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.803 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.803 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.803 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900155+0000", 2024-09-16T14:49:28.803 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8710000000000002e-06 2024-09-16T14:49:28.803 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.803 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.803 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.804 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900164+0000", 2024-09-16T14:49:28.804 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5489999999999995e-06 2024-09-16T14:49:28.804 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.804 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.804 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.804 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900171+0000", 2024-09-16T14:49:28.804 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8900000000000001e-06 2024-09-16T14:49:28.804 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.804 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.804 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.804 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900239+0000", 2024-09-16T14:49:28.804 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8232999999999995e-05 2024-09-16T14:49:28.804 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900245+0000", 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.879e-06 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903508+0000", 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0032631159999999999 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903672+0000", 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00016352700000000001 2024-09-16T14:49:28.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.806 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.806 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.806 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.922202+0000", 2024-09-16T14:49:28.806 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.018529869000000001 2024-09-16T14:49:28.806 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.806 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.806 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.806 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.806 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.806 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:706 3.7s0 3:e72b26aa:::smithi04040151-26:2e [read 0~2670592,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:28.806 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.900176+0000", 2024-09-16T14:49:28.807 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.8438931470000002, 2024-09-16T14:49:28.807 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.024066440000000001, 2024-09-16T14:49:28.807 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.807 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:28.807 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.807 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.807 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.807 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 706 2024-09-16T14:49:28.807 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.807 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.807 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.807 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.807 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900176+0000", 2024-09-16T14:49:28.807 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.808 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.808 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.808 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.808 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900176+0000", 2024-09-16T14:49:28.808 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.808 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.808 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.808 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.808 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900176+0000", 2024-09-16T14:49:28.808 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.808 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.808 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.809 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.809 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900179+0000", 2024-09-16T14:49:28.809 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.54e-06 2024-09-16T14:49:28.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.809 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.809 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.809 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900185+0000", 2024-09-16T14:49:28.809 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.2990000000000002e-06 2024-09-16T14:49:28.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.809 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.809 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.809 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900190+0000", 2024-09-16T14:49:28.809 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2599999999999999e-06 2024-09-16T14:49:28.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.810 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.810 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900266+0000", 2024-09-16T14:49:28.810 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.6600999999999994e-05 2024-09-16T14:49:28.810 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.810 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:28.810 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.900272+0000", 2024-09-16T14:49:28.810 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.327e-06 2024-09-16T14:49:28.810 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.810 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.810 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910599+0000", 2024-09-16T14:49:28.811 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010327462000000001 2024-09-16T14:49:28.811 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.811 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.811 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910715+0000", 2024-09-16T14:49:28.811 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000115967 2024-09-16T14:49:28.811 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.811 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.811 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.924242+0000", 2024-09-16T14:49:28.811 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013527384999999999 2024-09-16T14:49:28.811 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.812 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.812 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.812 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.812 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:710 3.2s0 3:421e4b92:::smithi04040151-11:head [append 0~753664] snapc 2f=[2f,2e,29,28,25] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:28.812 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.905803+0000", 2024-09-16T14:49:28.812 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.8382660829999997, 2024-09-16T14:49:28.812 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.018574232999999999, 2024-09-16T14:49:28.812 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.812 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:28.812 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.812 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.812 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.813 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 710 2024-09-16T14:49:28.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.813 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.813 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.813 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.813 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905803+0000", 2024-09-16T14:49:28.813 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.813 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.813 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.813 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905803+0000", 2024-09-16T14:49:28.813 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.814 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.814 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905801+0000", 2024-09-16T14:49:28.814 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:28.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.814 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.814 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.906276+0000", 2024-09-16T14:49:28.814 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00047506499999999999 2024-09-16T14:49:28.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.814 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.815 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.906296+0000", 2024-09-16T14:49:28.815 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0038999999999999e-05 2024-09-16T14:49:28.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.815 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.815 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.906305+0000", 2024-09-16T14:49:28.815 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6119999999999999e-06 2024-09-16T14:49:28.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.815 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.815 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.906393+0000", 2024-09-16T14:49:28.815 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7987999999999999e-05 2024-09-16T14:49:28.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.816 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.816 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.816 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.906550+0000", 2024-09-16T14:49:28.816 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000157312 2024-09-16T14:49:28.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.816 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.816 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.816 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907921+0000", 2024-09-16T14:49:28.816 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001370846 2024-09-16T14:49:28.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.816 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.816 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.817 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916208+0000", 2024-09-16T14:49:28.817 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0082873579999999999 2024-09-16T14:49:28.817 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.817 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.817 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:28.817 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.924316+0000", 2024-09-16T14:49:28.817 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0081081039999999997 2024-09-16T14:49:28.817 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.817 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.817 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.817 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.924377+0000", 2024-09-16T14:49:28.817 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0782999999999997e-05 2024-09-16T14:49:28.817 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.817 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.818 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.818 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.818 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:712 3.2s0 3:421e4b92:::smithi04040151-11:head [append 0~475136] snapc 2f=[2f,2e,29,28,25] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:28.818 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.917571+0000", 2024-09-16T14:49:28.818 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.8264981779999996, 2024-09-16T14:49:28.818 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014221094, 2024-09-16T14:49:28.818 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.818 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:28.818 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.818 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.818 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.818 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 712 2024-09-16T14:49:28.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.819 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.819 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.819 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.819 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.917571+0000", 2024-09-16T14:49:28.819 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.819 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.819 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.819 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.819 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.917571+0000", 2024-09-16T14:49:28.819 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.819 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.819 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.820 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.820 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.917569+0000", 2024-09-16T14:49:28.820 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:28.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.820 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.820 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.820 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.917886+0000", 2024-09-16T14:49:28.820 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00031667700000000002 2024-09-16T14:49:28.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.820 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.820 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.820 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.917899+0000", 2024-09-16T14:49:28.820 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3118e-05 2024-09-16T14:49:28.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.821 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.821 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.821 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.917905+0000", 2024-09-16T14:49:28.821 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6449999999999999e-06 2024-09-16T14:49:28.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.821 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.821 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.821 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918063+0000", 2024-09-16T14:49:28.821 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00015757799999999999 2024-09-16T14:49:28.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.821 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.821 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.822 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918271+0000", 2024-09-16T14:49:28.822 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000207503 2024-09-16T14:49:28.822 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.822 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.822 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.822 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.921028+0000", 2024-09-16T14:49:28.822 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.002757921 2024-09-16T14:49:28.822 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.822 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.822 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.822 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.931651+0000", 2024-09-16T14:49:28.822 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010622354000000001 2024-09-16T14:49:28.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.823 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.823 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:28.823 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.931740+0000", 2024-09-16T14:49:28.823 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9185999999999996e-05 2024-09-16T14:49:28.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.823 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.823 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.823 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.931792+0000", 2024-09-16T14:49:28.823 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2024000000000002e-05 2024-09-16T14:49:28.823 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.823 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.824 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.824 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.824 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:713 3.2s0 3:421e4b92:::smithi04040151-11:head [setxattr _header (57) in=7b] snapc 2f=[2f,2e,29,28,25] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:28.824 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.917936+0000", 2024-09-16T14:49:28.824 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.8261327359999999, 2024-09-16T14:49:28.824 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014238389000000001, 2024-09-16T14:49:28.824 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.824 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:28.824 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.824 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.824 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.824 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 713 2024-09-16T14:49:28.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.825 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.825 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.825 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.825 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.917936+0000", 2024-09-16T14:49:28.825 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.825 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.825 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.825 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.825 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.917936+0000", 2024-09-16T14:49:28.825 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.825 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.825 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.825 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.826 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.917936+0000", 2024-09-16T14:49:28.826 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:28.826 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.826 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.826 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.826 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.917942+0000", 2024-09-16T14:49:28.826 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0820000000000004e-06 2024-09-16T14:49:28.826 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.826 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.826 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.826 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.917953+0000", 2024-09-16T14:49:28.826 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1377e-05 2024-09-16T14:49:28.826 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.826 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.827 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.827 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.917960+0000", 2024-09-16T14:49:28.827 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.0489999999999998e-06 2024-09-16T14:49:28.827 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.827 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.827 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.827 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.922997+0000", 2024-09-16T14:49:28.827 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0050365979999999998 2024-09-16T14:49:28.827 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.827 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.827 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.827 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.923066+0000", 2024-09-16T14:49:28.827 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.9323e-05 2024-09-16T14:49:28.827 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.828 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.828 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.828 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.923503+0000", 2024-09-16T14:49:28.828 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00043735699999999998 2024-09-16T14:49:28.828 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.828 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.828 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.828 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.931797+0000", 2024-09-16T14:49:28.828 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0082938749999999992 2024-09-16T14:49:28.828 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.828 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.828 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:28.829 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.931853+0000", 2024-09-16T14:49:28.829 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5618999999999997e-05 2024-09-16T14:49:28.829 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.829 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.829 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.829 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.932175+0000", 2024-09-16T14:49:28.829 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00032195699999999999 2024-09-16T14:49:28.829 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.829 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.829 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.829 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.829 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.829 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:714 3.2s0 3:421e4b92:::smithi04040151-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.918000+0000", 2024-09-16T14:49:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.8260686750000001, 2024-09-16T14:49:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.019169845000000001, 2024-09-16T14:49:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 714 2024-09-16T14:49:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.830 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.831 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.831 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918000+0000", 2024-09-16T14:49:28.831 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.831 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.831 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.831 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.831 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918000+0000", 2024-09-16T14:49:28.831 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.831 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.831 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.831 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.831 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918000+0000", 2024-09-16T14:49:28.832 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.832 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.832 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.832 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.832 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918003+0000", 2024-09-16T14:49:28.832 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3069999999999998e-06 2024-09-16T14:49:28.832 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.832 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.832 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.832 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918011+0000", 2024-09-16T14:49:28.832 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4039999999999998e-06 2024-09-16T14:49:28.832 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.833 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.833 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.833 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918016+0000", 2024-09-16T14:49:28.833 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.874e-06 2024-09-16T14:49:28.833 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.833 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.833 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.833 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.924952+0000", 2024-09-16T14:49:28.833 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0069352179999999999 2024-09-16T14:49:28.833 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.833 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.833 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:28.834 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.925026+0000", 2024-09-16T14:49:28.834 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4216999999999998e-05 2024-09-16T14:49:28.834 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.834 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.834 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.834 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.932209+0000", 2024-09-16T14:49:28.834 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071830009999999996 2024-09-16T14:49:28.834 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.834 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.834 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.835 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.932297+0000", 2024-09-16T14:49:28.835 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8536999999999996e-05 2024-09-16T14:49:28.835 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.835 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.835 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.835 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.937170+0000", 2024-09-16T14:49:28.835 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0048728699999999996 2024-09-16T14:49:28.835 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.835 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.835 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.835 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.835 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.835 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:715 3.7s0 3:e91f0d6c:::smithi04040151-31:25 [read 0~4390912,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:28.836 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.918021+0000", 2024-09-16T14:49:28.836 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.8260484310000002, 2024-09-16T14:49:28.836 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.018099726, 2024-09-16T14:49:28.836 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.836 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:28.836 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.836 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.836 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.836 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 715 2024-09-16T14:49:28.836 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.836 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.836 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.836 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.836 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918021+0000", 2024-09-16T14:49:28.837 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.837 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.837 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.837 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918021+0000", 2024-09-16T14:49:28.837 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.837 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.837 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.837 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918020+0000", 2024-09-16T14:49:28.837 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.837 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.837 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.838 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918024+0000", 2024-09-16T14:49:28.838 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5020000000000001e-06 2024-09-16T14:49:28.838 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.838 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.838 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.838 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918029+0000", 2024-09-16T14:49:28.838 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5230000000000003e-06 2024-09-16T14:49:28.838 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.838 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.838 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.838 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918033+0000", 2024-09-16T14:49:28.838 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.155e-06 2024-09-16T14:49:28.838 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.839 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.839 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.839 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.924413+0000", 2024-09-16T14:49:28.839 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0063794719999999997 2024-09-16T14:49:28.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.839 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.839 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.839 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.924511+0000", 2024-09-16T14:49:28.839 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8009000000000003e-05 2024-09-16T14:49:28.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.839 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.839 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.839 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.936120+0000", 2024-09-16T14:49:28.839 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011609491 2024-09-16T14:49:28.840 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.840 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.840 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.840 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.840 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.840 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:716 3.7s0 3:e91f0d6c:::smithi04040151-31:25 [read 0~4390912,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:28.840 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.918038+0000", 2024-09-16T14:49:28.840 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.8260314930000003, 2024-09-16T14:49:28.840 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.021225872, 2024-09-16T14:49:28.840 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.840 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:28.840 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.840 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.840 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.841 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 716 2024-09-16T14:49:28.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.841 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.841 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.841 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.841 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918038+0000", 2024-09-16T14:49:28.841 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.841 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.841 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.841 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918038+0000", 2024-09-16T14:49:28.841 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.842 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.842 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918037+0000", 2024-09-16T14:49:28.842 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.842 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.842 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918040+0000", 2024-09-16T14:49:28.842 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8059999999999999e-06 2024-09-16T14:49:28.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.842 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.842 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918045+0000", 2024-09-16T14:49:28.843 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2499999999999997e-06 2024-09-16T14:49:28.843 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.843 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.843 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918050+0000", 2024-09-16T14:49:28.843 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2339999999999996e-06 2024-09-16T14:49:28.843 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.843 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.843 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.924806+0000", 2024-09-16T14:49:28.843 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0067567620000000004 2024-09-16T14:49:28.843 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.843 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.844 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.924887+0000", 2024-09-16T14:49:28.844 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0950000000000003e-05 2024-09-16T14:49:28.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.844 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.844 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.939263+0000", 2024-09-16T14:49:28.844 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014376194 2024-09-16T14:49:28.844 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.844 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.844 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.844 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:717 3.7s0 3:e91f0d6c:::smithi04040151-31:25 [read 0~4390912,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:28.844 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.918053+0000", 2024-09-16T14:49:28.845 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.8260158759999996, 2024-09-16T14:49:28.845 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.024574242999999999, 2024-09-16T14:49:28.845 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.845 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:28.845 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:28.845 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:28.845 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:28.845 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 717 2024-09-16T14:49:28.845 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.845 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.845 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.845 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.846 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918053+0000", 2024-09-16T14:49:28.846 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.846 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.846 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918053+0000", 2024-09-16T14:49:28.846 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.846 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.846 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918053+0000", 2024-09-16T14:49:28.846 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.847 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.847 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918055+0000", 2024-09-16T14:49:28.847 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5809999999999999e-06 2024-09-16T14:49:28.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.847 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.847 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918061+0000", 2024-09-16T14:49:28.847 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1610000000000002e-06 2024-09-16T14:49:28.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.847 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.847 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918065+0000", 2024-09-16T14:49:28.847 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.189e-06 2024-09-16T14:49:28.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.848 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.848 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.925208+0000", 2024-09-16T14:49:28.848 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071427390000000004 2024-09-16T14:49:28.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.848 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:28.848 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.925431+0000", 2024-09-16T14:49:28.848 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000223711 2024-09-16T14:49:28.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.848 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.848 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.942627+0000", 2024-09-16T14:49:28.849 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.017196174000000002 2024-09-16T14:49:28.849 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.849 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.849 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.849 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.as6 77/72 ECSubWrite(tid=14, reqid=client.4500.0:702, at_version=77'33, trim_to=0'0, roll_forward_to=77'32))", 2024-09-16T14:49:28.849 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.918067+0000", 2024-09-16T14:49:28.849 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 4.8260024279999998, 2024-09-16T14:49:28.849 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014135468999999999, 2024-09-16T14:49:28.849 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:28.849 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:28.849 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:28.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.850 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:28.850 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918067+0000", 2024-09-16T14:49:28.850 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.850 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:28.850 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918067+0000", 2024-09-16T14:49:28.850 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:28.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.850 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:28.850 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918066+0000", 2024-09-16T14:49:28.850 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:28.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.851 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:28.851 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918175+0000", 2024-09-16T14:49:28.851 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000109197 2024-09-16T14:49:28.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.851 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:28.851 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918185+0000", 2024-09-16T14:49:28.851 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9110000000000005e-06 2024-09-16T14:49:28.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.852 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:28.852 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.918192+0000", 2024-09-16T14:49:28.852 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.0650000000000001e-06 2024-09-16T14:49:28.852 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.852 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.852 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:28.852 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.924862+0000", 2024-09-16T14:49:28.852 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0066701479999999999 2024-09-16T14:49:28.852 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.852 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.852 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:28.852 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.924871+0000", 2024-09-16T14:49:28.852 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4360000000000002e-06 2024-09-16T14:49:28.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.853 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:28.853 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.932181+0000", 2024-09-16T14:49:28.853 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073105619999999996 2024-09-16T14:49:28.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:28.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:28.853 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:28.853 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.932202+0000", 2024-09-16T14:49:28.853 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0764999999999999e-05 2024-09-16T14:49:28.853 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.853 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.854 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.854 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:28.854 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:28.854 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:28.855 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:49:28.856 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:28.856 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:28.856 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:28.857 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:28.857 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:28.854+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:28.929 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is unset 2024-09-16T14:49:28.930 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T14:49:28.930 INFO:tasks.rados.rados.0.smithi040.stdout:312: done (4 left) 2024-09-16T14:49:28.930 INFO:tasks.rados.rados.0.smithi040.stdout:313: rollback oid 39 current snap is 27 2024-09-16T14:49:28.930 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 39 to 24 2024-09-16T14:49:28.930 INFO:tasks.rados.rados.0.smithi040.stdout:307: finishing write tid 3 to smithi04040151-1 2024-09-16T14:49:28.930 INFO:tasks.rados.rados.0.smithi040.stdout:307: finishing write tid 4 to smithi04040151-1 2024-09-16T14:49:28.930 INFO:tasks.rados.rados.0.smithi040.stdout:308: finishing copy_from racing read to smithi04040151-49 2024-09-16T14:49:28.930 INFO:tasks.rados.rados.0.smithi040.stdout:307: finishing write tid 5 to smithi04040151-1 2024-09-16T14:49:28.930 INFO:tasks.rados.rados.0.smithi040.stdout:308: finishing copy_from to smithi04040151-49 2024-09-16T14:49:28.930 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 49 v 4 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-16T14:49:28.930 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 9 v 104 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-16T14:49:28.931 INFO:tasks.rados.rados.0.smithi040.stdout:307: finishing write tid 6 to smithi04040151-1 2024-09-16T14:49:28.931 INFO:tasks.rados.rados.0.smithi040.stdout:307: oid 1 updating version 0 to 61 2024-09-16T14:49:28.931 INFO:tasks.rados.rados.0.smithi040.stdout:307: oid 1 version 61 is already newer than 60 2024-09-16T14:49:28.931 INFO:tasks.rados.rados.0.smithi040.stdout:307: oid 1 version 61 is already newer than 59 2024-09-16T14:49:28.931 INFO:tasks.rados.rados.0.smithi040.stdout:307: oid 1 updating version 61 to 63 2024-09-16T14:49:28.931 INFO:tasks.rados.rados.0.smithi040.stdout:307: oid 1 version 63 is already newer than 62 2024-09-16T14:49:28.931 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 1 v 63 (ObjNum 117 snap 27 seq_num 117) dirty exists 2024-09-16T14:49:28.931 INFO:tasks.rados.rados.0.smithi040.stdout:307: left oid 1 (ObjNum 117 snap 27 seq_num 117) 2024-09-16T14:49:28.931 INFO:tasks.rados.rados.0.smithi040.stdout:310: expect (ObjNum 104 snap 22 seq_num 104) 2024-09-16T14:49:28.950 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T14:49:28.950 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:49:28.990 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:28.990 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:28.990 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:28.990 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:28.993 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:28.991+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:28.999 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:49:29.037 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:29.037 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:29.053 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:29.056 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:29.055+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:29.070 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:29.070 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:29.076 INFO:tasks.rados.rados.0.smithi040.stdout:313: finishing rollback tid 0 to smithi04040151-39 2024-09-16T14:49:29.076 INFO:tasks.rados.rados.0.smithi040.stdout:313: finishing rollback tid 1 to smithi04040151-39 2024-09-16T14:49:29.077 INFO:tasks.rados.rados.0.smithi040.stdout:313: finishing rollback tid 2 to smithi04040151-39 2024-09-16T14:49:29.077 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 39 v 57 (ObjNum 105 snap 24 seq_num 105) dirty exists 2024-09-16T14:49:29.077 INFO:tasks.rados.rados.0.smithi040.stdout:307: done (4 left) 2024-09-16T14:49:29.077 INFO:tasks.rados.rados.0.smithi040.stdout:308: done (3 left) 2024-09-16T14:49:29.077 INFO:tasks.rados.rados.0.smithi040.stdout:309: done (2 left) 2024-09-16T14:49:29.077 INFO:tasks.rados.rados.0.smithi040.stdout:310: done (1 left) 2024-09-16T14:49:29.077 INFO:tasks.rados.rados.0.smithi040.stdout:313: done (0 left) 2024-09-16T14:49:29.077 INFO:tasks.rados.rados.0.smithi040.stdout:314: read oid 17 snap -1 2024-09-16T14:49:29.077 INFO:tasks.rados.rados.0.smithi040.stdout:314: expect (ObjNum 31 snap 0 seq_num 31) 2024-09-16T14:49:29.077 INFO:tasks.rados.rados.0.smithi040.stdout:315: setattr oid 26 current snap is 27 2024-09-16T14:49:29.078 INFO:tasks.rados.rados.0.smithi040.stdout:316: copy_from oid 20 from oid 1 current snap is 27 2024-09-16T14:49:29.078 INFO:tasks.rados.rados.0.smithi040.stdout:317: snap_remove snap 24 2024-09-16T14:49:29.090 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:29.090 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:29.107 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:29.107 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:29.115 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:29.115 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:29.116 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:49:29.133 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:29.133 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:29.147 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:29.148 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:29.157 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:29.155+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:29.164 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:29.164 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:29.181 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:29.181 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:29.197 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:29.197 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:29.258 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:29.258 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:29.258 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:29.256+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:29.267 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:49:29.290 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:49:29.291 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:49:29.302 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-16T14:49:28.923908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":78152243,"num_objects":64,"num_object_clones":14,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":479,"num_read_kb":89983,"num_write":910,"num_write_kb":98220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":31408128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1034,"ondisk_log_size":1034,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":518424,"kb_used_data":161476,"kb_used_omap":19,"kb_used_meta":356716,"kb_avail":1131943656,"statfs":{"total":1159641169920,"available":1159110303744,"internally_reserved":0,"allocated":165351424,"data_stored":189059136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19502,"internal_metadata":365278162},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9404416,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":-1,"num_read":26,"num_read_kb":29681,"num_write":42,"num_write_kb":9792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.024070"},"pg_stats":[{"pgid":"3.f","version":"79'40","reported_seq":203,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.925158+0000","last_change":"2024-09-16T14:49:25.925849+0000","last_active":"2024-09-16T14:49:25.925158+0000","last_peered":"2024-09-16T14:49:25.925158+0000","last_clean":"2024-09-16T14:49:25.925158+0000","last_became_active":"2024-09-16T14:48:35.075111+0000","last_became_peered":"2024-09-16T14:48:35.075111+0000","last_unstale":"2024-09-16T14:49:25.925158+0000","last_undegraded":"2024-09-16T14:49:25.925158+0000","last_fullsized":"2024-09-16T14:49:25.925158+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:58.979883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014227001,"stat_sum":{"num_bytes":5144576,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":36,"num_write_kb":5424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"3.e","version":"79'56","reported_seq":228,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T14:49:26.916219+0000","last_change":"2024-09-16T14:49:26.916645+0000","last_active":"2024-09-16T14:49:26.916219+0000","last_peered":"2024-09-16T14:49:26.916219+0000","last_clean":"2024-09-16T14:49:26.916219+0000","last_became_active":"2024-09-16T14:48:35.075626+0000","last_became_peered":"2024-09-16T14:48:35.075626+0000","last_unstale":"2024-09-16T14:49:26.916219+0000","last_undegraded":"2024-09-16T14:49:26.916219+0000","last_fullsized":"2024-09-16T14:49:26.916219+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.908348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000549626,"stat_sum":{"num_bytes":6520832,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4438,"num_write":60,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"},{"start":"31","length":"1"}]},{"pgid":"3.d","version":"68'57","reported_seq":277,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.911972+0000","last_change":"2024-09-16T14:49:25.913782+0000","last_active":"2024-09-16T14:49:25.911972+0000","last_peered":"2024-09-16T14:49:25.911972+0000","last_clean":"2024-09-16T14:49:25.911972+0000","last_became_active":"2024-09-16T14:49:12.979812+0000","last_became_peered":"2024-09-16T14:49:12.979812+0000","last_unstale":"2024-09-16T14:49:25.911972+0000","last_undegraded":"2024-09-16T14:49:25.911972+0000","last_fullsized":"2024-09-16T14:49:25.911972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:42.685792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020135090000000001,"stat_sum":{"num_bytes":6160384,"num_objects":4,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":31,"num_read_kb":6204,"num_write":52,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4489216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"3.c","version":"79'56","reported_seq":265,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.949541+0000","last_change":"2024-09-16T14:49:25.911762+0000","last_active":"2024-09-16T14:49:25.949541+0000","last_peered":"2024-09-16T14:49:25.949541+0000","last_clean":"2024-09-16T14:49:25.949541+0000","last_became_active":"2024-09-16T14:49:12.979797+0000","last_became_peered":"2024-09-16T14:49:12.979797+0000","last_unstale":"2024-09-16T14:49:25.949541+0000","last_undegraded":"2024-09-16T14:49:25.949541+0000","last_fullsized":"2024-09-16T14:49:25.949541+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.201481+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0085036810000000008,"stat_sum":{"num_bytes":4833280,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":27,"num_read_kb":9848,"num_write":24,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3440640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"3.b","version":"79'57","reported_seq":246,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.925112+0000","last_change":"2024-09-16T14:49:25.926204+0000","last_active":"2024-09-16T14:49:25.925112+0000","last_peered":"2024-09-16T14:49:25.925112+0000","last_clean":"2024-09-16T14:49:25.925112+0000","last_became_active":"2024-09-16T14:49:12.972366+0000","last_became_peered":"2024-09-16T14:49:12.972366+0000","last_unstale":"2024-09-16T14:49:25.925112+0000","last_undegraded":"2024-09-16T14:49:25.925112+0000","last_fullsized":"2024-09-16T14:49:25.925112+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:03.456665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017735721999999999,"stat_sum":{"num_bytes":3522560,"num_objects":5,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":4902,"num_write":38,"num_write_kb":3440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1785856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,1,6],"acting":[3,0,4,8,11,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"3.a","version":"81'45","reported_seq":113,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T14:49:27.932083+0000","last_change":"2024-09-16T14:49:27.932897+0000","last_active":"2024-09-16T14:49:27.932083+0000","last_peered":"2024-09-16T14:49:27.932083+0000","last_clean":"2024-09-16T14:49:27.932083+0000","last_became_active":"2024-09-16T14:49:19.868841+0000","last_became_peered":"2024-09-16T14:49:19.868841+0000","last_unstale":"2024-09-16T14:49:27.932083+0000","last_undegraded":"2024-09-16T14:49:27.932083+0000","last_fullsized":"2024-09-16T14:49:27.932083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:45.972141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084944459999999992,"stat_sum":{"num_bytes":5357568,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":1062,"num_write":40,"num_write_kb":5584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":9224192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,10,8,9,3,7],"acting":[6,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.9","version":"79'63","reported_seq":241,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.921962+0000","last_change":"2024-09-16T14:49:25.923372+0000","last_active":"2024-09-16T14:49:25.921962+0000","last_peered":"2024-09-16T14:49:25.921962+0000","last_clean":"2024-09-16T14:49:25.921962+0000","last_became_active":"2024-09-16T14:48:35.075076+0000","last_became_peered":"2024-09-16T14:48:35.075076+0000","last_unstale":"2024-09-16T14:49:25.921962+0000","last_undegraded":"2024-09-16T14:49:25.921962+0000","last_fullsized":"2024-09-16T14:49:25.921962+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:46.349013+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015876023999999999,"stat_sum":{"num_bytes":6144000,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":27,"num_read_kb":2555,"num_write":28,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"3.8","version":"79'93","reported_seq":267,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T14:49:26.916280+0000","last_change":"2024-09-16T14:49:26.917640+0000","last_active":"2024-09-16T14:49:26.916280+0000","last_peered":"2024-09-16T14:49:26.916280+0000","last_clean":"2024-09-16T14:49:26.916280+0000","last_became_active":"2024-09-16T14:48:34.881078+0000","last_became_peered":"2024-09-16T14:48:34.881078+0000","last_unstale":"2024-09-16T14:49:26.916280+0000","last_undegraded":"2024-09-16T14:49:26.916280+0000","last_fullsized":"2024-09-16T14:49:26.916280+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:03.772938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001533146,"stat_sum":{"num_bytes":3899392,"num_objects":4,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":37,"num_read_kb":15126,"num_write":92,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2,4,3,5,10,8],"acting":[11,1,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"},{"start":"31","length":"1"}]},{"pgid":"1.0","version":"20'137","reported_seq":242,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.225891+0000","last_change":"2024-09-16T14:48:22.359960+0000","last_active":"2024-09-16T14:49:25.225891+0000","last_peered":"2024-09-16T14:49:25.225891+0000","last_clean":"2024-09-16T14:49:25.225891+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:49:25.225891+0000","last_undegraded":"2024-09-16T14:49:25.225891+0000","last_fullsized":"2024-09-16T14:49:25.225891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:48:21.118926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:37.878583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":73,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T14:49:24.901497+0000","last_change":"2024-09-16T14:48:26.159458+0000","last_active":"2024-09-16T14:49:24.901497+0000","last_peered":"2024-09-16T14:49:24.901497+0000","last_clean":"2024-09-16T14:49:24.901497+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:49:24.901497+0000","last_undegraded":"2024-09-16T14:49:24.901497+0000","last_fullsized":"2024-09-16T14:49:24.901497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:33.086957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"77'100","reported_seq":307,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.908636+0000","last_change":"2024-09-16T14:49:25.912589+0000","last_active":"2024-09-16T14:49:25.908636+0000","last_peered":"2024-09-16T14:49:25.908636+0000","last_clean":"2024-09-16T14:49:25.908636+0000","last_became_active":"2024-09-16T14:49:12.975112+0000","last_became_peered":"2024-09-16T14:49:12.975112+0000","last_unstale":"2024-09-16T14:49:25.908636+0000","last_undegraded":"2024-09-16T14:49:25.908636+0000","last_fullsized":"2024-09-16T14:49:25.908636+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:42.640118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004737804,"stat_sum":{"num_bytes":6176768,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":6825,"num_write":39,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4259840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.225803+0000","last_change":"2024-09-16T14:48:26.568303+0000","last_active":"2024-09-16T14:49:25.225803+0000","last_peered":"2024-09-16T14:49:25.225803+0000","last_clean":"2024-09-16T14:49:25.225803+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:49:25.225803+0000","last_undegraded":"2024-09-16T14:49:25.225803+0000","last_fullsized":"2024-09-16T14:49:25.225803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:35.481460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":178,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T14:49:26.921556+0000","last_change":"2024-09-16T14:49:26.921911+0000","last_active":"2024-09-16T14:49:26.921556+0000","last_peered":"2024-09-16T14:49:26.921556+0000","last_clean":"2024-09-16T14:49:26.921556+0000","last_became_active":"2024-09-16T14:49:12.975153+0000","last_became_peered":"2024-09-16T14:49:12.975153+0000","last_unstale":"2024-09-16T14:49:26.921556+0000","last_undegraded":"2024-09-16T14:49:26.921556+0000","last_fullsized":"2024-09-16T14:49:26.921556+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:35.471823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049311699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T14:49:24.901550+0000","last_change":"2024-09-16T14:48:26.156468+0000","last_active":"2024-09-16T14:49:24.901550+0000","last_peered":"2024-09-16T14:49:24.901550+0000","last_clean":"2024-09-16T14:49:24.901550+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:49:24.901550+0000","last_undegraded":"2024-09-16T14:49:24.901550+0000","last_fullsized":"2024-09-16T14:49:24.901550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"79'39","reported_seq":233,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T14:49:26.916703+0000","last_change":"2024-09-16T14:49:26.917035+0000","last_active":"2024-09-16T14:49:26.916703+0000","last_peered":"2024-09-16T14:49:26.916703+0000","last_clean":"2024-09-16T14:49:26.916703+0000","last_became_active":"2024-09-16T14:49:12.974901+0000","last_became_peered":"2024-09-16T14:49:12.974901+0000","last_unstale":"2024-09-16T14:49:26.916703+0000","last_undegraded":"2024-09-16T14:49:26.916703+0000","last_fullsized":"2024-09-16T14:49:26.916703+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.845606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078664200000000005,"stat_sum":{"num_bytes":5767168,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":1111,"num_write":21,"num_write_kb":6000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1294336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":74,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T14:49:24.901383+0000","last_change":"2024-09-16T14:48:26.159919+0000","last_active":"2024-09-16T14:49:24.901383+0000","last_peered":"2024-09-16T14:49:24.901383+0000","last_clean":"2024-09-16T14:49:24.901383+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:49:24.901383+0000","last_undegraded":"2024-09-16T14:49:24.901383+0000","last_fullsized":"2024-09-16T14:49:24.901383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"66'27","reported_seq":229,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.904464+0000","last_change":"2024-09-16T14:49:25.904809+0000","last_active":"2024-09-16T14:49:25.904464+0000","last_peered":"2024-09-16T14:49:25.904464+0000","last_clean":"2024-09-16T14:49:25.904464+0000","last_became_active":"2024-09-16T14:49:12.979630+0000","last_became_peered":"2024-09-16T14:49:12.979630+0000","last_unstale":"2024-09-16T14:49:25.904464+0000","last_undegraded":"2024-09-16T14:49:25.904464+0000","last_fullsized":"2024-09-16T14:49:25.904464+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:51.166853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070440300000000004,"stat_sum":{"num_bytes":1736704,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":858,"num_write":32,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":81920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"3.5","version":"79'53","reported_seq":304,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.929737+0000","last_change":"2024-09-16T14:49:25.911425+0000","last_active":"2024-09-16T14:49:25.929737+0000","last_peered":"2024-09-16T14:49:25.929737+0000","last_clean":"2024-09-16T14:49:25.929737+0000","last_became_active":"2024-09-16T14:49:19.867059+0000","last_became_peered":"2024-09-16T14:49:19.867059+0000","last_unstale":"2024-09-16T14:49:25.929737+0000","last_undegraded":"2024-09-16T14:49:25.929737+0000","last_fullsized":"2024-09-16T14:49:25.929737+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:41.579001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082413220000000006,"stat_sum":{"num_bytes":4374528,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":3942,"num_write":17,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":3440640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,1,8,10],"acting":[5,3,2,6,7,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":72,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.913021+0000","last_change":"2024-09-16T14:48:26.161822+0000","last_active":"2024-09-16T14:49:25.913021+0000","last_peered":"2024-09-16T14:49:25.913021+0000","last_clean":"2024-09-16T14:49:25.913021+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:49:25.913021+0000","last_undegraded":"2024-09-16T14:49:25.913021+0000","last_fullsized":"2024-09-16T14:49:25.913021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:48.998147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"80'64","reported_seq":267,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T14:49:26.920883+0000","last_change":"2024-09-16T14:49:26.921573+0000","last_active":"2024-09-16T14:49:26.920883+0000","last_peered":"2024-09-16T14:49:26.920883+0000","last_clean":"2024-09-16T14:49:26.920883+0000","last_became_active":"2024-09-16T14:49:12.982763+0000","last_became_peered":"2024-09-16T14:49:12.982763+0000","last_unstale":"2024-09-16T14:49:26.920883+0000","last_undegraded":"2024-09-16T14:49:26.920883+0000","last_fullsized":"2024-09-16T14:49:26.920883+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:00.568854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063023250000000001,"stat_sum":{"num_bytes":3391488,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":3466,"num_write":80,"num_write_kb":5024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":3391488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,9,5,10,2,6,0],"acting":[1,11,9,5,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.227026+0000","last_change":"2024-09-16T14:48:26.156486+0000","last_active":"2024-09-16T14:49:25.227026+0000","last_peered":"2024-09-16T14:49:25.227026+0000","last_clean":"2024-09-16T14:49:25.227026+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:49:25.227026+0000","last_undegraded":"2024-09-16T14:49:25.227026+0000","last_fullsized":"2024-09-16T14:49:25.227026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.015966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"79'102","reported_seq":282,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.947496+0000","last_change":"2024-09-16T14:49:25.930747+0000","last_active":"2024-09-16T14:49:25.947496+0000","last_peered":"2024-09-16T14:49:25.947496+0000","last_clean":"2024-09-16T14:49:25.947496+0000","last_became_active":"2024-09-16T14:48:35.075213+0000","last_became_peered":"2024-09-16T14:48:35.075213+0000","last_unstale":"2024-09-16T14:49:25.947496+0000","last_undegraded":"2024-09-16T14:49:25.947496+0000","last_fullsized":"2024-09-16T14:49:25.947496+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:38.895447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021874023999999999,"stat_sum":{"num_bytes":10731520,"num_objects":6,"num_object_clones":1,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":28660,"num_write":106,"num_write_kb":11920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.915396+0000","last_change":"2024-09-16T14:48:26.161840+0000","last_active":"2024-09-16T14:49:25.915396+0000","last_peered":"2024-09-16T14:49:25.915396+0000","last_clean":"2024-09-16T14:49:25.915396+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:49:25.915396+0000","last_undegraded":"2024-09-16T14:49:25.915396+0000","last_fullsized":"2024-09-16T14:49:25.915396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.436189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"79'42","reported_seq":220,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T14:49:26.922198+0000","last_change":"2024-09-16T14:49:26.922198+0000","last_active":"2024-09-16T14:49:26.922198+0000","last_peered":"2024-09-16T14:49:26.922198+0000","last_clean":"2024-09-16T14:49:26.922198+0000","last_became_active":"2024-09-16T14:48:35.076301+0000","last_became_peered":"2024-09-16T14:48:35.076301+0000","last_unstale":"2024-09-16T14:49:26.922198+0000","last_undegraded":"2024-09-16T14:49:26.922198+0000","last_fullsized":"2024-09-16T14:49:26.922198+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:50.342709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071213299999999999,"stat_sum":{"num_bytes":3801088,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":872,"num_write":62,"num_write_kb":5392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,5],"acting":[0,1,4,7,8,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T14:49:26.911533+0000","last_change":"2024-09-16T14:49:12.961994+0000","last_active":"2024-09-16T14:49:26.911533+0000","last_peered":"2024-09-16T14:49:26.911533+0000","last_clean":"2024-09-16T14:49:26.911533+0000","last_became_active":"2024-09-16T14:49:12.961697+0000","last_became_peered":"2024-09-16T14:49:12.961697+0000","last_unstale":"2024-09-16T14:49:26.911533+0000","last_undegraded":"2024-09-16T14:49:26.911533+0000","last_fullsized":"2024-09-16T14:49:26.911533+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:46.679090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":77561856,"num_objects":60,"num_object_clones":14,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":353,"num_read_kb":89874,"num_write":727,"num_write_kb":95280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":31408128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":156774400,"data_stored":188182528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":894,"ondisk_log_size":894,"up":128,"acting":128,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411343,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46656,"kb_used_data":15536,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94325184,"statfs":{"total":96636764160,"available":96588988416,"internally_reserved":0,"allocated":15908864,"data_stored":18438067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.434}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.47299999999999998}]},{"osd":2,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.372}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.42299999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.58799999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.502}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53600000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.625}]}]},{"osd":10,"up_from":18,"seq":77309411345,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47176,"kb_used_data":16052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94324664,"statfs":{"total":96636764160,"available":96588455936,"internally_reserved":0,"allocated":16437248,"data_stored":18547159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.36399999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.41699999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.375},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.33900000000000002}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.63400000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.47199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.502},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58099999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45600000000000002}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.439},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.48099999999999998}]}]},{"osd":6,"up_from":41,"seq":176093659146,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38416,"kb_used_data":11516,"kb_used_omap":1,"kb_used_meta":26878,"kb_avail":94333424,"statfs":{"total":96636764160,"available":96597426176,"internally_reserved":0,"allocated":11792384,"data_stored":14411324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1965,"internal_metadata":27523155},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":1,"up_from":18,"seq":77309411345,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46592,"kb_used_data":15468,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94325248,"statfs":{"total":96636764160,"available":96589053952,"internally_reserved":0,"allocated":15839232,"data_stored":18384294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.32500000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.0590000000000002,"5min":4.0590000000000002,"15min":4.0590000000000002},"last":0.46300000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":3.9620000000000002,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.53100000000000003}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.62}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.67500000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":11,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":4.6589999999999998,"5min":4.6589999999999998,"15min":4.6589999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.69499999999999995}]}]},{"osd":0,"up_from":18,"seq":77309411343,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44616,"kb_used_data":13492,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327224,"statfs":{"total":96636764160,"available":96591077376,"internally_reserved":0,"allocated":13815808,"data_stored":16229779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.40500000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45000000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.5},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.64400000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.59099999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.626},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.70299999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":11,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66100000000000003}]}]},{"osd":5,"up_from":18,"seq":77309411344,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42984,"kb_used_data":11864,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94328856,"statfs":{"total":96636764160,"available":96592748544,"internally_reserved":0,"allocated":12148736,"data_stored":13350107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.45400000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.502},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.252},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.34599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.432}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.63},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52500000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.67400000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.58199999999999996}]}]},{"osd":9,"up_from":17,"seq":73014444048,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47732,"kb_used_data":16612,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94324108,"statfs":{"total":96636764160,"available":96587886592,"internally_reserved":0,"allocated":17010688,"data_stored":19395784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.35299999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.55200000000000005}]},{"osd":2,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.51800000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.379},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.315}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.47299999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.46300000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.60799999999999998}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.499},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.57999999999999996}]},{"osd":11,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.40899999999999997}]}]},{"osd":8,"up_from":17,"seq":73014444048,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40300,"kb_used_data":13272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331540,"statfs":{"total":96636764160,"available":96595496960,"internally_reserved":0,"allocated":13590528,"data_stored":16000403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.38500000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52500000000000002}]},{"osd":4,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.629},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.67300000000000004}]},{"osd":9,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.65700000000000003}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.69699999999999995}]},{"osd":11,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.72699999999999998}]}]},{"osd":7,"up_from":17,"seq":73014444048,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44984,"kb_used_data":13864,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94326856,"statfs":{"total":96636764160,"available":96590700544,"internally_reserved":0,"allocated":14196736,"data_stored":15992755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.501},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5620000000000003,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.51300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.45300000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58599999999999997}]},{"osd":4,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.34799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":8,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.44},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.42699999999999999}]},{"osd":9,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.46600000000000003}]},{"osd":10,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.48899999999999999}]},{"osd":11,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.55000000000000004}]}]},{"osd":4,"up_from":17,"seq":73014444047,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45976,"kb_used_data":14856,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94325864,"statfs":{"total":96636764160,"available":96589684736,"internally_reserved":0,"allocated":15212544,"data_stored":17458376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.437}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.41399999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.499},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.48899999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.379},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.32500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47699999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.71199999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72799999999999998}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.58599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":3,"up_from":17,"seq":73014444049,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40124,"kb_used_data":13100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331716,"statfs":{"total":96636764160,"available":96595677184,"internally_reserved":0,"allocated":13414400,"data_stored":14562504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.65700000000000003}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.73299999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.71099999999999997}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":1.069},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.89400000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.0069999999999999}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.876}]},{"osd":11,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.90900000000000003}]}]},{"osd":2,"up_from":17,"seq":73014444048,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":5844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":5984256,"data_stored":6288584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.35299999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.83299999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.56899999999999995}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.49099999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":1.083},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64300000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":1.042}]},{"osd":10,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.752}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13438976,"data_stored":17371136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14962688,"data_stored":18190336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5410816,"data_stored":6529024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13328384,"data_stored":15171584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14798848,"data_stored":17018880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11571200,"data_stored":12869632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10866688,"data_stored":13848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12750848,"data_stored":14909440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12677120,"data_stored":15446016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16441344,"data_stored":19259392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":15527936,"data_stored":18518016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":14999552,"data_stored":19050496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:49:29.306 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:49:29.360 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:29.358+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:29.410 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:29.410 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:29.419 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:49:29.460 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:29.459+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:29.520 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:29.521 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:29.529 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:49:29.562 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:29.560+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:49:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:49:29.603 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-16T14:49:28.923908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":78152243,"num_objects":64,"num_object_clones":14,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":479,"num_read_kb":89983,"num_write":910,"num_write_kb":98220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":31408128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1034,"ondisk_log_size":1034,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":518424,"kb_used_data":161476,"kb_used_omap":19,"kb_used_meta":356716,"kb_avail":1131943656,"statfs":{"total":1159641169920,"available":1159110303744,"internally_reserved":0,"allocated":165351424,"data_stored":189059136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19502,"internal_metadata":365278162},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9404416,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":-1,"num_read":26,"num_read_kb":29681,"num_write":42,"num_write_kb":9792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.024070"},"pg_stats":[{"pgid":"3.f","version":"79'40","reported_seq":203,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.925158+0000","last_change":"2024-09-16T14:49:25.925849+0000","last_active":"2024-09-16T14:49:25.925158+0000","last_peered":"2024-09-16T14:49:25.925158+0000","last_clean":"2024-09-16T14:49:25.925158+0000","last_became_active":"2024-09-16T14:48:35.075111+0000","last_became_peered":"2024-09-16T14:48:35.075111+0000","last_unstale":"2024-09-16T14:49:25.925158+0000","last_undegraded":"2024-09-16T14:49:25.925158+0000","last_fullsized":"2024-09-16T14:49:25.925158+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:58.979883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014227001,"stat_sum":{"num_bytes":5144576,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":36,"num_write_kb":5424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"3.e","version":"79'56","reported_seq":228,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T14:49:26.916219+0000","last_change":"2024-09-16T14:49:26.916645+0000","last_active":"2024-09-16T14:49:26.916219+0000","last_peered":"2024-09-16T14:49:26.916219+0000","last_clean":"2024-09-16T14:49:26.916219+0000","last_became_active":"2024-09-16T14:48:35.075626+0000","last_became_peered":"2024-09-16T14:48:35.075626+0000","last_unstale":"2024-09-16T14:49:26.916219+0000","last_undegraded":"2024-09-16T14:49:26.916219+0000","last_fullsized":"2024-09-16T14:49:26.916219+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.908348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000549626,"stat_sum":{"num_bytes":6520832,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4438,"num_write":60,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"},{"start":"31","length":"1"}]},{"pgid":"3.d","version":"68'57","reported_seq":277,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.911972+0000","last_change":"2024-09-16T14:49:25.913782+0000","last_active":"2024-09-16T14:49:25.911972+0000","last_peered":"2024-09-16T14:49:25.911972+0000","last_clean":"2024-09-16T14:49:25.911972+0000","last_became_active":"2024-09-16T14:49:12.979812+0000","last_became_peered":"2024-09-16T14:49:12.979812+0000","last_unstale":"2024-09-16T14:49:25.911972+0000","last_undegraded":"2024-09-16T14:49:25.911972+0000","last_fullsized":"2024-09-16T14:49:25.911972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:42.685792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020135090000000001,"stat_sum":{"num_bytes":6160384,"num_objects":4,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":31,"num_read_kb":6204,"num_write":52,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4489216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"3.c","version":"79'56","reported_seq":265,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.949541+0000","last_change":"2024-09-16T14:49:25.911762+0000","last_active":"2024-09-16T14:49:25.949541+0000","last_peered":"2024-09-16T14:49:25.949541+0000","last_clean":"2024-09-16T14:49:25.949541+0000","last_became_active":"2024-09-16T14:49:12.979797+0000","last_became_peered":"2024-09-16T14:49:12.979797+0000","last_unstale":"2024-09-16T14:49:25.949541+0000","last_undegraded":"2024-09-16T14:49:25.949541+0000","last_fullsized":"2024-09-16T14:49:25.949541+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.201481+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0085036810000000008,"stat_sum":{"num_bytes":4833280,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":27,"num_read_kb":9848,"num_write":24,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3440640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"3.b","version":"79'57","reported_seq":246,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.925112+0000","last_change":"2024-09-16T14:49:25.926204+0000","last_active":"2024-09-16T14:49:25.925112+0000","last_peered":"2024-09-16T14:49:25.925112+0000","last_clean":"2024-09-16T14:49:25.925112+0000","last_became_active":"2024-09-16T14:49:12.972366+0000","last_became_peered":"2024-09-16T14:49:12.972366+0000","last_unstale":"2024-09-16T14:49:25.925112+0000","last_undegraded":"2024-09-16T14:49:25.925112+0000","last_fullsized":"2024-09-16T14:49:25.925112+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:03.456665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017735721999999999,"stat_sum":{"num_bytes":3522560,"num_objects":5,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":4902,"num_write":38,"num_write_kb":3440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1785856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,1,6],"acting":[3,0,4,8,11,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"3.a","version":"81'45","reported_seq":113,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-16T14:49:27.932083+0000","last_change":"2024-09-16T14:49:27.932897+0000","last_active":"2024-09-16T14:49:27.932083+0000","last_peered":"2024-09-16T14:49:27.932083+0000","last_clean":"2024-09-16T14:49:27.932083+0000","last_became_active":"2024-09-16T14:49:19.868841+0000","last_became_peered":"2024-09-16T14:49:19.868841+0000","last_unstale":"2024-09-16T14:49:27.932083+0000","last_undegraded":"2024-09-16T14:49:27.932083+0000","last_fullsized":"2024-09-16T14:49:27.932083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:45.972141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0084944459999999992,"stat_sum":{"num_bytes":5357568,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":13,"num_read_kb":1062,"num_write":40,"num_write_kb":5584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":9224192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,10,8,9,3,7],"acting":[6,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.9","version":"79'63","reported_seq":241,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.921962+0000","last_change":"2024-09-16T14:49:25.923372+0000","last_active":"2024-09-16T14:49:25.921962+0000","last_peered":"2024-09-16T14:49:25.921962+0000","last_clean":"2024-09-16T14:49:25.921962+0000","last_became_active":"2024-09-16T14:48:35.075076+0000","last_became_peered":"2024-09-16T14:48:35.075076+0000","last_unstale":"2024-09-16T14:49:25.921962+0000","last_undegraded":"2024-09-16T14:49:25.921962+0000","last_fullsized":"2024-09-16T14:49:25.921962+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:46.349013+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015876023999999999,"stat_sum":{"num_bytes":6144000,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":27,"num_read_kb":2555,"num_write":28,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"3.8","version":"79'93","reported_seq":267,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T14:49:26.916280+0000","last_change":"2024-09-16T14:49:26.917640+0000","last_active":"2024-09-16T14:49:26.916280+0000","last_peered":"2024-09-16T14:49:26.916280+0000","last_clean":"2024-09-16T14:49:26.916280+0000","last_became_active":"2024-09-16T14:48:34.881078+0000","last_became_peered":"2024-09-16T14:48:34.881078+0000","last_unstale":"2024-09-16T14:49:26.916280+0000","last_undegraded":"2024-09-16T14:49:26.916280+0000","last_fullsized":"2024-09-16T14:49:26.916280+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:03.772938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001533146,"stat_sum":{"num_bytes":3899392,"num_objects":4,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":37,"num_read_kb":15126,"num_write":92,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2,4,3,5,10,8],"acting":[11,1,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"},{"start":"31","length":"1"}]},{"pgid":"1.0","version":"20'137","reported_seq":242,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.225891+0000","last_change":"2024-09-16T14:48:22.359960+0000","last_active":"2024-09-16T14:49:25.225891+0000","last_peered":"2024-09-16T14:49:25.225891+0000","last_clean":"2024-09-16T14:49:25.225891+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:49:25.225891+0000","last_undegraded":"2024-09-16T14:49:25.225891+0000","last_fullsized":"2024-09-16T14:49:25.225891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:48:21.118926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:37.878583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":73,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T14:49:24.901497+0000","last_change":"2024-09-16T14:48:26.159458+0000","last_active":"2024-09-16T14:49:24.901497+0000","last_peered":"2024-09-16T14:49:24.901497+0000","last_clean":"2024-09-16T14:49:24.901497+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:49:24.901497+0000","last_undegraded":"2024-09-16T14:49:24.901497+0000","last_fullsized":"2024-09-16T14:49:24.901497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:33.086957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"77'100","reported_seq":307,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.908636+0000","last_change":"2024-09-16T14:49:25.912589+0000","last_active":"2024-09-16T14:49:25.908636+0000","last_peered":"2024-09-16T14:49:25.908636+0000","last_clean":"2024-09-16T14:49:25.908636+0000","last_became_active":"2024-09-16T14:49:12.975112+0000","last_became_peered":"2024-09-16T14:49:12.975112+0000","last_unstale":"2024-09-16T14:49:25.908636+0000","last_undegraded":"2024-09-16T14:49:25.908636+0000","last_fullsized":"2024-09-16T14:49:25.908636+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:42.640118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004737804,"stat_sum":{"num_bytes":6176768,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":6825,"num_write":39,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4259840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.225803+0000","last_change":"2024-09-16T14:48:26.568303+0000","last_active":"2024-09-16T14:49:25.225803+0000","last_peered":"2024-09-16T14:49:25.225803+0000","last_clean":"2024-09-16T14:49:25.225803+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:49:25.225803+0000","last_undegraded":"2024-09-16T14:49:25.225803+0000","last_fullsized":"2024-09-16T14:49:25.225803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:35.481460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":178,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T14:49:26.921556+0000","last_change":"2024-09-16T14:49:26.921911+0000","last_active":"2024-09-16T14:49:26.921556+0000","last_peered":"2024-09-16T14:49:26.921556+0000","last_clean":"2024-09-16T14:49:26.921556+0000","last_became_active":"2024-09-16T14:49:12.975153+0000","last_became_peered":"2024-09-16T14:49:12.975153+0000","last_unstale":"2024-09-16T14:49:26.921556+0000","last_undegraded":"2024-09-16T14:49:26.921556+0000","last_fullsized":"2024-09-16T14:49:26.921556+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:35.471823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049311699999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T14:49:24.901550+0000","last_change":"2024-09-16T14:48:26.156468+0000","last_active":"2024-09-16T14:49:24.901550+0000","last_peered":"2024-09-16T14:49:24.901550+0000","last_clean":"2024-09-16T14:49:24.901550+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:49:24.901550+0000","last_undegraded":"2024-09-16T14:49:24.901550+0000","last_fullsized":"2024-09-16T14:49:24.901550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"79'39","reported_seq":233,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T14:49:26.916703+0000","last_change":"2024-09-16T14:49:26.917035+0000","last_active":"2024-09-16T14:49:26.916703+0000","last_peered":"2024-09-16T14:49:26.916703+0000","last_clean":"2024-09-16T14:49:26.916703+0000","last_became_active":"2024-09-16T14:49:12.974901+0000","last_became_peered":"2024-09-16T14:49:12.974901+0000","last_unstale":"2024-09-16T14:49:26.916703+0000","last_undegraded":"2024-09-16T14:49:26.916703+0000","last_fullsized":"2024-09-16T14:49:26.916703+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.845606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078664200000000005,"stat_sum":{"num_bytes":5767168,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":1111,"num_write":21,"num_write_kb":6000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1294336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":74,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T14:49:24.901383+0000","last_change":"2024-09-16T14:48:26.159919+0000","last_active":"2024-09-16T14:49:24.901383+0000","last_peered":"2024-09-16T14:49:24.901383+0000","last_clean":"2024-09-16T14:49:24.901383+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:49:24.901383+0000","last_undegraded":"2024-09-16T14:49:24.901383+0000","last_fullsized":"2024-09-16T14:49:24.901383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"66'27","reported_seq":229,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.904464+0000","last_change":"2024-09-16T14:49:25.904809+0000","last_active":"2024-09-16T14:49:25.904464+0000","last_peered":"2024-09-16T14:49:25.904464+0000","last_clean":"2024-09-16T14:49:25.904464+0000","last_became_active":"2024-09-16T14:49:12.979630+0000","last_became_peered":"2024-09-16T14:49:12.979630+0000","last_unstale":"2024-09-16T14:49:25.904464+0000","last_undegraded":"2024-09-16T14:49:25.904464+0000","last_fullsized":"2024-09-16T14:49:25.904464+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:51.166853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070440300000000004,"stat_sum":{"num_bytes":1736704,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":858,"num_write":32,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":81920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"3.5","version":"79'53","reported_seq":304,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.929737+0000","last_change":"2024-09-16T14:49:25.911425+0000","last_active":"2024-09-16T14:49:25.929737+0000","last_peered":"2024-09-16T14:49:25.929737+0000","last_clean":"2024-09-16T14:49:25.929737+0000","last_became_active":"2024-09-16T14:49:19.867059+0000","last_became_peered":"2024-09-16T14:49:19.867059+0000","last_unstale":"2024-09-16T14:49:25.929737+0000","last_undegraded":"2024-09-16T14:49:25.929737+0000","last_fullsized":"2024-09-16T14:49:25.929737+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:41.579001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0082413220000000006,"stat_sum":{"num_bytes":4374528,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":3942,"num_write":17,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":3440640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,1,8,10],"acting":[5,3,2,6,7,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":72,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.913021+0000","last_change":"2024-09-16T14:48:26.161822+0000","last_active":"2024-09-16T14:49:25.913021+0000","last_peered":"2024-09-16T14:49:25.913021+0000","last_clean":"2024-09-16T14:49:25.913021+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:49:25.913021+0000","last_undegraded":"2024-09-16T14:49:25.913021+0000","last_fullsized":"2024-09-16T14:49:25.913021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:48.998147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"80'64","reported_seq":267,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T14:49:26.920883+0000","last_change":"2024-09-16T14:49:26.921573+0000","last_active":"2024-09-16T14:49:26.920883+0000","last_peered":"2024-09-16T14:49:26.920883+0000","last_clean":"2024-09-16T14:49:26.920883+0000","last_became_active":"2024-09-16T14:49:12.982763+0000","last_became_peered":"2024-09-16T14:49:12.982763+0000","last_unstale":"2024-09-16T14:49:26.920883+0000","last_undegraded":"2024-09-16T14:49:26.920883+0000","last_fullsized":"2024-09-16T14:49:26.920883+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:00.568854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063023250000000001,"stat_sum":{"num_bytes":3391488,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":3466,"num_write":80,"num_write_kb":5024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":3391488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,9,5,10,2,6,0],"acting":[1,11,9,5,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.227026+0000","last_change":"2024-09-16T14:48:26.156486+0000","last_active":"2024-09-16T14:49:25.227026+0000","last_peered":"2024-09-16T14:49:25.227026+0000","last_clean":"2024-09-16T14:49:25.227026+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:49:25.227026+0000","last_undegraded":"2024-09-16T14:49:25.227026+0000","last_fullsized":"2024-09-16T14:49:25.227026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.015966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"79'102","reported_seq":282,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.947496+0000","last_change":"2024-09-16T14:49:25.930747+0000","last_active":"2024-09-16T14:49:25.947496+0000","last_peered":"2024-09-16T14:49:25.947496+0000","last_clean":"2024-09-16T14:49:25.947496+0000","last_became_active":"2024-09-16T14:48:35.075213+0000","last_became_peered":"2024-09-16T14:48:35.075213+0000","last_unstale":"2024-09-16T14:49:25.947496+0000","last_undegraded":"2024-09-16T14:49:25.947496+0000","last_fullsized":"2024-09-16T14:49:25.947496+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:38.895447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.021874023999999999,"stat_sum":{"num_bytes":10731520,"num_objects":6,"num_object_clones":1,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":28660,"num_write":106,"num_write_kb":11920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":72,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-16T14:49:25.915396+0000","last_change":"2024-09-16T14:48:26.161840+0000","last_active":"2024-09-16T14:49:25.915396+0000","last_peered":"2024-09-16T14:49:25.915396+0000","last_clean":"2024-09-16T14:49:25.915396+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:49:25.915396+0000","last_undegraded":"2024-09-16T14:49:25.915396+0000","last_fullsized":"2024-09-16T14:49:25.915396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.436189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"79'42","reported_seq":220,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T14:49:26.922198+0000","last_change":"2024-09-16T14:49:26.922198+0000","last_active":"2024-09-16T14:49:26.922198+0000","last_peered":"2024-09-16T14:49:26.922198+0000","last_clean":"2024-09-16T14:49:26.922198+0000","last_became_active":"2024-09-16T14:48:35.076301+0000","last_became_peered":"2024-09-16T14:48:35.076301+0000","last_unstale":"2024-09-16T14:49:26.922198+0000","last_undegraded":"2024-09-16T14:49:26.922198+0000","last_fullsized":"2024-09-16T14:49:26.922198+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:50.342709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071213299999999999,"stat_sum":{"num_bytes":3801088,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":872,"num_write":62,"num_write_kb":5392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,5],"acting":[0,1,4,7,8,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"25","length":"2"},{"start":"2a","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-16T14:49:26.911533+0000","last_change":"2024-09-16T14:49:12.961994+0000","last_active":"2024-09-16T14:49:26.911533+0000","last_peered":"2024-09-16T14:49:26.911533+0000","last_clean":"2024-09-16T14:49:26.911533+0000","last_became_active":"2024-09-16T14:49:12.961697+0000","last_became_peered":"2024-09-16T14:49:12.961697+0000","last_unstale":"2024-09-16T14:49:26.911533+0000","last_undegraded":"2024-09-16T14:49:26.911533+0000","last_fullsized":"2024-09-16T14:49:26.911533+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:46.679090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":77561856,"num_objects":60,"num_object_clones":14,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":353,"num_read_kb":89874,"num_write":727,"num_write_kb":95280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":31408128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":156774400,"data_stored":188182528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":894,"ondisk_log_size":894,"up":128,"acting":128,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411343,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46656,"kb_used_data":15536,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94325184,"statfs":{"total":96636764160,"available":96588988416,"internally_reserved":0,"allocated":15908864,"data_stored":18438067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31848901},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.434}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.47299999999999998}]},{"osd":2,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.372}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.42299999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.58799999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.502}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53600000000000003}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.625}]}]},{"osd":10,"up_from":18,"seq":77309411345,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47176,"kb_used_data":16052,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94324664,"statfs":{"total":96636764160,"available":96588455936,"internally_reserved":0,"allocated":16437248,"data_stored":18547159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.36399999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.41699999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.375},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.33900000000000002}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.63400000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.47199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.502},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58099999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45600000000000002}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.439},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.48099999999999998}]}]},{"osd":6,"up_from":41,"seq":176093659146,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38416,"kb_used_data":11516,"kb_used_omap":1,"kb_used_meta":26878,"kb_avail":94333424,"statfs":{"total":96636764160,"available":96597426176,"internally_reserved":0,"allocated":11792384,"data_stored":14411324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1965,"internal_metadata":27523155},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":1,"up_from":18,"seq":77309411345,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46592,"kb_used_data":15468,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94325248,"statfs":{"total":96636764160,"available":96589053952,"internally_reserved":0,"allocated":15839232,"data_stored":18384294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.32500000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.0590000000000002,"5min":4.0590000000000002,"15min":4.0590000000000002},"last":0.46300000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":3.9620000000000002,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.53100000000000003}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.62}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.67500000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":11,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":4.6589999999999998,"5min":4.6589999999999998,"15min":4.6589999999999998},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.69499999999999995}]}]},{"osd":0,"up_from":18,"seq":77309411343,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44616,"kb_used_data":13492,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327224,"statfs":{"total":96636764160,"available":96591077376,"internally_reserved":0,"allocated":13815808,"data_stored":16229779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.40500000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45000000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.35999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.5},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.64400000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.59099999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.626},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.70299999999999996}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":11,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66100000000000003}]}]},{"osd":5,"up_from":18,"seq":77309411344,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42984,"kb_used_data":11864,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94328856,"statfs":{"total":96636764160,"available":96592748544,"internally_reserved":0,"allocated":12148736,"data_stored":13350107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.45400000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.502},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.252},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.34599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.432}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.63},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52500000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.67400000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.58199999999999996}]}]},{"osd":9,"up_from":17,"seq":73014444048,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47732,"kb_used_data":16612,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94324108,"statfs":{"total":96636764160,"available":96587886592,"internally_reserved":0,"allocated":17010688,"data_stored":19395784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.35299999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.55200000000000005}]},{"osd":2,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.51800000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.379},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.315}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.47299999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.46300000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.60799999999999998}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.499},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.57999999999999996}]},{"osd":11,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.40899999999999997}]}]},{"osd":8,"up_from":17,"seq":73014444048,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40300,"kb_used_data":13272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331540,"statfs":{"total":96636764160,"available":96595496960,"internally_reserved":0,"allocated":13590528,"data_stored":16000403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.38500000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52500000000000002}]},{"osd":4,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.47199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.629},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.67300000000000004}]},{"osd":9,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.65700000000000003}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.69699999999999995}]},{"osd":11,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.72699999999999998}]}]},{"osd":7,"up_from":17,"seq":73014444048,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44984,"kb_used_data":13864,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94326856,"statfs":{"total":96636764160,"available":96590700544,"internally_reserved":0,"allocated":14196736,"data_stored":15992755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.501},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5620000000000003,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.51300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.45300000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58599999999999997}]},{"osd":4,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.34799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":8,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.44},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.42699999999999999}]},{"osd":9,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.46600000000000003}]},{"osd":10,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.48899999999999999}]},{"osd":11,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.55000000000000004}]}]},{"osd":4,"up_from":17,"seq":73014444047,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45976,"kb_used_data":14856,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94325864,"statfs":{"total":96636764160,"available":96589684736,"internally_reserved":0,"allocated":15212544,"data_stored":17458376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.437}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.41399999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.499},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.48899999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.379},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.32500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47699999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.71199999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.72799999999999998}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.58599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":3,"up_from":17,"seq":73014444049,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40124,"kb_used_data":13100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331716,"statfs":{"total":96636764160,"available":96595677184,"internally_reserved":0,"allocated":13414400,"data_stored":14562504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.65700000000000003}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.73299999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.71099999999999997}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":1.069},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.89400000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.0069999999999999}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.876}]},{"osd":11,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.90900000000000003}]}]},{"osd":2,"up_from":17,"seq":73014444048,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":5844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":5984256,"data_stored":6288584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.35299999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.83299999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.56899999999999995}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.49099999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":1.083},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64300000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":1.042}]},{"osd":10,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.752}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13438976,"data_stored":17371136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14962688,"data_stored":18190336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5410816,"data_stored":6529024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13328384,"data_stored":15171584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14798848,"data_stored":17018880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":11571200,"data_stored":12869632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10866688,"data_stored":13848576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12750848,"data_stored":14909440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12677120,"data_stored":15446016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16441344,"data_stored":19259392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":15527936,"data_stored":18518016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":14999552,"data_stored":19050496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:49:29.606 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T14:49:29.607 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:49:29.607 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-16T14:49:29.607 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T14:49:29.607 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T14:49:29.607 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-16T14:49:29.631 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:29.631 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:29.640 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:49:29.663 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:29.661+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:29.713 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:29.713 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T14:49:29.714 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:29.745 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:29.745 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:29.750 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:49:29.764 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:29.763+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:29.852 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:29.853 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:29.855 INFO:tasks.rados.rados.0.smithi040.stdout:317: done (3 left) 2024-09-16T14:49:29.855 INFO:tasks.rados.rados.0.smithi040.stdout:318: read oid 24 snap 20 2024-09-16T14:49:29.856 INFO:tasks.rados.rados.0.smithi040.stdout:318: expect deleted 2024-09-16T14:49:29.856 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 31 current snap is 27 2024-09-16T14:49:29.856 INFO:tasks.rados.rados.0.smithi040.stdout:319: seq_num 118 ranges {4390912=442368,4833280=753664,5586944=655360,6242304=147456} 2024-09-16T14:49:29.865 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:29.863+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:29.866 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:49:29.883 INFO:tasks.rados.rados.0.smithi040.stdout:319: writing smithi04040151-31 from 4390912 to 4833280 tid 1 2024-09-16T14:49:29.888 INFO:tasks.rados.rados.0.smithi040.stdout:319: writing smithi04040151-31 from 4833280 to 5586944 tid 2 2024-09-16T14:49:29.892 INFO:tasks.rados.rados.0.smithi040.stdout:319: writing smithi04040151-31 from 5586944 to 6242304 tid 3 2024-09-16T14:49:29.893 INFO:tasks.rados.rados.0.smithi040.stdout:319: writing smithi04040151-31 from 6242304 to 6389760 tid 4 2024-09-16T14:49:29.893 INFO:tasks.rados.rados.0.smithi040.stdout:320: snap_create 2024-09-16T14:49:29.894 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 26 v 114 (ObjNum 87 snap 14 seq_num 87) dirty exists 2024-09-16T14:49:29.894 INFO:tasks.rados.rados.0.smithi040.stdout:314: expect (ObjNum 31 snap 0 seq_num 31) 2024-09-16T14:49:29.911 INFO:tasks.rados.rados.0.smithi040.stdout:316: finishing copy_from racing read to smithi04040151-20 2024-09-16T14:49:29.912 INFO:tasks.rados.rados.0.smithi040.stdout:316: finishing copy_from to smithi04040151-20 2024-09-16T14:49:29.912 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 20 v 63 (ObjNum 117 snap 27 seq_num 117) dirty exists 2024-09-16T14:49:29.912 INFO:tasks.rados.rados.0.smithi040.stdout:319: finishing write tid 1 to smithi04040151-31 2024-09-16T14:49:29.912 INFO:tasks.rados.rados.0.smithi040.stdout:319: finishing write tid 2 to smithi04040151-31 2024-09-16T14:49:29.912 INFO:tasks.rados.rados.0.smithi040.stdout:319: finishing write tid 3 to smithi04040151-31 2024-09-16T14:49:29.912 INFO:tasks.rados.rados.0.smithi040.stdout:319: finishing write tid 4 to smithi04040151-31 2024-09-16T14:49:29.912 INFO:tasks.rados.rados.0.smithi040.stdout:319: finishing write tid 5 to smithi04040151-31 2024-09-16T14:49:29.912 INFO:tasks.rados.rados.0.smithi040.stdout:319: finishing write tid 6 to smithi04040151-31 2024-09-16T14:49:29.912 INFO:tasks.rados.rados.0.smithi040.stdout:319: oid 31 updating version 0 to 120 2024-09-16T14:49:29.912 INFO:tasks.rados.rados.0.smithi040.stdout:319: oid 31 updating version 120 to 124 2024-09-16T14:49:29.912 INFO:tasks.rados.rados.0.smithi040.stdout:319: oid 31 version 124 is already newer than 123 2024-09-16T14:49:29.912 INFO:tasks.rados.rados.0.smithi040.stdout:319: oid 31 version 124 is already newer than 122 2024-09-16T14:49:29.913 INFO:tasks.rados.rados.0.smithi040.stdout:319: oid 31 version 124 is already newer than 121 2024-09-16T14:49:29.913 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 31 v 124 (ObjNum 118 snap 27 seq_num 118) dirty exists 2024-09-16T14:49:29.913 INFO:tasks.rados.rados.0.smithi040.stdout:319: left oid 31 (ObjNum 118 snap 27 seq_num 118) 2024-09-16T14:49:29.966 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:29.964+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:29.975 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:29.975 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:29.983 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:49:30.067 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:30.065+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:30.090 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:30.090 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:30.099 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:49:30.172 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:30.170+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:30.201 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:30.201 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:30.209 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:49:30.273 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:30.272+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:30.352 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:30.352 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:30.362 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:49:30.406 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:30.405+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:30.465 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:30.465 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:30.475 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:49:30.476 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:30.474+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:30.575 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:30.574+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:30.618 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:30.618 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:30.626 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:49:30.711 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:30.710+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:30.769 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:30.769 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:30.777 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:30.775+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:30.778 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:49:30.857 INFO:tasks.rados.rados.0.smithi040.stdout:314: done (5 left) 2024-09-16T14:49:30.857 INFO:tasks.rados.rados.0.smithi040.stdout:315: done (4 left) 2024-09-16T14:49:30.857 INFO:tasks.rados.rados.0.smithi040.stdout:316: done (3 left) 2024-09-16T14:49:30.857 INFO:tasks.rados.rados.0.smithi040.stdout:318: done (2 left) 2024-09-16T14:49:30.857 INFO:tasks.rados.rados.0.smithi040.stdout:319: done (1 left) 2024-09-16T14:49:30.857 INFO:tasks.rados.rados.0.smithi040.stdout:320: done (0 left) 2024-09-16T14:49:30.858 INFO:tasks.rados.rados.0.smithi040.stdout:321: setattr oid 13 current snap is 28 2024-09-16T14:49:30.858 INFO:tasks.rados.rados.0.smithi040.stdout:322: copy_from oid 36 from oid 44 current snap is 28 2024-09-16T14:49:30.858 INFO:tasks.rados.rados.0.smithi040.stdout:323: snap_create 2024-09-16T14:49:30.863 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 13 v 102 (ObjNum 59 snap 3 seq_num 59) dirty exists 2024-09-16T14:49:30.868 INFO:tasks.rados.rados.0.smithi040.stdout:322: finishing copy_from racing read to smithi04040151-36 2024-09-16T14:49:30.871 INFO:tasks.rados.rados.0.smithi040.stdout:322: finishing copy_from to smithi04040151-36 2024-09-16T14:49:30.871 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 36 v 6 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-16T14:49:30.880 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:30.877+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:30.924 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:30.925 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:30.933 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:49:30.952 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:49:30.980 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:30.979+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:31.034 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:31.035 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:31.042 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:49:31.081 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:31.079+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:31.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:31.180+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:31.184 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:31.184 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:31.193 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:49:31.209 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:49:31.283 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:31.281+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:31.295 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:31.296 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:31.304 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:49:31.333 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:31.333 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:31.349 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:31.349 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:31.365 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:31.365 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:31.381 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:31.381 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:31.383 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:31.381+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:31.398 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:31.398 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:31.415 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:31.415 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:31.432 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:31.433 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:31.445 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:31.445 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:31.449 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:31.449 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:31.454 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:49:31.465 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:31.466 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:31.482 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:31.482 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:31.485 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:31.483+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:31.498 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:31.498 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:31.514 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:31.514 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:31.559 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:31.559 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:31.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:31.560 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:31.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.560 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:229 3.es0 3:7ecfd044:::smithi04040151-35:head [copy-from ver 4] snapc 4=[4,2] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:49:31.560 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:31.560 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.652910368999997, 2024-09-16T14:49:31.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016506461, 2024-09-16T14:49:31.560 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.560 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:31.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.561 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.561 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 229 2024-09-16T14:49:31.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.561 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:31.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:31.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904168+0000", 2024-09-16T14:49:31.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:31.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904187+0000", 2024-09-16T14:49:31.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9449999999999998e-05 2024-09-16T14:49:31.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904206+0000", 2024-09-16T14:49:31.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8698e-05 2024-09-16T14:49:31.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904215+0000", 2024-09-16T14:49:31.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:49:31.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904276+0000", 2024-09-16T14:49:31.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1130999999999995e-05 2024-09-16T14:49:31.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:31.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904284+0000", 2024-09-16T14:49:31.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0609999999999996e-06 2024-09-16T14:49:31.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906272+0000", 2024-09-16T14:49:31.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019884210000000002 2024-09-16T14:49:31.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906391+0000", 2024-09-16T14:49:31.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000118843 2024-09-16T14:49:31.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:31.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.912595+0000", 2024-09-16T14:49:31.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062038190000000002 2024-09-16T14:49:31.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:31.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917584+0000", 2024-09-16T14:49:31.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049889069999999999 2024-09-16T14:49:31.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:31.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920592+0000", 2024-09-16T14:49:31.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030081639999999998 2024-09-16T14:49:31.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920676+0000", 2024-09-16T14:49:31.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3974999999999995e-05 2024-09-16T14:49:31.568 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.568 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.569 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.569 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:230 3.es0 3:7ecfd044:::smithi04040151-35:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:49:31.569 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:49:31.569 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.652848886000001, 2024-09-16T14:49:31.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0096834569999999995, 2024-09-16T14:49:31.569 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.570 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:31.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.570 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.570 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 230 2024-09-16T14:49:31.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.570 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:49:31.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:49:31.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:49:31.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:31.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904235+0000", 2024-09-16T14:49:31.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4059999999999998e-06 2024-09-16T14:49:31.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904244+0000", 2024-09-16T14:49:31.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-16T14:49:31.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904250+0000", 2024-09-16T14:49:31.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0719999999999996e-06 2024-09-16T14:49:31.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904338+0000", 2024-09-16T14:49:31.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7923000000000005e-05 2024-09-16T14:49:31.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:31.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904344+0000", 2024-09-16T14:49:31.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8669999999999999e-06 2024-09-16T14:49:31.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906575+0000", 2024-09-16T14:49:31.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002230994 2024-09-16T14:49:31.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:31.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906632+0000", 2024-09-16T14:49:31.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7357e-05 2024-09-16T14:49:31.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913716+0000", 2024-09-16T14:49:31.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070835070000000002 2024-09-16T14:49:31.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913822+0000", 2024-09-16T14:49:31.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010566299999999999 2024-09-16T14:49:31.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.577 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.577 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913915+0000", 2024-09-16T14:49:31.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3018999999999998e-05 2024-09-16T14:49:31.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.577 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.577 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs4 37/27 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:31.578 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:49:31.578 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 47.636312074999999, 2024-09-16T14:49:31.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0098104279999999995, 2024-09-16T14:49:31.578 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.578 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:31.578 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:49:31.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:49:31.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:49:31.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:31.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920785+0000", 2024-09-16T14:49:31.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7218e-05 2024-09-16T14:49:31.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920793+0000", 2024-09-16T14:49:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9300000000000003e-06 2024-09-16T14:49:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920800+0000", 2024-09-16T14:49:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1860000000000003e-06 2024-09-16T14:49:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923692+0000", 2024-09-16T14:49:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028921329999999999 2024-09-16T14:49:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923701+0000", 2024-09-16T14:49:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6109999999999994e-06 2024-09-16T14:49:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930554+0000", 2024-09-16T14:49:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068534019999999998 2024-09-16T14:49:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930578+0000", 2024-09-16T14:49:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4232000000000001e-05 2024-09-16T14:49:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:298 3.es0 3:7ecfd044:::smithi04040151-35:head [append 0~720896] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.628246554, 2024-09-16T14:49:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014187808, 2024-09-16T14:49:31.589 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.589 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:31.589 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.590 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.590 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.590 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 298 2024-09-16T14:49:31.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.590 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:31.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:31.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928832+0000", 2024-09-16T14:49:31.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:31.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929746+0000", 2024-09-16T14:49:31.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00091434199999999995 2024-09-16T14:49:31.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929760+0000", 2024-09-16T14:49:31.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3519e-05 2024-09-16T14:49:31.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929767+0000", 2024-09-16T14:49:31.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3180000000000001e-06 2024-09-16T14:49:31.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929849+0000", 2024-09-16T14:49:31.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2020000000000004e-05 2024-09-16T14:49:31.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:31.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929854+0000", 2024-09-16T14:49:31.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1209999999999998e-06 2024-09-16T14:49:31.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931714+0000", 2024-09-16T14:49:31.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001859989 2024-09-16T14:49:31.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931845+0000", 2024-09-16T14:49:31.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130901 2024-09-16T14:49:31.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:31.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933089+0000", 2024-09-16T14:49:31.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001243535 2024-09-16T14:49:31.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:31.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936370+0000", 2024-09-16T14:49:31.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032817660000000002 2024-09-16T14:49:31.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:31.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942945+0000", 2024-09-16T14:49:31.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065741560000000003 2024-09-16T14:49:31.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943021+0000", 2024-09-16T14:49:31.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6734000000000001e-05 2024-09-16T14:49:31.601 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.601 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.601 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.602 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:299 3.es0 3:7ecfd044:::smithi04040151-35:head [append 0~376832] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:31.602 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:31.602 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.625956754000001, 2024-09-16T14:49:31.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012149627, 2024-09-16T14:49:31.602 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.602 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:31.602 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.602 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.602 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.602 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 299 2024-09-16T14:49:31.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.602 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:31.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:31.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931122+0000", 2024-09-16T14:49:31.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:31.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931722+0000", 2024-09-16T14:49:31.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059944100000000004 2024-09-16T14:49:31.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931732+0000", 2024-09-16T14:49:31.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9369999999999999e-06 2024-09-16T14:49:31.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931739+0000", 2024-09-16T14:49:31.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2819999999999999e-06 2024-09-16T14:49:31.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933835+0000", 2024-09-16T14:49:31.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020961140000000001 2024-09-16T14:49:31.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933929+0000", 2024-09-16T14:49:31.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3683000000000004e-05 2024-09-16T14:49:31.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:31.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.934917+0000", 2024-09-16T14:49:31.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00098803200000000002 2024-09-16T14:49:31.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:31.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.939558+0000", 2024-09-16T14:49:31.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00464175 2024-09-16T14:49:31.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:31.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943205+0000", 2024-09-16T14:49:31.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036465820000000002 2024-09-16T14:49:31.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943273+0000", 2024-09-16T14:49:31.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7917999999999996e-05 2024-09-16T14:49:31.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.611 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.611 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.611 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:300 3.es0 3:7ecfd044:::smithi04040151-35:head [setxattr _header (56) in=7b] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:31.611 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:31.611 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.625330660000003, 2024-09-16T14:49:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011740911, 2024-09-16T14:49:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 300 2024-09-16T14:49:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:31.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:31.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:31.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:31.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931753+0000", 2024-09-16T14:49:31.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0380000000000001e-06 2024-09-16T14:49:31.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931759+0000", 2024-09-16T14:49:31.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3430000000000001e-06 2024-09-16T14:49:31.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931765+0000", 2024-09-16T14:49:31.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-16T14:49:31.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.935623+0000", 2024-09-16T14:49:31.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038583049999999998 2024-09-16T14:49:31.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.935717+0000", 2024-09-16T14:49:31.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4587999999999996e-05 2024-09-16T14:49:31.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:31.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936150+0000", 2024-09-16T14:49:31.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043270599999999999 2024-09-16T14:49:31.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:31.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.939576+0000", 2024-09-16T14:49:31.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034257620000000002 2024-09-16T14:49:31.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:31.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943399+0000", 2024-09-16T14:49:31.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003823145 2024-09-16T14:49:31.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943490+0000", 2024-09-16T14:49:31.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1241000000000003e-05 2024-09-16T14:49:31.617 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.617 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.617 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.617 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:301 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:31.617 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:31.617 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.625309360000003, 2024-09-16T14:49:31.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014215955000000001, 2024-09-16T14:49:31.617 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.617 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:31.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.618 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 301 2024-09-16T14:49:31.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.618 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:31.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:31.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931770+0000", 2024-09-16T14:49:31.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:31.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931775+0000", 2024-09-16T14:49:31.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4530000000000004e-06 2024-09-16T14:49:31.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931782+0000", 2024-09-16T14:49:31.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0550000000000002e-06 2024-09-16T14:49:31.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931787+0000", 2024-09-16T14:49:31.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2310000000000004e-06 2024-09-16T14:49:31.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936401+0000", 2024-09-16T14:49:31.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046143419999999996 2024-09-16T14:49:31.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:31.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936482+0000", 2024-09-16T14:49:31.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0407000000000005e-05 2024-09-16T14:49:31.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943851+0000", 2024-09-16T14:49:31.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073695000000000002 2024-09-16T14:49:31.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943951+0000", 2024-09-16T14:49:31.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9327000000000006e-05 2024-09-16T14:49:31.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.945987+0000", 2024-09-16T14:49:31.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002036057 2024-09-16T14:49:31.622 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.622 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.622 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.622 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:369 3.8s0 3:1f96bdfc:::smithi04040151-37:head [append 0~475136] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:31.622 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:31.623 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.599367366999999, 2024-09-16T14:49:31.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010979078999999999, 2024-09-16T14:49:31.623 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.623 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:31.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.623 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.623 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 369 2024-09-16T14:49:31.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.623 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:31.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:31.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957711+0000", 2024-09-16T14:49:31.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:31.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958323+0000", 2024-09-16T14:49:31.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000612439 2024-09-16T14:49:31.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958339+0000", 2024-09-16T14:49:31.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5985999999999999e-05 2024-09-16T14:49:31.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958348+0000", 2024-09-16T14:49:31.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8699999999999998e-06 2024-09-16T14:49:31.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958393+0000", 2024-09-16T14:49:31.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4981000000000002e-05 2024-09-16T14:49:31.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958478+0000", 2024-09-16T14:49:31.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4845999999999994e-05 2024-09-16T14:49:31.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:31.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.959659+0000", 2024-09-16T14:49:31.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011808999999999999 2024-09-16T14:49:31.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:31.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962687+0000", 2024-09-16T14:49:31.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030278330000000002 2024-09-16T14:49:31.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:31.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968651+0000", 2024-09-16T14:49:31.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005964764 2024-09-16T14:49:31.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968692+0000", 2024-09-16T14:49:31.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0374999999999999e-05 2024-09-16T14:49:31.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.628 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.628 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:372 3.8s0 3:1f96bdfc:::smithi04040151-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:31.628 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:49:31.628 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.596639031000002, 2024-09-16T14:49:31.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010116534999999999, 2024-09-16T14:49:31.629 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.629 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:31.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.629 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 372 2024-09-16T14:49:31.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.629 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:49:31.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:49:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:49:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960445+0000", 2024-09-16T14:49:31.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0579999999999999e-06 2024-09-16T14:49:31.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960452+0000", 2024-09-16T14:49:31.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5360000000000004e-06 2024-09-16T14:49:31.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960458+0000", 2024-09-16T14:49:31.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3260000000000003e-06 2024-09-16T14:49:31.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962902+0000", 2024-09-16T14:49:31.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002444313 2024-09-16T14:49:31.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:31.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962973+0000", 2024-09-16T14:49:31.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1179000000000004e-05 2024-09-16T14:49:31.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.969538+0000", 2024-09-16T14:49:31.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006565169 2024-09-16T14:49:31.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.969635+0000", 2024-09-16T14:49:31.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6427000000000004e-05 2024-09-16T14:49:31.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.970558+0000", 2024-09-16T14:49:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00092299600000000004 2024-09-16T14:49:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:387 3.8s0 3:1f96bdfc:::smithi04040151-37:head [read 0~2686976,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:31.634 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:49:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.692617007000003, 2024-09-16T14:49:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010154198999999999, 2024-09-16T14:49:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 387 2024-09-16T14:49:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.635 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:49:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:49:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:49:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864465+0000", 2024-09-16T14:49:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.774e-06 2024-09-16T14:49:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864472+0000", 2024-09-16T14:49:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0040000000000002e-06 2024-09-16T14:49:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864477+0000", 2024-09-16T14:49:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0779999999999997e-06 2024-09-16T14:49:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.866679+0000", 2024-09-16T14:49:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00220228 2024-09-16T14:49:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.866770+0000", 2024-09-16T14:49:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0952999999999995e-05 2024-09-16T14:49:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874617+0000", 2024-09-16T14:49:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007847527 2024-09-16T14:49:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:398 3.as0 3:572aff57:::smithi04040151-20:head [append 0~507904] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:31.640 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.678002601000003, 2024-09-16T14:49:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010447988, 2024-09-16T14:49:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 398 2024-09-16T14:49:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879076+0000", 2024-09-16T14:49:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881851+0000", 2024-09-16T14:49:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0027753420000000001 2024-09-16T14:49:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881863+0000", 2024-09-16T14:49:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2203e-05 2024-09-16T14:49:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881871+0000", 2024-09-16T14:49:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4059999999999999e-06 2024-09-16T14:49:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881924+0000", 2024-09-16T14:49:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2932000000000001e-05 2024-09-16T14:49:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882118+0000", 2024-09-16T14:49:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019413500000000001 2024-09-16T14:49:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884114+0000", 2024-09-16T14:49:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001996154 2024-09-16T14:49:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889295+0000", 2024-09-16T14:49:31.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051809899999999999 2024-09-16T14:49:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889480+0000", 2024-09-16T14:49:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018519700000000001 2024-09-16T14:49:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889525+0000", 2024-09-16T14:49:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5399999999999999e-05 2024-09-16T14:49:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:400 3.as0 3:572aff57:::smithi04040151-20:head [append 0~262144] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.673493571000002, 2024-09-16T14:49:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010619327, 2024-09-16T14:49:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 400 2024-09-16T14:49:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.647 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883585+0000", 2024-09-16T14:49:31.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883868+0000", 2024-09-16T14:49:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028385599999999997 2024-09-16T14:49:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883880+0000", 2024-09-16T14:49:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1181e-05 2024-09-16T14:49:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883888+0000", 2024-09-16T14:49:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.926e-06 2024-09-16T14:49:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887739+0000", 2024-09-16T14:49:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003851658 2024-09-16T14:49:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887849+0000", 2024-09-16T14:49:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109616 2024-09-16T14:49:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.888684+0000", 2024-09-16T14:49:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083518000000000004 2024-09-16T14:49:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894010+0000", 2024-09-16T14:49:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053260670000000003 2024-09-16T14:49:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894143+0000", 2024-09-16T14:49:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000133077 2024-09-16T14:49:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894206+0000", 2024-09-16T14:49:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2658999999999998e-05 2024-09-16T14:49:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:401 3.as0 3:572aff57:::smithi04040151-20:head [setxattr _header (57) in=7b] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.673183647000002, 2024-09-16T14:49:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010524774000000001, 2024-09-16T14:49:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.653 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 401 2024-09-16T14:49:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883900+0000", 2024-09-16T14:49:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1400000000000002e-06 2024-09-16T14:49:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883908+0000", 2024-09-16T14:49:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2689999999999999e-06 2024-09-16T14:49:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883913+0000", 2024-09-16T14:49:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-16T14:49:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889560+0000", 2024-09-16T14:49:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005647165 2024-09-16T14:49:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889627+0000", 2024-09-16T14:49:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6951000000000003e-05 2024-09-16T14:49:31.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889898+0000", 2024-09-16T14:49:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270709 2024-09-16T14:49:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894012+0000", 2024-09-16T14:49:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041137810000000004 2024-09-16T14:49:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:31.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894344+0000", 2024-09-16T14:49:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000332121 2024-09-16T14:49:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894421+0000", 2024-09-16T14:49:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7406999999999999e-05 2024-09-16T14:49:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:402 3.as0 3:572aff57:::smithi04040151-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:31.658 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 37.673161770999997, 2024-09-16T14:49:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011864743000000001, 2024-09-16T14:49:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 402 2024-09-16T14:49:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883921+0000", 2024-09-16T14:49:31.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7520000000000001e-06 2024-09-16T14:49:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883927+0000", 2024-09-16T14:49:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4679999999999997e-06 2024-09-16T14:49:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883932+0000", 2024-09-16T14:49:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1440000000000002e-06 2024-09-16T14:49:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890477+0000", 2024-09-16T14:49:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065446430000000002 2024-09-16T14:49:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890586+0000", 2024-09-16T14:49:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010909 2024-09-16T14:49:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894750+0000", 2024-09-16T14:49:31.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00416359 2024-09-16T14:49:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894834+0000", 2024-09-16T14:49:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4596000000000001e-05 2024-09-16T14:49:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895783+0000", 2024-09-16T14:49:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00094892000000000002 2024-09-16T14:49:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:443 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.680951198000002, 2024-09-16T14:49:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.013766494000000001, 2024-09-16T14:49:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:31.664 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 443 2024-09-16T14:49:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876127+0000", 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876138+0000", 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1344e-05 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876159+0000", 2024-09-16T14:49:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1157000000000001e-05 2024-09-16T14:49:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876170+0000", 2024-09-16T14:49:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0651e-05 2024-09-16T14:49:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876234+0000", 2024-09-16T14:49:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4114000000000003e-05 2024-09-16T14:49:31.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876243+0000", 2024-09-16T14:49:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0049999999999993e-06 2024-09-16T14:49:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879445+0000", 2024-09-16T14:49:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032019269999999998 2024-09-16T14:49:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879550+0000", 2024-09-16T14:49:31.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010536500000000001 2024-09-16T14:49:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889895+0000", 2024-09-16T14:49:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010345043 2024-09-16T14:49:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:444 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.680861491000002, 2024-09-16T14:49:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014716241, 2024-09-16T14:49:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.670 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 444 2024-09-16T14:49:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876218+0000", 2024-09-16T14:49:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876224+0000", 2024-09-16T14:49:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:49:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876235+0000", 2024-09-16T14:49:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.025e-05 2024-09-16T14:49:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876242+0000", 2024-09-16T14:49:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8790000000000003e-06 2024-09-16T14:49:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876315+0000", 2024-09-16T14:49:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2260000000000003e-05 2024-09-16T14:49:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876320+0000", 2024-09-16T14:49:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5690000000000004e-06 2024-09-16T14:49:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880078+0000", 2024-09-16T14:49:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0037576430000000002 2024-09-16T14:49:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880168+0000", 2024-09-16T14:49:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0353999999999996e-05 2024-09-16T14:49:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.890935+0000", 2024-09-16T14:49:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010766556 2024-09-16T14:49:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:445 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:31.677 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.680817509000001, 2024-09-16T14:49:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016529716999999999, 2024-09-16T14:49:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 445 2024-09-16T14:49:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876262+0000", 2024-09-16T14:49:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876269+0000", 2024-09-16T14:49:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9469999999999997e-06 2024-09-16T14:49:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876276+0000", 2024-09-16T14:49:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5950000000000004e-06 2024-09-16T14:49:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876281+0000", 2024-09-16T14:49:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:49:31.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876343+0000", 2024-09-16T14:49:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1685000000000003e-05 2024-09-16T14:49:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876348+0000", 2024-09-16T14:49:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7269999999999998e-06 2024-09-16T14:49:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881091+0000", 2024-09-16T14:49:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047434599999999997 2024-09-16T14:49:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881181+0000", 2024-09-16T14:49:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9838000000000003e-05 2024-09-16T14:49:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.892792+0000", 2024-09-16T14:49:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011611316 2024-09-16T14:49:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s1 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.624007258999999, 2024-09-16T14:49:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011783765999999999, 2024-09-16T14:49:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.683 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933072+0000", 2024-09-16T14:49:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933092+0000", 2024-09-16T14:49:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0000000000000002e-05 2024-09-16T14:49:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933119+0000", 2024-09-16T14:49:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6877e-05 2024-09-16T14:49:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933127+0000", 2024-09-16T14:49:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4600000000000003e-06 2024-09-16T14:49:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.937087+0000", 2024-09-16T14:49:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0039599129999999998 2024-09-16T14:49:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.937096+0000", 2024-09-16T14:49:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8910000000000001e-06 2024-09-16T14:49:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944834+0000", 2024-09-16T14:49:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077376809999999997 2024-09-16T14:49:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944857+0000", 2024-09-16T14:49:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2785999999999998e-05 2024-09-16T14:49:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.687 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:591 3.0s0 3:060eef9f:::smithi04040151-16:head [copy-from ver 27] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.581208251, 2024-09-16T14:49:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0279670919999999, 2024-09-16T14:49:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 591 2024-09-16T14:49:31.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975870+0000", 2024-09-16T14:49:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975878+0000", 2024-09-16T14:49:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2619999999999998e-06 2024-09-16T14:49:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975902+0000", 2024-09-16T14:49:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4036000000000002e-05 2024-09-16T14:49:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975912+0000", 2024-09-16T14:49:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-16T14:49:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975986+0000", 2024-09-16T14:49:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3548999999999997e-05 2024-09-16T14:49:31.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975990+0000", 2024-09-16T14:49:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7670000000000001e-06 2024-09-16T14:49:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.975658+0000", 2024-09-16T14:49:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99966882800000001 2024-09-16T14:49:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for missing object", 2024-09-16T14:49:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.975885+0000", 2024-09-16T14:49:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000226682 2024-09-16T14:49:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.982217+0000", 2024-09-16T14:49:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063321530000000001 2024-09-16T14:49:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.982350+0000", 2024-09-16T14:49:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013316700000000001 2024-09-16T14:49:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.999798+0000", 2024-09-16T14:49:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017447608 2024-09-16T14:49:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001884+0000", 2024-09-16T14:49:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020859889999999999 2024-09-16T14:49:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.003629+0000", 2024-09-16T14:49:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001745106 2024-09-16T14:49:31.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.003839+0000", 2024-09-16T14:49:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020980099999999999 2024-09-16T14:49:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:592 3.0s0 3:060eef9f:::smithi04040151-16:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.581151327000001, 2024-09-16T14:49:31.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0256284010000001, 2024-09-16T14:49:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 592 2024-09-16T14:49:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975928+0000", 2024-09-16T14:49:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975936+0000", 2024-09-16T14:49:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6090000000000003e-06 2024-09-16T14:49:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975943+0000", 2024-09-16T14:49:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:31.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975949+0000", 2024-09-16T14:49:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8300000000000001e-06 2024-09-16T14:49:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.976030+0000", 2024-09-16T14:49:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1267000000000006e-05 2024-09-16T14:49:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.976033+0000", 2024-09-16T14:49:31.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7590000000000002e-06 2024-09-16T14:49:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.977041+0000", 2024-09-16T14:49:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0010081639999999 2024-09-16T14:49:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for missing object", 2024-09-16T14:49:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.977086+0000", 2024-09-16T14:49:31.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4879999999999997e-05 2024-09-16T14:49:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.983015+0000", 2024-09-16T14:49:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059287480000000002 2024-09-16T14:49:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.983090+0000", 2024-09-16T14:49:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5081999999999997e-05 2024-09-16T14:49:31.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001193+0000", 2024-09-16T14:49:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.018102713999999999 2024-09-16T14:49:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001306+0000", 2024-09-16T14:49:31.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011330799999999999 2024-09-16T14:49:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001557+0000", 2024-09-16T14:49:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025100900000000001 2024-09-16T14:49:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:31.703 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:31.703 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:31.705 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:49:31.706 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:31.704+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:31.807 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:31.805+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:31.820 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:31.820 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:31.821 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:31.821 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:31.829 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:49:31.858 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is set 2024-09-16T14:49:31.859 INFO:tasks.rados.rados.0.smithi040.stdout:321: done (2 left) 2024-09-16T14:49:31.859 INFO:tasks.rados.rados.0.smithi040.stdout:322: done (1 left) 2024-09-16T14:49:31.859 INFO:tasks.rados.rados.0.smithi040.stdout:323: done (0 left) 2024-09-16T14:49:31.859 INFO:tasks.rados.rados.0.smithi040.stdout:324: snap_create 2024-09-16T14:49:31.908 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:31.907+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:31.932 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:31.932 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:31.932 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:31.932 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:31.932 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:31.941 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:49:32.009 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:32.008+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:32.045 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:32.045 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:32.046 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:32.046 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:32.046 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.046 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s6 66/65 ECSubWrite(tid=69, reqid=client.4500.0:588, at_version=66'56, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:32.046 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.992663+0000", 2024-09-16T14:49:32.046 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 19.050495414, 2024-09-16T14:49:32.046 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032913719999999999, 2024-09-16T14:49:32.046 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.046 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.046 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.046 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.046 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.046 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992663+0000", 2024-09-16T14:49:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992663+0000", 2024-09-16T14:49:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992662+0000", 2024-09-16T14:49:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.047 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992691+0000", 2024-09-16T14:49:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8535e-05 2024-09-16T14:49:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992702+0000", 2024-09-16T14:49:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1590000000000001e-05 2024-09-16T14:49:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992711+0000", 2024-09-16T14:49:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5310000000000001e-06 2024-09-16T14:49:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.048 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992772+0000", 2024-09-16T14:49:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.1235000000000006e-05 2024-09-16T14:49:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992783+0000", 2024-09-16T14:49:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1413999999999999e-05 2024-09-16T14:49:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995930+0000", 2024-09-16T14:49:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031467000000000001 2024-09-16T14:49:32.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.995954+0000", 2024-09-16T14:49:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4243000000000002e-05 2024-09-16T14:49:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds3 66/65 ECSubWrite(tid=256, reqid=client.4500.0:601, at_version=66'53, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.993987+0000", 2024-09-16T14:49:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 19.049171025, 2024-09-16T14:49:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036421859999999999, 2024-09-16T14:49:32.050 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.051 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.051 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.051 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.051 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.051 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.993987+0000", 2024-09-16T14:49:32.051 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.051 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.051 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.051 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.051 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.993987+0000", 2024-09-16T14:49:32.051 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.051 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.051 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.051 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.052 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.993987+0000", 2024-09-16T14:49:32.052 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.052 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.052 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.052 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994130+0000", 2024-09-16T14:49:32.052 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014385799999999999 2024-09-16T14:49:32.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.052 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.052 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.052 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994146+0000", 2024-09-16T14:49:32.052 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.588e-05 2024-09-16T14:49:32.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.052 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.053 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.053 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994166+0000", 2024-09-16T14:49:32.053 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9582999999999999e-05 2024-09-16T14:49:32.053 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.053 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.053 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.053 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994229+0000", 2024-09-16T14:49:32.053 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3495000000000001e-05 2024-09-16T14:49:32.053 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.053 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.053 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.053 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994239+0000", 2024-09-16T14:49:32.053 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:49:32.053 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.054 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.054 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.054 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.997601+0000", 2024-09-16T14:49:32.054 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0033624900000000001 2024-09-16T14:49:32.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.054 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.054 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.054 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.997630+0000", 2024-09-16T14:49:32.054 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8530999999999998e-05 2024-09-16T14:49:32.054 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.054 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.054 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.055 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.055 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s6 66/65 ECSubWrite(tid=72, reqid=client.4500.0:590, at_version=0'0, trim_to=0'0, roll_forward_to=66'58))", 2024-09-16T14:49:32.055 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.005902+0000", 2024-09-16T14:49:32.055 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 19.037256405000001, 2024-09-16T14:49:32.055 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032803229999999999, 2024-09-16T14:49:32.055 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.055 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.055 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.055 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.055 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.055 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005902+0000", 2024-09-16T14:49:32.055 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.055 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.056 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.056 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005902+0000", 2024-09-16T14:49:32.056 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.056 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.056 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.056 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005901+0000", 2024-09-16T14:49:32.056 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.056 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.056 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.056 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005961+0000", 2024-09-16T14:49:32.056 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.0185000000000001e-05 2024-09-16T14:49:32.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.057 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.057 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.057 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005976+0000", 2024-09-16T14:49:32.057 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4489e-05 2024-09-16T14:49:32.057 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.057 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.057 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.057 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005986+0000", 2024-09-16T14:49:32.057 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9119999999999993e-06 2024-09-16T14:49:32.057 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.057 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.057 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.057 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006031+0000", 2024-09-16T14:49:32.058 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5556000000000002e-05 2024-09-16T14:49:32.058 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.058 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.058 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.058 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006041+0000", 2024-09-16T14:49:32.058 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0161e-05 2024-09-16T14:49:32.058 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.058 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.058 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.058 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009154+0000", 2024-09-16T14:49:32.058 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031131010000000001 2024-09-16T14:49:32.058 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.058 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.058 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.059 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009182+0000", 2024-09-16T14:49:32.059 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7813999999999998e-05 2024-09-16T14:49:32.059 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.059 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.059 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.059 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.059 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 66/65 ECSubWrite(tid=38, reqid=client.4500.0:596, at_version=66'25, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:32.059 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.009720+0000", 2024-09-16T14:49:32.059 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 19.033438416999999, 2024-09-16T14:49:32.059 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053370680000000004, 2024-09-16T14:49:32.059 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.059 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.059 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.060 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.060 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.060 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009720+0000", 2024-09-16T14:49:32.060 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.060 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.060 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.060 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009720+0000", 2024-09-16T14:49:32.060 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.060 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.060 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.060 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009719+0000", 2024-09-16T14:49:32.061 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.061 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.061 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.061 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.061 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009878+0000", 2024-09-16T14:49:32.061 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015914100000000001 2024-09-16T14:49:32.061 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.061 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.061 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.061 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009902+0000", 2024-09-16T14:49:32.061 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3164999999999999e-05 2024-09-16T14:49:32.061 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.061 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.061 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009911+0000", 2024-09-16T14:49:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9429999999999995e-06 2024-09-16T14:49:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009974+0000", 2024-09-16T14:49:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2117000000000002e-05 2024-09-16T14:49:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009985+0000", 2024-09-16T14:49:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1539000000000001e-05 2024-09-16T14:49:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.062 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.063 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.063 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.015029+0000", 2024-09-16T14:49:32.063 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050442999999999998 2024-09-16T14:49:32.063 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.063 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.063 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.063 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.015057+0000", 2024-09-16T14:49:32.063 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7606999999999998e-05 2024-09-16T14:49:32.063 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.063 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.063 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.063 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.063 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.063 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 66/65 ECSubWrite(tid=39, reqid=client.4500.0:597, at_version=66'26, trim_to=0'0, roll_forward_to=66'23))", 2024-09-16T14:49:32.064 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:32.064 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 19.030769111000001, 2024-09-16T14:49:32.064 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0060634030000000002, 2024-09-16T14:49:32.064 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.064 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.064 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.064 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.064 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.064 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:32.064 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.064 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.064 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.065 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:32.065 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.065 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.065 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.065 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012388+0000", 2024-09-16T14:49:32.065 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.065 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.065 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.065 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012433+0000", 2024-09-16T14:49:32.065 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4588000000000001e-05 2024-09-16T14:49:32.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.066 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.066 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.066 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012604+0000", 2024-09-16T14:49:32.066 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017141300000000001 2024-09-16T14:49:32.066 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.066 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.066 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.066 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012617+0000", 2024-09-16T14:49:32.066 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2792999999999999e-05 2024-09-16T14:49:32.066 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.066 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.066 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.066 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012666+0000", 2024-09-16T14:49:32.066 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9079000000000002e-05 2024-09-16T14:49:32.067 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.067 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.067 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.067 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012678+0000", 2024-09-16T14:49:32.067 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1405e-05 2024-09-16T14:49:32.067 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.067 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.067 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.067 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018433+0000", 2024-09-16T14:49:32.067 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005755146 2024-09-16T14:49:32.067 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.067 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.067 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.067 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018453+0000", 2024-09-16T14:49:32.068 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9780999999999999e-05 2024-09-16T14:49:32.068 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.068 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.068 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.068 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.068 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 66/65 ECSubWrite(tid=40, reqid=client.4500.0:598, at_version=66'27, trim_to=0'0, roll_forward_to=66'23))", 2024-09-16T14:49:32.068 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.013745+0000", 2024-09-16T14:49:32.068 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 19.029413481999999, 2024-09-16T14:49:32.068 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0047198850000000001, 2024-09-16T14:49:32.068 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.068 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.068 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.069 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.069 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.069 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013745+0000", 2024-09-16T14:49:32.069 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.069 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.069 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.069 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013745+0000", 2024-09-16T14:49:32.069 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.069 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.069 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.069 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013744+0000", 2024-09-16T14:49:32.069 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.070 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.070 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.070 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.070 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013766+0000", 2024-09-16T14:49:32.070 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1721000000000001e-05 2024-09-16T14:49:32.070 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.070 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.070 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.070 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013788+0000", 2024-09-16T14:49:32.070 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1708e-05 2024-09-16T14:49:32.070 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.070 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.070 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.070 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013798+0000", 2024-09-16T14:49:32.071 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.026e-05 2024-09-16T14:49:32.071 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.071 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.071 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.071 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013847+0000", 2024-09-16T14:49:32.071 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9579000000000001e-05 2024-09-16T14:49:32.071 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.071 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.071 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.071 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013857+0000", 2024-09-16T14:49:32.071 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9939999999999995e-06 2024-09-16T14:49:32.071 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.071 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.072 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.072 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018456+0000", 2024-09-16T14:49:32.072 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.004598141 2024-09-16T14:49:32.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.072 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.072 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.072 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018465+0000", 2024-09-16T14:49:32.072 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2439999999999999e-06 2024-09-16T14:49:32.072 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.072 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.072 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.072 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.073 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:702 3.as0 3:572aff57:::smithi04040151-20:head [copy-from ver 4] snapc 2f=[2f,2e,29,28,25] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:32.073 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:32.073 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 8.1430808290000005, 2024-09-16T14:49:32.073 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.032342711000000003, 2024-09-16T14:49:32.073 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.073 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:32.073 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:32.073 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:32.073 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:32.073 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 702 2024-09-16T14:49:32.073 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.073 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.073 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.073 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.074 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:32.074 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.074 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.074 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.074 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.074 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:32.074 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.074 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.074 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.074 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.074 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900076+0000", 2024-09-16T14:49:32.074 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:32.074 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.075 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.075 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.075 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900082+0000", 2024-09-16T14:49:32.075 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3640000000000004e-06 2024-09-16T14:49:32.075 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.075 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.075 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.075 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900094+0000", 2024-09-16T14:49:32.075 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1208e-05 2024-09-16T14:49:32.075 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.075 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.075 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.075 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900105+0000", 2024-09-16T14:49:32.075 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1782e-05 2024-09-16T14:49:32.076 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.076 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.076 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.076 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900161+0000", 2024-09-16T14:49:32.076 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5244999999999997e-05 2024-09-16T14:49:32.076 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.076 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.076 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:32.076 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900170+0000", 2024-09-16T14:49:32.076 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4409999999999999e-06 2024-09-16T14:49:32.076 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.076 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.077 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.077 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.901930+0000", 2024-09-16T14:49:32.077 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001760246 2024-09-16T14:49:32.077 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.077 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.077 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:32.077 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902046+0000", 2024-09-16T14:49:32.077 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000115882 2024-09-16T14:49:32.077 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.077 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.077 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.077 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.917553+0000", 2024-09-16T14:49:32.077 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015506967999999999 2024-09-16T14:49:32.078 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.078 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.078 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.078 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918732+0000", 2024-09-16T14:49:32.078 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001178733 2024-09-16T14:49:32.078 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.078 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.078 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:32.078 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932379+0000", 2024-09-16T14:49:32.078 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013647276999999999 2024-09-16T14:49:32.078 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.078 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.078 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.078 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932420+0000", 2024-09-16T14:49:32.079 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1066999999999998e-05 2024-09-16T14:49:32.079 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.079 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.079 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.079 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.079 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.079 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:703 3.as0 3:572aff57:::smithi04040151-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:32.079 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:32.079 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 8.1430394310000001, 2024-09-16T14:49:32.079 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.018566787000000001, 2024-09-16T14:49:32.079 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.079 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:32.079 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:32.080 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:32.080 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:32.080 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 703 2024-09-16T14:49:32.080 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.080 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.080 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.080 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.080 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:32.080 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.080 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.080 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.080 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.080 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:32.081 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.081 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.081 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.081 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900118+0000", 2024-09-16T14:49:32.081 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.081 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.081 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.081 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900122+0000", 2024-09-16T14:49:32.081 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5669999999999999e-06 2024-09-16T14:49:32.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.081 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.081 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.082 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900129+0000", 2024-09-16T14:49:32.082 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0360000000000001e-06 2024-09-16T14:49:32.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.082 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.082 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.082 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900134+0000", 2024-09-16T14:49:32.082 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.916e-06 2024-09-16T14:49:32.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.082 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.082 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.082 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900217+0000", 2024-09-16T14:49:32.082 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3189000000000006e-05 2024-09-16T14:49:32.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.082 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.083 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:32.083 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900222+0000", 2024-09-16T14:49:32.083 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7430000000000002e-06 2024-09-16T14:49:32.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.083 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.083 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.083 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902209+0000", 2024-09-16T14:49:32.083 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0019871440000000001 2024-09-16T14:49:32.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.083 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.083 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:32.083 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902254+0000", 2024-09-16T14:49:32.083 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4969000000000003e-05 2024-09-16T14:49:32.084 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.084 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.084 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.084 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918411+0000", 2024-09-16T14:49:32.084 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.016157207999999999 2024-09-16T14:49:32.084 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.084 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.084 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:32.084 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918534+0000", 2024-09-16T14:49:32.085 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000123146 2024-09-16T14:49:32.085 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.085 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.085 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.085 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918686+0000", 2024-09-16T14:49:32.085 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015134799999999999 2024-09-16T14:49:32.085 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.085 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.085 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.085 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.085 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.085 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s2 77/65 ECSubWrite(tid=312, reqid=client.4500.0:711, at_version=77'98, trim_to=0'0, roll_forward_to=73'95))", 2024-09-16T14:49:32.085 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.915065+0000", 2024-09-16T14:49:32.085 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 8.1280929700000009, 2024-09-16T14:49:32.086 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003580101, 2024-09-16T14:49:32.086 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.086 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.086 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.086 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.086 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.086 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915065+0000", 2024-09-16T14:49:32.086 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.086 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.086 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.086 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.086 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915065+0000", 2024-09-16T14:49:32.086 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.087 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.087 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.087 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915065+0000", 2024-09-16T14:49:32.087 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.087 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.087 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.087 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915202+0000", 2024-09-16T14:49:32.087 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00013698000000000001 2024-09-16T14:49:32.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.087 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.087 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.087 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915211+0000", 2024-09-16T14:49:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1940000000000004e-06 2024-09-16T14:49:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915218+0000", 2024-09-16T14:49:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2230000000000001e-06 2024-09-16T14:49:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915387+0000", 2024-09-16T14:49:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00016841100000000001 2024-09-16T14:49:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.088 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.089 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915395+0000", 2024-09-16T14:49:32.089 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4530000000000002e-06 2024-09-16T14:49:32.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.089 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.089 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.089 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918628+0000", 2024-09-16T14:49:32.089 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032327760000000001 2024-09-16T14:49:32.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.089 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.089 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.089 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918646+0000", 2024-09-16T14:49:32.089 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7750000000000001e-05 2024-09-16T14:49:32.089 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.090 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.090 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.090 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.090 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.090 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 80/65 ECSubWrite(tid=46, reqid=client.4500.0:745, at_version=80'31, trim_to=0'0, roll_forward_to=66'27))", 2024-09-16T14:49:32.090 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:26.992801+0000", 2024-09-16T14:49:32.090 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.0503571049999998, 2024-09-16T14:49:32.090 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045201599999999996, 2024-09-16T14:49:32.090 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.090 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.090 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.090 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.090 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.090 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992801+0000", 2024-09-16T14:49:32.091 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.091 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.091 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.091 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.091 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992801+0000", 2024-09-16T14:49:32.091 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.091 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.091 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.091 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.091 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992801+0000", 2024-09-16T14:49:32.091 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.091 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.091 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.092 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.092 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992818+0000", 2024-09-16T14:49:32.092 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7876e-05 2024-09-16T14:49:32.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.092 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.092 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.092 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992829+0000", 2024-09-16T14:49:32.092 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0756e-05 2024-09-16T14:49:32.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.092 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.092 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.092 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992838+0000", 2024-09-16T14:49:32.092 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8969999999999997e-06 2024-09-16T14:49:32.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.093 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.093 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.093 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992889+0000", 2024-09-16T14:49:32.093 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.1236000000000002e-05 2024-09-16T14:49:32.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.093 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.097 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.097 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992900+0000", 2024-09-16T14:49:32.097 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0183999999999999e-05 2024-09-16T14:49:32.097 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.098 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.098 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.098 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.997301+0000", 2024-09-16T14:49:32.098 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044018879999999996 2024-09-16T14:49:32.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.098 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.098 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.098 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.997321+0000", 2024-09-16T14:49:32.098 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0026000000000001e-05 2024-09-16T14:49:32.098 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.098 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.098 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.098 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.099 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.cs2 82/65 ECSubWrite(tid=158, reqid=osd.5.0:158, at_version=82'58, trim_to=0'0, roll_forward_to=79'56))", 2024-09-16T14:49:32.099 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.928127+0000", 2024-09-16T14:49:32.099 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.1150311830000001, 2024-09-16T14:49:32.099 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032496740000000001, 2024-09-16T14:49:32.099 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.099 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.099 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.099 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.099 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.099 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928127+0000", 2024-09-16T14:49:32.099 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.099 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.100 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.100 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928127+0000", 2024-09-16T14:49:32.100 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.100 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.100 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.100 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928127+0000", 2024-09-16T14:49:32.100 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:32.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.100 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.100 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.100 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928150+0000", 2024-09-16T14:49:32.100 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3357e-05 2024-09-16T14:49:32.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.101 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.101 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.101 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928161+0000", 2024-09-16T14:49:32.101 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0276e-05 2024-09-16T14:49:32.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.101 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.101 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.101 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928170+0000", 2024-09-16T14:49:32.101 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9660000000000002e-06 2024-09-16T14:49:32.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.101 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.101 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.101 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928249+0000", 2024-09-16T14:49:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8912999999999995e-05 2024-09-16T14:49:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928261+0000", 2024-09-16T14:49:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2174999999999999e-05 2024-09-16T14:49:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.931329+0000", 2024-09-16T14:49:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003068004 2024-09-16T14:49:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.103 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.931377+0000", 2024-09-16T14:49:32.103 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8195999999999997e-05 2024-09-16T14:49:32.103 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.103 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.103 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.103 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.103 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 82/65 ECSubWrite(tid=50, reqid=osd.4.0:50, at_version=82'35, trim_to=0'0, roll_forward_to=81'33))", 2024-09-16T14:49:32.103 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.928596+0000", 2024-09-16T14:49:32.103 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.1145621810000002, 2024-09-16T14:49:32.103 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031879780000000002, 2024-09-16T14:49:32.103 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.103 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.104 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.104 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.104 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.104 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928596+0000", 2024-09-16T14:49:32.104 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.104 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.104 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.104 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928596+0000", 2024-09-16T14:49:32.104 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.104 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.104 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.104 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928596+0000", 2024-09-16T14:49:32.105 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.105 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.105 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.105 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928621+0000", 2024-09-16T14:49:32.105 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4918000000000001e-05 2024-09-16T14:49:32.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.105 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.105 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.105 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928634+0000", 2024-09-16T14:49:32.105 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3035e-05 2024-09-16T14:49:32.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.105 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.105 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.106 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928642+0000", 2024-09-16T14:49:32.106 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.1659999999999994e-06 2024-09-16T14:49:32.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.106 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.106 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.106 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928700+0000", 2024-09-16T14:49:32.106 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.7522999999999998e-05 2024-09-16T14:49:32.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.106 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.106 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.106 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928710+0000", 2024-09-16T14:49:32.106 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0268999999999999e-05 2024-09-16T14:49:32.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.106 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.107 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.107 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.931755+0000", 2024-09-16T14:49:32.107 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0030452719999999999 2024-09-16T14:49:32.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.107 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.107 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.107 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.931784+0000", 2024-09-16T14:49:32.107 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.9107000000000001e-05 2024-09-16T14:49:32.107 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.107 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.107 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.107 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.108 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s6 82/65 ECSubWrite(tid=85, reqid=osd.1.0:85, at_version=82'68, trim_to=0'0, roll_forward_to=81'66))", 2024-09-16T14:49:32.108 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.928836+0000", 2024-09-16T14:49:32.108 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.1143223149999999, 2024-09-16T14:49:32.108 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0035656770000000002, 2024-09-16T14:49:32.108 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.108 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.108 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.108 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.108 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.108 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928836+0000", 2024-09-16T14:49:32.108 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.108 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.109 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.109 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928836+0000", 2024-09-16T14:49:32.109 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.109 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.109 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.109 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928836+0000", 2024-09-16T14:49:32.109 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.109 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.109 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.109 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928858+0000", 2024-09-16T14:49:32.109 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2561e-05 2024-09-16T14:49:32.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.110 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.110 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.110 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928871+0000", 2024-09-16T14:49:32.110 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3054000000000001e-05 2024-09-16T14:49:32.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.110 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.110 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.110 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928880+0000", 2024-09-16T14:49:32.110 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.32e-06 2024-09-16T14:49:32.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.110 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.110 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.110 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928936+0000", 2024-09-16T14:49:32.111 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.6548999999999997e-05 2024-09-16T14:49:32.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.111 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.111 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.111 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928946+0000", 2024-09-16T14:49:32.111 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0064999999999999e-05 2024-09-16T14:49:32.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.111 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.111 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.111 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.932369+0000", 2024-09-16T14:49:32.111 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034226220000000002 2024-09-16T14:49:32.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.111 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.111 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.112 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.932402+0000", 2024-09-16T14:49:32.112 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2755000000000001e-05 2024-09-16T14:49:32.112 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.112 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.112 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.112 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.112 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs7 82/65 ECSubWrite(tid=345, reqid=osd.3.0:345, at_version=82'69, trim_to=0'0, roll_forward_to=81'65))", 2024-09-16T14:49:32.112 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.934477+0000", 2024-09-16T14:49:32.112 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.1086815259999998, 2024-09-16T14:49:32.112 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032851439999999998, 2024-09-16T14:49:32.112 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.112 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.113 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.113 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.113 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.113 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934477+0000", 2024-09-16T14:49:32.113 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.113 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.113 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.113 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.113 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934477+0000", 2024-09-16T14:49:32.113 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.113 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.113 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.113 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.114 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934476+0000", 2024-09-16T14:49:32.114 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.114 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.114 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.114 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934503+0000", 2024-09-16T14:49:32.114 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6469e-05 2024-09-16T14:49:32.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.114 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.114 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.114 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934516+0000", 2024-09-16T14:49:32.114 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3525e-05 2024-09-16T14:49:32.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.114 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.115 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.115 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934580+0000", 2024-09-16T14:49:32.115 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3800999999999994e-05 2024-09-16T14:49:32.115 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.115 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.115 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.115 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934907+0000", 2024-09-16T14:49:32.115 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00032683199999999999 2024-09-16T14:49:32.115 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.115 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.115 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.115 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934913+0000", 2024-09-16T14:49:32.115 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8100000000000003e-06 2024-09-16T14:49:32.116 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.116 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.116 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.116 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.937739+0000", 2024-09-16T14:49:32.116 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002826121 2024-09-16T14:49:32.116 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.116 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.116 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.116 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.937762+0000", 2024-09-16T14:49:32.116 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3036000000000001e-05 2024-09-16T14:49:32.116 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.116 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.116 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.116 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.117 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.117 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.0s2 82/65 ECSubWrite(tid=289, reqid=osd.11.0:289, at_version=82'41, trim_to=0'0, roll_forward_to=79'39))", 2024-09-16T14:49:32.117 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.936772+0000", 2024-09-16T14:49:32.117 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.1063863870000001, 2024-09-16T14:49:32.117 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048157149999999999, 2024-09-16T14:49:32.117 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.117 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.117 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.117 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.117 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.117 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936772+0000", 2024-09-16T14:49:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936772+0000", 2024-09-16T14:49:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936772+0000", 2024-09-16T14:49:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.118 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.119 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936794+0000", 2024-09-16T14:49:32.119 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2670000000000001e-05 2024-09-16T14:49:32.119 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.119 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.119 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.119 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936811+0000", 2024-09-16T14:49:32.119 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6971000000000001e-05 2024-09-16T14:49:32.119 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.119 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.119 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.119 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936820+0000", 2024-09-16T14:49:32.119 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6600000000000001e-06 2024-09-16T14:49:32.119 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.119 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.120 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.120 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936895+0000", 2024-09-16T14:49:32.120 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4531999999999997e-05 2024-09-16T14:49:32.120 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.120 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.120 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.120 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936904+0000", 2024-09-16T14:49:32.120 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1900000000000001e-06 2024-09-16T14:49:32.120 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.121 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.121 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.121 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.941567+0000", 2024-09-16T14:49:32.121 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0046633009999999999 2024-09-16T14:49:32.121 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.121 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.121 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.121 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.941588+0000", 2024-09-16T14:49:32.121 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0682e-05 2024-09-16T14:49:32.121 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.121 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.121 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.121 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.121 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.122 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds3 82/65 ECSubWrite(tid=296, reqid=osd.7.0:295, at_version=0'0, trim_to=0'0, roll_forward_to=82'67))", 2024-09-16T14:49:32.122 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.940379+0000", 2024-09-16T14:49:32.122 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.1027792120000002, 2024-09-16T14:49:32.122 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003481591, 2024-09-16T14:49:32.122 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.122 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.122 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.122 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.122 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.122 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940379+0000", 2024-09-16T14:49:32.122 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.122 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.122 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.122 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.123 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940379+0000", 2024-09-16T14:49:32.123 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.123 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.123 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.123 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.123 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940378+0000", 2024-09-16T14:49:32.123 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.123 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.123 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.123 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.123 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940390+0000", 2024-09-16T14:49:32.123 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.24e-05 2024-09-16T14:49:32.123 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.123 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.124 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.124 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940405+0000", 2024-09-16T14:49:32.124 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4786999999999999e-05 2024-09-16T14:49:32.124 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.124 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.124 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.124 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940413+0000", 2024-09-16T14:49:32.124 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8539999999999997e-06 2024-09-16T14:49:32.124 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.124 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.124 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.124 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940465+0000", 2024-09-16T14:49:32.124 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.1403999999999998e-05 2024-09-16T14:49:32.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.125 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.125 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.125 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940474+0000", 2024-09-16T14:49:32.125 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6500000000000008e-06 2024-09-16T14:49:32.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.125 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.125 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.125 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.943834+0000", 2024-09-16T14:49:32.125 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003359534 2024-09-16T14:49:32.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.125 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.125 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.125 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.943861+0000", 2024-09-16T14:49:32.126 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7072000000000001e-05 2024-09-16T14:49:32.126 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.126 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.126 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.126 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.126 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.126 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.0s2 82/65 ECSubWrite(tid=291, reqid=osd.11.0:291, at_version=82'43, trim_to=0'0, roll_forward_to=79'39))", 2024-09-16T14:49:32.126 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.940578+0000", 2024-09-16T14:49:32.126 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.1025799799999998, 2024-09-16T14:49:32.127 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032913970000000002, 2024-09-16T14:49:32.127 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.127 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.127 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.127 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.127 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.127 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940578+0000", 2024-09-16T14:49:32.127 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.127 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.127 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.127 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.128 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940578+0000", 2024-09-16T14:49:32.128 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.128 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.128 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.128 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.128 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940578+0000", 2024-09-16T14:49:32.128 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.128 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.128 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.128 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.129 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940599+0000", 2024-09-16T14:49:32.129 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1311999999999999e-05 2024-09-16T14:49:32.129 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.129 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.129 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.129 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940611+0000", 2024-09-16T14:49:32.129 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2279000000000001e-05 2024-09-16T14:49:32.129 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.129 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.129 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.129 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940621+0000", 2024-09-16T14:49:32.129 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0118999999999999e-05 2024-09-16T14:49:32.129 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.129 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.130 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.130 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940678+0000", 2024-09-16T14:49:32.130 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.6329999999999998e-05 2024-09-16T14:49:32.130 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.130 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.130 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.130 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940688+0000", 2024-09-16T14:49:32.130 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9539999999999999e-06 2024-09-16T14:49:32.130 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.130 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.130 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.130 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.943844+0000", 2024-09-16T14:49:32.130 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003156191 2024-09-16T14:49:32.130 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.131 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.131 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.131 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.943870+0000", 2024-09-16T14:49:32.131 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5918000000000002e-05 2024-09-16T14:49:32.131 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.131 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.131 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.131 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.131 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.5s3 82/81 ECSubWrite(tid=165, reqid=client.4500.0:762, at_version=82'57, trim_to=0'0, roll_forward_to=82'55))", 2024-09-16T14:49:32.131 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.959197+0000", 2024-09-16T14:49:32.131 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.08396165, 2024-09-16T14:49:32.131 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003410115, 2024-09-16T14:49:32.131 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.132 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.132 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.132 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.132 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.132 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.959197+0000", 2024-09-16T14:49:32.132 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.132 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.132 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.132 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.959197+0000", 2024-09-16T14:49:32.132 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.132 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.133 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.133 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.959196+0000", 2024-09-16T14:49:32.133 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.133 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.133 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.133 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.959223+0000", 2024-09-16T14:49:32.133 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6937999999999999e-05 2024-09-16T14:49:32.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.133 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.133 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.133 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.959246+0000", 2024-09-16T14:49:32.133 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2972999999999998e-05 2024-09-16T14:49:32.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.134 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.134 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.134 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.959255+0000", 2024-09-16T14:49:32.134 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4180000000000003e-06 2024-09-16T14:49:32.134 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.134 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.134 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.134 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.959336+0000", 2024-09-16T14:49:32.134 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.1250999999999998e-05 2024-09-16T14:49:32.134 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.134 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.134 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.134 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.959347+0000", 2024-09-16T14:49:32.134 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0307e-05 2024-09-16T14:49:32.135 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.135 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.135 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.135 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.962586+0000", 2024-09-16T14:49:32.135 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003239324 2024-09-16T14:49:32.135 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.135 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.135 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.135 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.962607+0000", 2024-09-16T14:49:32.135 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0723000000000001e-05 2024-09-16T14:49:32.135 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.135 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.135 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.135 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.136 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.136 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:767 3.as0 3:572aff57:::smithi04040151-20:head [copy-from ver 63] snapc 31=[2e,28] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:49:32.136 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:29.077215+0000", 2024-09-16T14:49:32.136 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 2.9659431189999999, 2024-09-16T14:49:32.136 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.021272019999999999, 2024-09-16T14:49:32.136 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.136 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:32.136 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:32.136 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:32.136 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:32.136 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 767 2024-09-16T14:49:32.136 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.136 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.136 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.137 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.137 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077215+0000", 2024-09-16T14:49:32.137 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.137 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.137 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.137 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.137 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077215+0000", 2024-09-16T14:49:32.137 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.137 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.137 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.137 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.137 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077213+0000", 2024-09-16T14:49:32.137 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:32.138 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.138 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.138 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.138 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077223+0000", 2024-09-16T14:49:32.138 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9890000000000004e-06 2024-09-16T14:49:32.138 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.138 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.138 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.138 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077242+0000", 2024-09-16T14:49:32.138 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8286000000000001e-05 2024-09-16T14:49:32.138 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.138 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.138 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.138 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077253+0000", 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1807999999999999e-05 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077321+0000", 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7131999999999994e-05 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077431+0000", 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0001102 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.139 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.090085+0000", 2024-09-16T14:49:32.140 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012654349 2024-09-16T14:49:32.140 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.140 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.140 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.140 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092962+0000", 2024-09-16T14:49:32.140 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002877133 2024-09-16T14:49:32.140 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.140 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.140 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:32.140 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.098442+0000", 2024-09-16T14:49:32.140 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0054794149999999996 2024-09-16T14:49:32.140 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.098487+0000", 2024-09-16T14:49:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5587000000000002e-05 2024-09-16T14:49:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:768 3.as0 3:572aff57:::smithi04040151-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:49:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 2.9658919790000002, 2024-09-16T14:49:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015338218000000001, 2024-09-16T14:49:32.141 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 768 2024-09-16T14:49:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.142 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:32.143 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:32.143 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:32.143 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.143 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.143 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:32.143 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:32.143 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.143 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.143 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.143 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:32.143 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077268+0000", 2024-09-16T14:49:32.143 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.266e-06 2024-09-16T14:49:32.143 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.144 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.144 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:32.144 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077275+0000", 2024-09-16T14:49:32.144 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6200000000000001e-06 2024-09-16T14:49:32.144 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.144 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.144 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.144 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077278+0000", 2024-09-16T14:49:32.144 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.055e-06 2024-09-16T14:49:32.144 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.144 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.144 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.144 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077612+0000", 2024-09-16T14:49:32.144 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000334321 2024-09-16T14:49:32.145 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.145 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.145 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:32.145 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077660+0000", 2024-09-16T14:49:32.145 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8161999999999997e-05 2024-09-16T14:49:32.145 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.145 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.145 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:32.145 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092440+0000", 2024-09-16T14:49:32.145 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014779825999999999 2024-09-16T14:49:32.145 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.145 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.145 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:32.146 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092530+0000", 2024-09-16T14:49:32.146 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9232000000000002e-05 2024-09-16T14:49:32.146 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:32.146 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:32.146 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:32.146 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092605+0000", 2024-09-16T14:49:32.146 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5086999999999994e-05 2024-09-16T14:49:32.146 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.146 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.146 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.146 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:32.146 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:32.147 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:32.148 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:49:32.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:32.146+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:32.249 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:32.247+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:32.261 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:32.262 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:32.262 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:32.262 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:32.270 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:49:32.350 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:32.349+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:32.418 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:32.418 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:32.418 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:32.418 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:32.419 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:32.427 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:49:32.470 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:32.468+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:32.535 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:32.535 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:32.535 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:32.535 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:32.535 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.535 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:454 3.6s0 3:608d2d89:::smithi04040151-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:32.536 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:32.536 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 34.603807336999999, 2024-09-16T14:49:32.536 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013776338000000001, 2024-09-16T14:49:32.536 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.536 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:32.536 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:32.536 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:32.536 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:32.536 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 454 2024-09-16T14:49:32.536 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.536 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.536 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.536 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.536 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:32.537 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.537 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.537 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.537 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:32.537 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.537 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.537 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.537 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:32.537 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.537 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.537 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.538 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929436+0000", 2024-09-16T14:49:32.538 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7939999999999998e-06 2024-09-16T14:49:32.538 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.538 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.538 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.538 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929444+0000", 2024-09-16T14:49:32.538 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4749999999999993e-06 2024-09-16T14:49:32.538 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.538 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.538 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.538 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929449+0000", 2024-09-16T14:49:32.538 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2750000000000003e-06 2024-09-16T14:49:32.538 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.538 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.933117+0000", 2024-09-16T14:49:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0036674250000000002 2024-09-16T14:49:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.933207+0000", 2024-09-16T14:49:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0358000000000005e-05 2024-09-16T14:49:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.937436+0000", 2024-09-16T14:49:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0042287089999999998 2024-09-16T14:49:32.539 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.937535+0000", 2024-09-16T14:49:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9501999999999997e-05 2024-09-16T14:49:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.943210+0000", 2024-09-16T14:49:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056742429999999998 2024-09-16T14:49:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.540 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 52/27 ECSubWrite(tid=182, reqid=osd.11.0:182, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.645995591000002, 2024-09-16T14:49:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014202568, 2024-09-16T14:49:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.541 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887302+0000", 2024-09-16T14:49:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.7673999999999997e-05 2024-09-16T14:49:32.542 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887313+0000", 2024-09-16T14:49:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0712e-05 2024-09-16T14:49:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887321+0000", 2024-09-16T14:49:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1329999999999999e-06 2024-09-16T14:49:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887397+0000", 2024-09-16T14:49:32.543 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5564999999999999e-05 2024-09-16T14:49:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887407+0000", 2024-09-16T14:49:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9690000000000006e-06 2024-09-16T14:49:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901422+0000", 2024-09-16T14:49:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01401468 2024-09-16T14:49:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.544 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901448+0000", 2024-09-16T14:49:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6191999999999999e-05 2024-09-16T14:49:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 52/27 ECSubWrite(tid=214, reqid=osd.7.0:214, at_version=52'35, trim_to=0'0, roll_forward_to=51'31))", 2024-09-16T14:49:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.645952391999998, 2024-09-16T14:49:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014116002000000001, 2024-09-16T14:49:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.545 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.546 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887318+0000", 2024-09-16T14:49:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0477999999999999e-05 2024-09-16T14:49:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887329+0000", 2024-09-16T14:49:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0845999999999999e-05 2024-09-16T14:49:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887337+0000", 2024-09-16T14:49:32.547 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7880000000000007e-06 2024-09-16T14:49:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887631+0000", 2024-09-16T14:49:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029412399999999999 2024-09-16T14:49:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887640+0000", 2024-09-16T14:49:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0180000000000007e-06 2024-09-16T14:49:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.548 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901368+0000", 2024-09-16T14:49:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013728346000000001 2024-09-16T14:49:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901404+0000", 2024-09-16T14:49:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5964000000000003e-05 2024-09-16T14:49:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.549 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 52/27 ECSubWrite(tid=51, reqid=osd.1.0:50, at_version=0'0, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T14:49:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.644079423999997, 2024-09-16T14:49:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013036324, 2024-09-16T14:49:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.550 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889160+0000", 2024-09-16T14:49:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889174+0000", 2024-09-16T14:49:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3416e-05 2024-09-16T14:49:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.551 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889186+0000", 2024-09-16T14:49:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2369e-05 2024-09-16T14:49:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889195+0000", 2024-09-16T14:49:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6540000000000005e-06 2024-09-16T14:49:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889243+0000", 2024-09-16T14:49:32.552 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.8455000000000003e-05 2024-09-16T14:49:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889253+0000", 2024-09-16T14:49:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3700000000000001e-06 2024-09-16T14:49:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902168+0000", 2024-09-16T14:49:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012915311 2024-09-16T14:49:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.553 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902198+0000", 2024-09-16T14:49:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9641e-05 2024-09-16T14:49:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s2 52/27 ECSubWrite(tid=29, reqid=osd.4.0:29, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 33.643276249000003, 2024-09-16T14:49:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012371216000000001, 2024-09-16T14:49:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.554 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:32.555 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889990+0000", 2024-09-16T14:49:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5548999999999999e-05 2024-09-16T14:49:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890001+0000", 2024-09-16T14:49:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.098e-05 2024-09-16T14:49:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.556 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890009+0000", 2024-09-16T14:49:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5930000000000006e-06 2024-09-16T14:49:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890069+0000", 2024-09-16T14:49:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.9855999999999997e-05 2024-09-16T14:49:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890082+0000", 2024-09-16T14:49:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3312999999999999e-05 2024-09-16T14:49:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.557 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902310+0000", 2024-09-16T14:49:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012227780000000001 2024-09-16T14:49:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902336+0000", 2024-09-16T14:49:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5652000000000001e-05 2024-09-16T14:49:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.558 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s2 61/27 ECSubWrite(tid=35, reqid=osd.4.0:34, at_version=0'0, trim_to=0'0, roll_forward_to=61'22))", 2024-09-16T14:49:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.603830747, 2024-09-16T14:49:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012712925, 2024-09-16T14:49:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.559 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929409+0000", 2024-09-16T14:49:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929422+0000", 2024-09-16T14:49:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3497e-05 2024-09-16T14:49:32.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929434+0000", 2024-09-16T14:49:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.205e-05 2024-09-16T14:49:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929443+0000", 2024-09-16T14:49:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8419999999999994e-06 2024-09-16T14:49:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929490+0000", 2024-09-16T14:49:32.561 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6825e-05 2024-09-16T14:49:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929500+0000", 2024-09-16T14:49:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8810000000000008e-06 2024-09-16T14:49:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942092+0000", 2024-09-16T14:49:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012591955 2024-09-16T14:49:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.562 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942123+0000", 2024-09-16T14:49:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1015e-05 2024-09-16T14:49:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs6 61/27 ECSubWrite(tid=235, reqid=osd.7.0:235, at_version=61'21, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.602403836000001, 2024-09-16T14:49:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011696734, 2024-09-16T14:49:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.563 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.564 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930866+0000", 2024-09-16T14:49:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9138000000000001e-05 2024-09-16T14:49:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930874+0000", 2024-09-16T14:49:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5890000000000003e-06 2024-09-16T14:49:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930882+0000", 2024-09-16T14:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.25e-06 2024-09-16T14:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930933+0000", 2024-09-16T14:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1536999999999998e-05 2024-09-16T14:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930941+0000", 2024-09-16T14:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7510000000000008e-06 2024-09-16T14:49:32.566 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942507+0000", 2024-09-16T14:49:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011565765 2024-09-16T14:49:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942534+0000", 2024-09-16T14:49:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6900999999999999e-05 2024-09-16T14:49:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.567 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.568 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 61/27 ECSubWrite(tid=63, reqid=osd.1.0:63, at_version=61'50, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:32.568 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:32.568 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.600867399999998, 2024-09-16T14:49:32.568 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014509462000000001, 2024-09-16T14:49:32.568 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.568 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.568 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.568 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.568 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:32.568 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.568 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.568 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932395+0000", 2024-09-16T14:49:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2039999999999999e-05 2024-09-16T14:49:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.569 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932404+0000", 2024-09-16T14:49:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4879999999999996e-06 2024-09-16T14:49:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932412+0000", 2024-09-16T14:49:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.61e-06 2024-09-16T14:49:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932577+0000", 2024-09-16T14:49:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00016469799999999999 2024-09-16T14:49:32.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932605+0000", 2024-09-16T14:49:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7963e-05 2024-09-16T14:49:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946863+0000", 2024-09-16T14:49:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014258600999999999 2024-09-16T14:49:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946883+0000", 2024-09-16T14:49:32.571 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9494e-05 2024-09-16T14:49:32.572 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.572 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.572 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.572 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs6 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:32.572 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:32.572 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.600783995, 2024-09-16T14:49:32.572 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014400098, 2024-09-16T14:49:32.572 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.572 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.572 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.572 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.573 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:32.573 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.573 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.573 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:32.573 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.573 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.573 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932456+0000", 2024-09-16T14:49:32.573 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.574 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.574 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932475+0000", 2024-09-16T14:49:32.574 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8695999999999999e-05 2024-09-16T14:49:32.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.574 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.574 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932485+0000", 2024-09-16T14:49:32.574 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7799999999999995e-06 2024-09-16T14:49:32.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.574 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.574 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932492+0000", 2024-09-16T14:49:32.574 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5209999999999997e-06 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932705+0000", 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00021277700000000001 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932721+0000", 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6095000000000001e-05 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946825+0000", 2024-09-16T14:49:32.575 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014103269999999999 2024-09-16T14:49:32.576 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.576 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.576 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.576 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946857+0000", 2024-09-16T14:49:32.576 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2271999999999999e-05 2024-09-16T14:49:32.576 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.576 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.576 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.576 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.576 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.576 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:32.576 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:32.576 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.600587145999999, 2024-09-16T14:49:32.576 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015300085999999999, 2024-09-16T14:49:32.577 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.577 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.577 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.577 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.577 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.577 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:32.577 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.577 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.577 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.577 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:32.577 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.578 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.578 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932653+0000", 2024-09-16T14:49:32.578 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.578 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.578 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932674+0000", 2024-09-16T14:49:32.578 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1047e-05 2024-09-16T14:49:32.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.578 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.578 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932685+0000", 2024-09-16T14:49:32.578 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0733999999999999e-05 2024-09-16T14:49:32.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.579 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.579 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932693+0000", 2024-09-16T14:49:32.579 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0819999999999999e-06 2024-09-16T14:49:32.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.579 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.579 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945297+0000", 2024-09-16T14:49:32.579 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012604198 2024-09-16T14:49:32.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.579 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.579 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945306+0000", 2024-09-16T14:49:32.580 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1549999999999996e-06 2024-09-16T14:49:32.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.580 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.580 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.580 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947927+0000", 2024-09-16T14:49:32.580 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.002621295 2024-09-16T14:49:32.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.580 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.580 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.580 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947954+0000", 2024-09-16T14:49:32.580 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6219000000000001e-05 2024-09-16T14:49:32.580 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.580 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.581 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.581 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.581 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.581 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s3 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:32.581 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:32.581 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.600499417999998, 2024-09-16T14:49:32.581 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013712046, 2024-09-16T14:49:32.581 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.581 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.581 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.581 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.581 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.581 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:32.582 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.582 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.582 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.582 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:32.582 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.582 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.582 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.582 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:32.582 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.582 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.582 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.583 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932760+0000", 2024-09-16T14:49:32.583 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9247999999999999e-05 2024-09-16T14:49:32.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.583 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.583 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.583 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932773+0000", 2024-09-16T14:49:32.583 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3081000000000001e-05 2024-09-16T14:49:32.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.583 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.583 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.583 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932783+0000", 2024-09-16T14:49:32.583 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7850000000000003e-06 2024-09-16T14:49:32.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.583 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.584 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.584 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932830+0000", 2024-09-16T14:49:32.584 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7608999999999998e-05 2024-09-16T14:49:32.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.584 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.584 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.584 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932842+0000", 2024-09-16T14:49:32.584 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1082e-05 2024-09-16T14:49:32.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.584 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.584 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.584 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946430+0000", 2024-09-16T14:49:32.584 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013588141999999999 2024-09-16T14:49:32.585 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.585 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.585 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.585 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946453+0000", 2024-09-16T14:49:32.585 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3714999999999999e-05 2024-09-16T14:49:32.585 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.585 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.585 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.585 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.585 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.585 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:32.585 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:32.585 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.600084475999999, 2024-09-16T14:49:32.586 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013268299000000001, 2024-09-16T14:49:32.586 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.586 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.586 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.586 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.586 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.586 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:32.586 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.586 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.586 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.586 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.586 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:32.586 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.587 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.587 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.587 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.587 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933155+0000", 2024-09-16T14:49:32.587 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.587 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.587 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.587 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.587 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933182+0000", 2024-09-16T14:49:32.587 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7090999999999999e-05 2024-09-16T14:49:32.587 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.587 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.587 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.587 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933202+0000", 2024-09-16T14:49:32.588 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9459000000000001e-05 2024-09-16T14:49:32.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.588 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.588 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.588 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933211+0000", 2024-09-16T14:49:32.588 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.397e-06 2024-09-16T14:49:32.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.588 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.588 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.588 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933284+0000", 2024-09-16T14:49:32.588 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2711000000000002e-05 2024-09-16T14:49:32.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.588 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.588 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.589 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933294+0000", 2024-09-16T14:49:32.589 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0079e-05 2024-09-16T14:49:32.589 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.589 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.589 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.589 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946408+0000", 2024-09-16T14:49:32.589 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013114097999999999 2024-09-16T14:49:32.589 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.589 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.589 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.589 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946425+0000", 2024-09-16T14:49:32.589 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6568000000000001e-05 2024-09-16T14:49:32.589 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.589 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.590 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.590 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.590 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs6 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:32.590 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:32.590 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.599867575000001, 2024-09-16T14:49:32.590 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014598939999999999, 2024-09-16T14:49:32.590 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.590 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.590 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.590 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.590 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.590 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:32.590 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.591 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.591 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:32.591 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.591 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.591 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933372+0000", 2024-09-16T14:49:32.591 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.591 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.591 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933398+0000", 2024-09-16T14:49:32.592 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6129999999999999e-05 2024-09-16T14:49:32.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.592 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.592 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933414+0000", 2024-09-16T14:49:32.592 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5549999999999999e-05 2024-09-16T14:49:32.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.592 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.592 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933422+0000", 2024-09-16T14:49:32.592 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8620000000000003e-06 2024-09-16T14:49:32.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.593 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.593 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945189+0000", 2024-09-16T14:49:32.593 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011767552000000001 2024-09-16T14:49:32.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.593 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.593 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945200+0000", 2024-09-16T14:49:32.593 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0766e-05 2024-09-16T14:49:32.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.593 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.593 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947957+0000", 2024-09-16T14:49:32.593 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0027565319999999999 2024-09-16T14:49:32.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.594 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.594 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947972+0000", 2024-09-16T14:49:32.594 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5359999999999999e-05 2024-09-16T14:49:32.594 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.594 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.594 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.594 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs6 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:32.594 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:32.594 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.598861309, 2024-09-16T14:49:32.594 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010566212, 2024-09-16T14:49:32.595 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.595 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.595 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.595 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.595 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:32.595 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.595 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.595 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:32.595 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.596 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.596 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:32.596 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.596 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.596 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934728+0000", 2024-09-16T14:49:32.596 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034932000000000002 2024-09-16T14:49:32.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.596 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.596 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934739+0000", 2024-09-16T14:49:32.596 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1318000000000001e-05 2024-09-16T14:49:32.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.597 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.597 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934752+0000", 2024-09-16T14:49:32.597 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2901999999999999e-05 2024-09-16T14:49:32.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.597 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.597 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934823+0000", 2024-09-16T14:49:32.597 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1118000000000006e-05 2024-09-16T14:49:32.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.597 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.597 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934834+0000", 2024-09-16T14:49:32.597 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0591000000000001e-05 2024-09-16T14:49:32.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.598 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.598 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.944917+0000", 2024-09-16T14:49:32.598 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010082906000000001 2024-09-16T14:49:32.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.598 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.598 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.944946+0000", 2024-09-16T14:49:32.598 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8895999999999999e-05 2024-09-16T14:49:32.598 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.598 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.598 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.599 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 61/27 ECSubWrite(tid=237, reqid=osd.7.0:237, at_version=61'50, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:32.599 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:32.599 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.59766119, 2024-09-16T14:49:32.599 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011297481, 2024-09-16T14:49:32.599 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.599 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.599 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.599 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.599 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:32.599 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.600 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.600 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:32.600 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.600 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.600 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.600 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935579+0000", 2024-09-16T14:49:32.600 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.600 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.600 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.600 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935608+0000", 2024-09-16T14:49:32.600 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9578999999999999e-05 2024-09-16T14:49:32.600 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935625+0000", 2024-09-16T14:49:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7075000000000002e-05 2024-09-16T14:49:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935642+0000", 2024-09-16T14:49:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6480000000000001e-05 2024-09-16T14:49:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935874+0000", 2024-09-16T14:49:32.601 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00023162399999999999 2024-09-16T14:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935887+0000", 2024-09-16T14:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3811e-05 2024-09-16T14:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946860+0000", 2024-09-16T14:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010972552999999999 2024-09-16T14:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946877+0000", 2024-09-16T14:49:32.603 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7132000000000001e-05 2024-09-16T14:49:32.603 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.603 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.603 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.603 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 61/27 ECSubWrite(tid=231, reqid=osd.11.0:231, at_version=61'37, trim_to=0'0, roll_forward_to=60'33))", 2024-09-16T14:49:32.603 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:32.603 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.579412646000002, 2024-09-16T14:49:32.603 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012494261, 2024-09-16T14:49:32.603 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.603 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.603 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953840+0000", 2024-09-16T14:49:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1983e-05 2024-09-16T14:49:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953846+0000", 2024-09-16T14:49:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.319e-06 2024-09-16T14:49:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953852+0000", 2024-09-16T14:49:32.605 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.823e-06 2024-09-16T14:49:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954193+0000", 2024-09-16T14:49:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034066199999999999 2024-09-16T14:49:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954203+0000", 2024-09-16T14:49:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0421999999999999e-05 2024-09-16T14:49:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.606 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966300+0000", 2024-09-16T14:49:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012096519 2024-09-16T14:49:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966322+0000", 2024-09-16T14:49:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2829000000000001e-05 2024-09-16T14:49:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 61/27 ECSubWrite(tid=244, reqid=osd.3.0:243, at_version=0'0, trim_to=0'0, roll_forward_to=61'81))", 2024-09-16T14:49:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:32.607 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 24.578962376, 2024-09-16T14:49:32.608 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012038332000000001, 2024-09-16T14:49:32.608 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.608 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.608 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.608 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.608 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:32.608 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.608 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.608 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:32.608 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.609 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.609 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954277+0000", 2024-09-16T14:49:32.609 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:32.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.609 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.609 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954290+0000", 2024-09-16T14:49:32.609 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2364000000000001e-05 2024-09-16T14:49:32.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.609 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.609 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954302+0000", 2024-09-16T14:49:32.610 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2665e-05 2024-09-16T14:49:32.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.610 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.610 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954322+0000", 2024-09-16T14:49:32.610 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9429e-05 2024-09-16T14:49:32.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.610 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.610 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954374+0000", 2024-09-16T14:49:32.610 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2182000000000003e-05 2024-09-16T14:49:32.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.610 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.611 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954384+0000", 2024-09-16T14:49:32.611 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0022e-05 2024-09-16T14:49:32.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.611 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.611 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966295+0000", 2024-09-16T14:49:32.611 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011910399 2024-09-16T14:49:32.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.611 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.611 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966317+0000", 2024-09-16T14:49:32.611 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2169999999999999e-05 2024-09-16T14:49:32.611 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.611 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.612 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.612 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:688 3.6s0 3:608d2d89:::smithi04040151-48:head [copy-from ver 4] snapc 2e=[2e,2b,29,28,25] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T14:49:32.612 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:21.913077+0000", 2024-09-16T14:49:32.612 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.620163764000001, 2024-09-16T14:49:32.612 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.023450634000000001, 2024-09-16T14:49:32.612 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.612 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:32.612 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:32.612 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:32.612 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:32.612 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 688 2024-09-16T14:49:32.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.613 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.613 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.613 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913077+0000", 2024-09-16T14:49:32.613 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.613 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.613 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913077+0000", 2024-09-16T14:49:32.613 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.613 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913075+0000", 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913085+0000", 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8810000000000008e-06 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913099+0000", 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.382e-05 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.614 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.615 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913108+0000", 2024-09-16T14:49:32.615 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7390000000000005e-06 2024-09-16T14:49:32.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.616 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.616 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913174+0000", 2024-09-16T14:49:32.616 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6598000000000002e-05 2024-09-16T14:49:32.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.616 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:32.616 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913288+0000", 2024-09-16T14:49:32.616 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011353000000000001 2024-09-16T14:49:32.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.617 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.617 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.929027+0000", 2024-09-16T14:49:32.617 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015739758999999999 2024-09-16T14:49:32.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.617 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.617 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.931900+0000", 2024-09-16T14:49:32.617 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0028725310000000002 2024-09-16T14:49:32.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.617 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:32.617 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.936419+0000", 2024-09-16T14:49:32.618 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0045194340000000001 2024-09-16T14:49:32.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.618 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.618 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.936528+0000", 2024-09-16T14:49:32.618 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010822199999999999 2024-09-16T14:49:32.618 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.618 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.618 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.618 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:689 3.6s0 3:608d2d89:::smithi04040151-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-09-16T14:49:32.618 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:32.619 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 10.620115631999999, 2024-09-16T14:49:32.619 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.018229170999999999, 2024-09-16T14:49:32.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.619 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:32.619 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:32.619 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:32.619 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:32.619 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 689 2024-09-16T14:49:32.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.619 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.620 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.620 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:32.620 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.620 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.620 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:32.620 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.620 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.620 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:32.621 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.621 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.621 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913128+0000", 2024-09-16T14:49:32.621 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.4580000000000002e-06 2024-09-16T14:49:32.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.621 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.621 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913137+0000", 2024-09-16T14:49:32.621 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6039999999999993e-06 2024-09-16T14:49:32.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.621 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.622 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913145+0000", 2024-09-16T14:49:32.622 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6200000000000005e-06 2024-09-16T14:49:32.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.622 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.622 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913491+0000", 2024-09-16T14:49:32.622 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034584200000000001 2024-09-16T14:49:32.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.622 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:32.622 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913536+0000", 2024-09-16T14:49:32.622 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.5269000000000003e-05 2024-09-16T14:49:32.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.623 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.623 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.930844+0000", 2024-09-16T14:49:32.623 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.017307802000000001 2024-09-16T14:49:32.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.623 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:32.623 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.930954+0000", 2024-09-16T14:49:32.623 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011033500000000001 2024-09-16T14:49:32.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.623 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.623 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.931354+0000", 2024-09-16T14:49:32.624 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00039983500000000002 2024-09-16T14:49:32.624 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.624 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.624 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.624 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 77/27 ECSubWrite(tid=311, reqid=osd.3.0:311, at_version=77'100, trim_to=0'0, roll_forward_to=76'98))", 2024-09-16T14:49:32.624 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.915984+0000", 2024-09-16T14:49:32.624 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 8.6172565599999995, 2024-09-16T14:49:32.624 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010184956, 2024-09-16T14:49:32.624 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:32.624 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:32.624 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:32.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.625 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:32.625 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.915984+0000", 2024-09-16T14:49:32.625 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.625 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:32.625 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.915984+0000", 2024-09-16T14:49:32.625 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:32.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.625 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:32.625 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.915984+0000", 2024-09-16T14:49:32.625 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:32.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.626 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:32.626 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916003+0000", 2024-09-16T14:49:32.626 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9877999999999998e-05 2024-09-16T14:49:32.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.626 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:32.626 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916014+0000", 2024-09-16T14:49:32.626 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:49:32.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.626 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:32.626 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916022+0000", 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8930000000000005e-06 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916084+0000", 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1385999999999998e-05 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916095+0000", 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0852999999999999e-05 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:32.627 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.926132+0000", 2024-09-16T14:49:32.628 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010037842999999999 2024-09-16T14:49:32.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:32.628 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:32.628 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:32.628 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.926169+0000", 2024-09-16T14:49:32.628 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6777999999999999e-05 2024-09-16T14:49:32.628 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.628 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.628 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.628 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:32.628 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:32.628 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:32.630 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:49:32.630 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:32.628+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:32.731 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:32.730+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:32.778 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:32.778 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:32.778 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:32.778 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:32.787 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:49:32.832 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:32.830+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:32.861 INFO:tasks.rados.rados.0.smithi040.stdout:324: done (0 left) 2024-09-16T14:49:32.861 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 23 current snap is 30 2024-09-16T14:49:32.861 INFO:tasks.rados.rados.0.smithi040.stdout:325: seq_num 119 ranges {835584=458752,1294336=507904,1802240=507904,2310144=458752,2768896=81920} 2024-09-16T14:49:32.868 INFO:tasks.rados.rados.0.smithi040.stdout:325: writing smithi04040151-23 from 835584 to 1294336 tid 1 2024-09-16T14:49:32.871 INFO:tasks.rados.rados.0.smithi040.stdout:325: writing smithi04040151-23 from 1294336 to 1802240 tid 2 2024-09-16T14:49:32.874 INFO:tasks.rados.rados.0.smithi040.stdout:325: writing smithi04040151-23 from 1802240 to 2310144 tid 3 2024-09-16T14:49:32.877 INFO:tasks.rados.rados.0.smithi040.stdout:325: writing smithi04040151-23 from 2310144 to 2768896 tid 4 2024-09-16T14:49:32.878 INFO:tasks.rados.rados.0.smithi040.stdout:325: writing smithi04040151-23 from 2768896 to 2850816 tid 5 2024-09-16T14:49:32.878 INFO:tasks.rados.rados.0.smithi040.stdout:326: delete oid 19 current snap is 30 2024-09-16T14:49:32.878 INFO:tasks.rados.rados.0.smithi040.stdout:325: finishing write tid 1 to smithi04040151-23 2024-09-16T14:49:32.878 INFO:tasks.rados.rados.0.smithi040.stdout:325: finishing write tid 2 to smithi04040151-23 2024-09-16T14:49:32.883 INFO:tasks.rados.rados.0.smithi040.stdout:325: finishing write tid 3 to smithi04040151-23 2024-09-16T14:49:32.884 INFO:tasks.rados.rados.0.smithi040.stdout:325: finishing write tid 4 to smithi04040151-23 2024-09-16T14:49:32.884 INFO:tasks.rados.rados.0.smithi040.stdout:325: finishing write tid 5 to smithi04040151-23 2024-09-16T14:49:32.884 INFO:tasks.rados.rados.0.smithi040.stdout:325: finishing write tid 6 to smithi04040151-23 2024-09-16T14:49:32.885 INFO:tasks.rados.rados.0.smithi040.stdout:325: finishing write tid 7 to smithi04040151-23 2024-09-16T14:49:32.885 INFO:tasks.rados.rados.0.smithi040.stdout:325: oid 23 updating version 0 to 68 2024-09-16T14:49:32.885 INFO:tasks.rados.rados.0.smithi040.stdout:325: oid 23 version 68 is already newer than 64 2024-09-16T14:49:32.886 INFO:tasks.rados.rados.0.smithi040.stdout:325: oid 23 version 68 is already newer than 66 2024-09-16T14:49:32.886 INFO:tasks.rados.rados.0.smithi040.stdout:325: oid 23 version 68 is already newer than 65 2024-09-16T14:49:32.886 INFO:tasks.rados.rados.0.smithi040.stdout:325: oid 23 updating version 68 to 69 2024-09-16T14:49:32.886 INFO:tasks.rados.rados.0.smithi040.stdout:325: oid 23 version 69 is already newer than 67 2024-09-16T14:49:32.886 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 23 v 69 (ObjNum 119 snap 30 seq_num 119) dirty exists 2024-09-16T14:49:32.886 INFO:tasks.rados.rados.0.smithi040.stdout:325: left oid 23 (ObjNum 119 snap 30 seq_num 119) 2024-09-16T14:49:32.888 INFO:tasks.rados.rados.0.smithi040.stdout:325: done (1 left) 2024-09-16T14:49:32.888 INFO:tasks.rados.rados.0.smithi040.stdout:326: done (0 left) 2024-09-16T14:49:32.888 INFO:tasks.rados.rados.0.smithi040.stdout:327: read oid 42 snap -1 2024-09-16T14:49:32.888 INFO:tasks.rados.rados.0.smithi040.stdout:327: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-16T14:49:32.888 INFO:tasks.rados.rados.0.smithi040.stdout:328: setattr oid 37 current snap is 30 2024-09-16T14:49:32.889 INFO:tasks.rados.rados.0.smithi040.stdout:329: snap_create 2024-09-16T14:49:32.893 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 37 v 95 (ObjNum 65 snap 0 seq_num 210044579) dirty exists 2024-09-16T14:49:32.897 INFO:tasks.rados.rados.0.smithi040.stdout:327: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-16T14:49:32.902 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:32.902 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:32.903 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:32.903 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:32.903 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:32.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:49:32.952 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:32.950+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:33.034 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:33.033+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:33.060 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:33.060 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:33.060 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:33.060 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:33.060 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.060 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.2s5 37/27 ECSubWrite(tid=123, reqid=osd.3.0:123, at_version=37'38, trim_to=0'0, roll_forward_to=37'34))", 2024-09-16T14:49:33.060 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:33.060 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 49.121598712999997, 2024-09-16T14:49:33.060 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0094962529999999996, 2024-09-16T14:49:33.060 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.060 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.061 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.061 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.061 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.061 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:33.061 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.061 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.061 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.061 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.061 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935903+0000", 2024-09-16T14:49:33.061 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.061 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.061 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.061 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935902+0000", 2024-09-16T14:49:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935938+0000", 2024-09-16T14:49:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5731000000000001e-05 2024-09-16T14:49:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935949+0000", 2024-09-16T14:49:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1229999999999999e-05 2024-09-16T14:49:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.062 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.063 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.063 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.935957+0000", 2024-09-16T14:49:33.063 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.473e-06 2024-09-16T14:49:33.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.063 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.063 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.063 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.936913+0000", 2024-09-16T14:49:33.063 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00095599900000000002 2024-09-16T14:49:33.063 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.063 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.063 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.063 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.936923+0000", 2024-09-16T14:49:33.063 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4059999999999994e-06 2024-09-16T14:49:33.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.064 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.064 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.064 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945371+0000", 2024-09-16T14:49:33.064 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084484279999999991 2024-09-16T14:49:33.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.064 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.064 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.064 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:43.945399+0000", 2024-09-16T14:49:33.064 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7716e-05 2024-09-16T14:49:33.064 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.064 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.064 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.065 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.065 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.065 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:302 3.4s0 3:255172f3:::smithi04040151-3:head [create,setxattr qPd\\P_iff[nbYodrbXe[dQ_okmVWqe[RceaqQRNm (630) in=40b,setxattr NWYlZi[dPXnppNo]`fatPqfSaTlsQVetOmrQm^ap (138) in=40b,setxattr iVfcaeRT]TQppeYbq_`lRobpUsaVcWXphplasbgY (1507) in=40b,setxattr YeaeOf[jt`g\\k^]VfXTjOONb`jT[WYUgdiglVjYl (8841) in=40b,setxattr [iqZhSWl_sclXkajVYTXlOX]eeeWkPi\\dOYYt_Wb (1928) in=40b,setxattr \\[gaSPWYO`NQXpTccfbk`SZPaU`lbpTk]a] (1262) in=35b,setxattr ]\\]U^niYqQmjcgO[dsO`YpRZeRV]`X]bU[X^_Qis (1086) in=40b,setxattr VafomYY]aa`a^pbWebSRaithSn_dQjQYq]aTarm\\ (1402) in=40b,setxattr acj^VZhfaaikXrXq]_haYqkl[^_atWUTaloiVj`a (105) in=40b,setxattr VNUlaeZOglfodan^Ycb`dqVeZhSTmO_rabViYVch (70) in=40b,setxattr Qka\\PoUoghRc\\Qcnl`kVbj[rs\\hSdVXmlNqha`ta (594) in=40b,setxattr qsSQhb_apjdTlikdpj\\maaVdS[mVfsh_Vqi[ogWo (771) in=40b,setxattr Tm]oi_Rf_V`pSY[VhYSXSQN^bqjtgmm^fUo^f\\ZV (1087) in=40b,setxattr \\dTq (1262) in=4b,setxattr nettWfThbXj]cUpsYrTjZ^qk^\\[ZoZ[qb]Sf[ZbN (1684) in=40b,setxattr Sa\\hrdN^YombnT\\torZ`N]aRqicS_lP^XgP`e`Oc (1963) in=40b,setxattr dWhoTgYPSTh]jfWhPTs^iVdYbtfhi_WjOdes]NXh (1051) in=40b,setxattr Qbp_QrhXQf`O]blcqeqgRchO]RrhbshoV\\cW[_es (10033) in=40b,setxattr WYbQqgdRjmdbeoaP]U]n]PTWOYkP^lWOanNTnaOr (1998) in=40b,setxattr Rfbdg`[^ZYrN]cl][Ypr^tpXmb_mbh[gf\\idocW] (10804) in=40b] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:33.065 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:33.065 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 48.125086316000001, 2024-09-16T14:49:33.065 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.009468509, 2024-09-16T14:49:33.066 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.066 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:33.066 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:33.066 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:33.066 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:33.066 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 302 2024-09-16T14:49:33.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.066 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.066 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.066 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.066 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:33.066 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.067 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.067 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.067 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932415+0000", 2024-09-16T14:49:33.067 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.067 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.067 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.067 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932413+0000", 2024-09-16T14:49:33.067 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:33.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.067 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.067 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.067 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932459+0000", 2024-09-16T14:49:33.068 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6414e-05 2024-09-16T14:49:33.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.068 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.068 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.068 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932491+0000", 2024-09-16T14:49:33.068 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1656000000000003e-05 2024-09-16T14:49:33.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.068 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.068 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.068 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932499+0000", 2024-09-16T14:49:33.068 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4940000000000003e-06 2024-09-16T14:49:33.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.068 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.069 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.069 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932557+0000", 2024-09-16T14:49:33.069 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8044000000000002e-05 2024-09-16T14:49:33.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.069 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.069 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:33.069 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.932704+0000", 2024-09-16T14:49:33.069 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00014615 2024-09-16T14:49:33.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.069 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.069 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.069 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.934021+0000", 2024-09-16T14:49:33.069 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001317052 2024-09-16T14:49:33.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.070 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.070 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.070 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941512+0000", 2024-09-16T14:49:33.070 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074917789999999996 2024-09-16T14:49:33.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.070 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.070 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:33.070 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941804+0000", 2024-09-16T14:49:33.070 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029206099999999998 2024-09-16T14:49:33.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.070 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.070 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.070 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:44.941884+0000", 2024-09-16T14:49:33.071 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9231000000000001e-05 2024-09-16T14:49:33.071 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.071 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.071 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.071 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.071 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:365 3.4s0 3:2b1464a3:::smithi04040151-28:head [copy-from ver 15] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:33.071 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:33.071 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 44.118232919, 2024-09-16T14:49:33.071 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.019409261000000001, 2024-09-16T14:49:33.071 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.071 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:33.071 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:33.072 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:33.072 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:33.072 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 365 2024-09-16T14:49:33.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.072 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.072 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.072 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.072 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:33.072 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.072 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.072 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.072 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939269+0000", 2024-09-16T14:49:33.073 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.073 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.073 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.073 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939267+0000", 2024-09-16T14:49:33.073 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:33.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.073 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.073 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.073 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939278+0000", 2024-09-16T14:49:33.073 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.093e-05 2024-09-16T14:49:33.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.073 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.074 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.074 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939295+0000", 2024-09-16T14:49:33.074 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7161000000000001e-05 2024-09-16T14:49:33.074 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.074 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.074 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.074 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939304+0000", 2024-09-16T14:49:33.074 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.329e-06 2024-09-16T14:49:33.074 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.074 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.074 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.074 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939378+0000", 2024-09-16T14:49:33.074 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3456999999999998e-05 2024-09-16T14:49:33.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.075 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.075 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:33.075 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939387+0000", 2024-09-16T14:49:33.075 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1449999999999997e-06 2024-09-16T14:49:33.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.075 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.075 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.075 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.941885+0000", 2024-09-16T14:49:33.075 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0024978349999999999 2024-09-16T14:49:33.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.075 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.076 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:33.076 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942001+0000", 2024-09-16T14:49:33.076 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011674499999999999 2024-09-16T14:49:33.076 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.076 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.076 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.076 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.953426+0000", 2024-09-16T14:49:33.076 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011424808999999999 2024-09-16T14:49:33.076 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.076 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.076 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.076 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956855+0000", 2024-09-16T14:49:33.077 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0034292160000000001 2024-09-16T14:49:33.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.077 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.077 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:33.077 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.958596+0000", 2024-09-16T14:49:33.077 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001740969 2024-09-16T14:49:33.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.077 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.077 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.077 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.958678+0000", 2024-09-16T14:49:33.077 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1592000000000001e-05 2024-09-16T14:49:33.077 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.077 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.078 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.078 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.078 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:366 3.4s0 3:2b1464a3:::smithi04040151-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:33.078 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:33.078 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 44.118183348999999, 2024-09-16T14:49:33.078 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.017101175, 2024-09-16T14:49:33.078 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.078 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:33.079 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:33.079 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:33.079 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:33.079 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 366 2024-09-16T14:49:33.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.079 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.079 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.079 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.079 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:33.079 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.079 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.079 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.080 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:33.080 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.080 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.080 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.080 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939318+0000", 2024-09-16T14:49:33.080 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.080 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.080 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.080 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939321+0000", 2024-09-16T14:49:33.080 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.49e-06 2024-09-16T14:49:33.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.080 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.081 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.081 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939331+0000", 2024-09-16T14:49:33.081 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.006e-05 2024-09-16T14:49:33.081 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.081 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.081 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.081 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939336+0000", 2024-09-16T14:49:33.081 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2959999999999998e-06 2024-09-16T14:49:33.081 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.081 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.081 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.081 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939454+0000", 2024-09-16T14:49:33.081 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000117726 2024-09-16T14:49:33.081 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.082 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.082 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:33.082 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.939461+0000", 2024-09-16T14:49:33.082 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1940000000000001e-06 2024-09-16T14:49:33.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.082 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.082 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.082 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942256+0000", 2024-09-16T14:49:33.082 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0027949350000000001 2024-09-16T14:49:33.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.082 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.082 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:33.082 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.942325+0000", 2024-09-16T14:49:33.083 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.9119e-05 2024-09-16T14:49:33.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.083 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.083 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.083 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956179+0000", 2024-09-16T14:49:33.083 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013853343000000001 2024-09-16T14:49:33.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.083 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.083 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:33.083 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956295+0000", 2024-09-16T14:49:33.083 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000115899 2024-09-16T14:49:33.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.083 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.084 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.084 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.956419+0000", 2024-09-16T14:49:33.084 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012467399999999999 2024-09-16T14:49:33.084 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.084 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.084 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.084 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.084 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.084 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:391 3.4s0 3:20ea1aa7:::smithi04040151-17:head [append 0~0] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:33.084 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:33.084 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 39.192971303, 2024-09-16T14:49:33.084 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011589693999999999, 2024-09-16T14:49:33.084 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.085 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:33.085 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:33.085 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:33.085 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:33.085 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 391 2024-09-16T14:49:33.085 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.085 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.085 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.085 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.085 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:33.085 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.085 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.085 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.086 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.086 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864530+0000", 2024-09-16T14:49:33.086 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.086 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.086 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.086 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864528+0000", 2024-09-16T14:49:33.086 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:33.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.086 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.086 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.086 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864540+0000", 2024-09-16T14:49:33.086 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2514000000000001e-05 2024-09-16T14:49:33.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.087 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.087 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.087 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864569+0000", 2024-09-16T14:49:33.087 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8716000000000001e-05 2024-09-16T14:49:33.087 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.087 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.087 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.087 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864580+0000", 2024-09-16T14:49:33.087 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1564e-05 2024-09-16T14:49:33.087 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.087 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.087 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.087 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864681+0000", 2024-09-16T14:49:33.087 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010063100000000001 2024-09-16T14:49:33.088 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.088 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.088 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "op must wait for map", 2024-09-16T14:49:33.088 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864690+0000", 2024-09-16T14:49:33.088 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6300000000000004e-06 2024-09-16T14:49:33.088 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.088 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.088 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.088 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.869283+0000", 2024-09-16T14:49:33.088 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.004593186 2024-09-16T14:49:33.088 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.088 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.088 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:33.089 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.869365+0000", 2024-09-16T14:49:33.089 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.174e-05 2024-09-16T14:49:33.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.089 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.089 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.089 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870206+0000", 2024-09-16T14:49:33.089 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00084176100000000001 2024-09-16T14:49:33.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.089 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.089 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.089 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.873261+0000", 2024-09-16T14:49:33.089 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0030546549999999999 2024-09-16T14:49:33.089 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.089 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.090 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:33.090 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.876069+0000", 2024-09-16T14:49:33.090 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0028082210000000001 2024-09-16T14:49:33.090 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.090 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.090 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.090 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.876120+0000", 2024-09-16T14:49:33.090 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.0581999999999998e-05 2024-09-16T14:49:33.090 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.090 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.090 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.090 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.090 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.090 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:392 3.4s0 3:20ea1aa7:::smithi04040151-17:head [rollback c] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:33.091 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:33.091 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 39.192880723000002, 2024-09-16T14:49:33.091 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013845265000000001, 2024-09-16T14:49:33.091 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.091 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:33.091 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:33.091 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:33.091 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:33.091 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 392 2024-09-16T14:49:33.091 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.091 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.091 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.091 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.091 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:33.092 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.092 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.092 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.092 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864621+0000", 2024-09-16T14:49:33.092 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.092 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.092 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.092 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864620+0000", 2024-09-16T14:49:33.092 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:33.092 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.092 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.093 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864626+0000", 2024-09-16T14:49:33.093 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.6699999999999999e-06 2024-09-16T14:49:33.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.093 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.093 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864643+0000", 2024-09-16T14:49:33.093 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7404000000000001e-05 2024-09-16T14:49:33.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.093 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.093 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864651+0000", 2024-09-16T14:49:33.093 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0860000000000002e-06 2024-09-16T14:49:33.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.094 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.094 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864759+0000", 2024-09-16T14:49:33.094 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010751000000000001 2024-09-16T14:49:33.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.094 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:33.094 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864765+0000", 2024-09-16T14:49:33.094 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.737e-06 2024-09-16T14:49:33.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.094 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.094 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870832+0000", 2024-09-16T14:49:33.095 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060664930000000001 2024-09-16T14:49:33.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.095 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.095 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:33.095 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.870972+0000", 2024-09-16T14:49:33.095 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00014033599999999999 2024-09-16T14:49:33.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.095 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.095 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.095 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.871637+0000", 2024-09-16T14:49:33.095 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00066502800000000002 2024-09-16T14:49:33.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.095 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.095 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.873263+0000", 2024-09-16T14:49:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0016256409999999999 2024-09-16T14:49:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.878403+0000", 2024-09-16T14:49:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0051401600000000004 2024-09-16T14:49:33.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.878466+0000", 2024-09-16T14:49:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.2964000000000002e-05 2024-09-16T14:49:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:393 3.4s0 3:20ea1aa7:::smithi04040151-17:head [append 0~0] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 39.192821129000002, 2024-09-16T14:49:33.097 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.016150277000000001, 2024-09-16T14:49:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 393 2024-09-16T14:49:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.098 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.099 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.099 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:33.099 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.099 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.099 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864680+0000", 2024-09-16T14:49:33.099 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.099 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.099 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864688+0000", 2024-09-16T14:49:33.099 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5259999999999993e-06 2024-09-16T14:49:33.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.100 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.100 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864705+0000", 2024-09-16T14:49:33.100 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6368999999999999e-05 2024-09-16T14:49:33.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.100 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.100 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864712+0000", 2024-09-16T14:49:33.100 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4749999999999996e-06 2024-09-16T14:49:33.100 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.100 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.100 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864791+0000", 2024-09-16T14:49:33.101 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9159000000000006e-05 2024-09-16T14:49:33.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.101 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:33.101 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.864797+0000", 2024-09-16T14:49:33.101 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5400000000000003e-06 2024-09-16T14:49:33.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.101 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.101 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872109+0000", 2024-09-16T14:49:33.101 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073124569999999996 2024-09-16T14:49:33.101 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.102 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:33.102 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872237+0000", 2024-09-16T14:49:33.102 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012735099999999999 2024-09-16T14:49:33.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.102 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.102 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.872865+0000", 2024-09-16T14:49:33.102 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00062808000000000004 2024-09-16T14:49:33.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.102 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.102 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.102 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.874081+0000", 2024-09-16T14:49:33.102 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001216186 2024-09-16T14:49:33.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.103 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:33.103 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.880752+0000", 2024-09-16T14:49:33.103 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0066711339999999996 2024-09-16T14:49:33.103 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.103 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.103 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.103 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:53.880831+0000", 2024-09-16T14:49:33.104 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8681000000000001e-05 2024-09-16T14:49:33.104 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.104 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.104 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.104 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.104 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.104 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.9s5 51/27 ECSubWrite(tid=35, reqid=osd.8.0:35, at_version=51'21, trim_to=0'0, roll_forward_to=47'19))", 2024-09-16T14:49:33.104 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:33.104 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 35.168375875999999, 2024-09-16T14:49:33.104 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010449557999999999, 2024-09-16T14:49:33.104 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.104 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.104 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.105 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.105 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:33.105 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.105 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.105 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889126+0000", 2024-09-16T14:49:33.105 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.105 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.105 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889125+0000", 2024-09-16T14:49:33.106 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.106 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.106 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889142+0000", 2024-09-16T14:49:33.106 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7186000000000001e-05 2024-09-16T14:49:33.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.106 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.106 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889152+0000", 2024-09-16T14:49:33.106 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7920000000000004e-06 2024-09-16T14:49:33.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.107 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.107 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889159+0000", 2024-09-16T14:49:33.107 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.0319999999999998e-06 2024-09-16T14:49:33.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.107 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.107 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889203+0000", 2024-09-16T14:49:33.107 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.4264000000000002e-05 2024-09-16T14:49:33.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.107 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.107 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889212+0000", 2024-09-16T14:49:33.107 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6940000000000001e-06 2024-09-16T14:49:33.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.108 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.108 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.899537+0000", 2024-09-16T14:49:33.108 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010325328 2024-09-16T14:49:33.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.108 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.108 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.899575+0000", 2024-09-16T14:49:33.108 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.7741999999999999e-05 2024-09-16T14:49:33.108 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.108 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.108 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.109 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s5 51/27 ECSubWrite(tid=60, reqid=client.4500.0:467, at_version=51'19, trim_to=0'0, roll_forward_to=51'16))", 2024-09-16T14:49:33.109 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:33.109 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 35.059576671000002, 2024-09-16T14:49:33.109 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0099497439999999999, 2024-09-16T14:49:33.109 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.109 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.109 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.109 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.109 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:33.109 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.109 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.110 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997925+0000", 2024-09-16T14:49:33.110 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.110 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.110 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997924+0000", 2024-09-16T14:49:33.110 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:33.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.110 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.110 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997944+0000", 2024-09-16T14:49:33.110 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0404e-05 2024-09-16T14:49:33.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.111 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.111 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997957+0000", 2024-09-16T14:49:33.111 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2123000000000001e-05 2024-09-16T14:49:33.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.111 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.111 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.997966+0000", 2024-09-16T14:49:33.111 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6549999999999999e-06 2024-09-16T14:49:33.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.112 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.112 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002158+0000", 2024-09-16T14:49:33.112 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0041915930000000004 2024-09-16T14:49:33.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.112 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.112 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.002168+0000", 2024-09-16T14:49:33.112 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0071000000000001e-05 2024-09-16T14:49:33.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.112 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.112 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007845+0000", 2024-09-16T14:49:33.113 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056765690000000002 2024-09-16T14:49:33.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.113 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.113 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.007875+0000", 2024-09-16T14:49:33.113 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0077e-05 2024-09-16T14:49:33.113 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.113 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.113 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.113 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:506 3.4s0 3:255172f3:::smithi04040151-3:head [rollback 13] snapc 1b=[1b,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-16T14:49:33.113 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:33.113 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 30.163139041000001, 2024-09-16T14:49:33.114 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0092517710000000006, 2024-09-16T14:49:33.114 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.114 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:33.114 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:33.114 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:33.114 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:33.114 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 506 2024-09-16T14:49:33.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.114 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.114 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.115 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.115 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:33.115 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.115 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.115 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:33.115 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.115 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.115 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894362+0000", 2024-09-16T14:49:33.115 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.116 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.116 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894365+0000", 2024-09-16T14:49:33.116 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.571e-06 2024-09-16T14:49:33.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.116 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.116 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894370+0000", 2024-09-16T14:49:33.116 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8279999999999999e-06 2024-09-16T14:49:33.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.117 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.117 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894374+0000", 2024-09-16T14:49:33.117 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5920000000000001e-06 2024-09-16T14:49:33.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.117 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.117 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894483+0000", 2024-09-16T14:49:33.117 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010895699999999999 2024-09-16T14:49:33.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.117 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:33.117 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894491+0000", 2024-09-16T14:49:33.118 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.9389999999999997e-06 2024-09-16T14:49:33.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.118 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.118 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.897713+0000", 2024-09-16T14:49:33.118 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0032221060000000002 2024-09-16T14:49:33.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.118 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:33.118 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.897829+0000", 2024-09-16T14:49:33.118 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000116082 2024-09-16T14:49:33.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.898559+0000", 2024-09-16T14:49:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00072999800000000004 2024-09-16T14:49:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.901829+0000", 2024-09-16T14:49:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0032702159999999998 2024-09-16T14:49:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903546+0000", 2024-09-16T14:49:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001716827 2024-09-16T14:49:33.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.120 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.120 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903614+0000", 2024-09-16T14:49:33.120 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8058999999999993e-05 2024-09-16T14:49:33.120 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.120 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.120 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.120 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:507 3.4s0 3:255172f3:::smithi04040151-3:head [append 0~0] snapc 1b=[1b,15,13,10] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-16T14:49:33.120 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:33.120 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 30.163107065999998, 2024-09-16T14:49:33.121 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0095393700000000001, 2024-09-16T14:49:33.121 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.121 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:33.121 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:33.121 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:33.121 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:33.121 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 507 2024-09-16T14:49:33.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.121 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.121 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.121 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:33.121 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.122 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.122 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:33.122 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.122 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.122 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894394+0000", 2024-09-16T14:49:33.122 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.122 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.122 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894397+0000", 2024-09-16T14:49:33.122 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.774e-06 2024-09-16T14:49:33.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.123 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.123 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894403+0000", 2024-09-16T14:49:33.123 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1079999999999998e-06 2024-09-16T14:49:33.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.123 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.123 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894406+0000", 2024-09-16T14:49:33.123 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.337e-06 2024-09-16T14:49:33.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.123 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.123 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894536+0000", 2024-09-16T14:49:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012969199999999999 2024-09-16T14:49:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.894545+0000", 2024-09-16T14:49:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7499999999999992e-06 2024-09-16T14:49:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899040+0000", 2024-09-16T14:49:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0044951289999999996 2024-09-16T14:49:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:33.125 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899164+0000", 2024-09-16T14:49:33.125 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00012377699999999999 2024-09-16T14:49:33.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.125 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.125 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.899721+0000", 2024-09-16T14:49:33.125 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00055683800000000004 2024-09-16T14:49:33.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.126 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.126 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.901911+0000", 2024-09-16T14:49:33.126 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0021908650000000002 2024-09-16T14:49:33.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.126 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:33.126 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903859+0000", 2024-09-16T14:49:33.126 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001947383 2024-09-16T14:49:33.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.126 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.126 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:02.903934+0000", 2024-09-16T14:49:33.126 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5056999999999996e-05 2024-09-16T14:49:33.127 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.127 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.127 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.127 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.127 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:588 3.4s0 3:255172f3:::smithi04040151-3:head [append 0~0] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:33.127 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:11.975961+0000", 2024-09-16T14:49:33.127 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 21.081540575999998, 2024-09-16T14:49:33.127 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0258510810000001, 2024-09-16T14:49:33.127 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.127 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:33.127 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:33.127 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:33.127 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:33.128 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 588 2024-09-16T14:49:33.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.128 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.128 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.128 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975961+0000", 2024-09-16T14:49:33.128 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.128 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.128 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975961+0000", 2024-09-16T14:49:33.128 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.129 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.129 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975959+0000", 2024-09-16T14:49:33.129 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:33.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.129 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.129 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975968+0000", 2024-09-16T14:49:33.129 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1020000000000003e-06 2024-09-16T14:49:33.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.129 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.129 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975982+0000", 2024-09-16T14:49:33.129 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.33e-05 2024-09-16T14:49:33.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.130 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.130 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.975990+0000", 2024-09-16T14:49:33.130 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4679999999999992e-06 2024-09-16T14:49:33.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.130 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.130 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976080+0000", 2024-09-16T14:49:33.130 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9642999999999995e-05 2024-09-16T14:49:33.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.130 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for peered", 2024-09-16T14:49:33.130 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976083+0000", 2024-09-16T14:49:33.131 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.92e-06 2024-09-16T14:49:33.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.131 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.131 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.983734+0000", 2024-09-16T14:49:33.131 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.007651056 2024-09-16T14:49:33.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.131 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:33.131 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.984659+0000", 2024-09-16T14:49:33.131 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000925493 2024-09-16T14:49:33.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.131 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.132 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.991407+0000", 2024-09-16T14:49:33.132 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0067480969999999998 2024-09-16T14:49:33.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.132 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:33.132 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.991550+0000", 2024-09-16T14:49:33.132 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00014299700000000001 2024-09-16T14:49:33.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.132 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.132 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.992634+0000", 2024-09-16T14:49:33.132 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.001083894 2024-09-16T14:49:33.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.133 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.133 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.998056+0000", 2024-09-16T14:49:33.133 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0054220070000000004 2024-09-16T14:49:33.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.133 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:33.133 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.001429+0000", 2024-09-16T14:49:33.133 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0033727480000000001 2024-09-16T14:49:33.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.133 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.133 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.001812+0000", 2024-09-16T14:49:33.134 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00038303600000000001 2024-09-16T14:49:33.134 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.134 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.134 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.134 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:589 3.4s0 3:255172f3:::smithi04040151-3:head [rollback 24] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:33.134 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:11.976008+0000", 2024-09-16T14:49:33.134 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 21.081493631000001, 2024-09-16T14:49:33.134 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.027673222, 2024-09-16T14:49:33.134 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.134 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:33.134 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:33.134 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:33.135 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:33.135 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 589 2024-09-16T14:49:33.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.135 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.135 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.135 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976008+0000", 2024-09-16T14:49:33.135 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.135 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.135 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976008+0000", 2024-09-16T14:49:33.135 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976007+0000", 2024-09-16T14:49:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976012+0000", 2024-09-16T14:49:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9760000000000003e-06 2024-09-16T14:49:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976025+0000", 2024-09-16T14:49:33.136 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2884000000000001e-05 2024-09-16T14:49:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976031+0000", 2024-09-16T14:49:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.0079999999999999e-06 2024-09-16T14:49:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976122+0000", 2024-09-16T14:49:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0946000000000006e-05 2024-09-16T14:49:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.137 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for peered", 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976123+0000", 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1489999999999999e-06 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.985902+0000", 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.009778528 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.986100+0000", 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00019825499999999999 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.138 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.139 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.993202+0000", 2024-09-16T14:49:33.139 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0071018089999999997 2024-09-16T14:49:33.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.139 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:33.139 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.993327+0000", 2024-09-16T14:49:33.139 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000125031 2024-09-16T14:49:33.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.139 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.139 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.994109+0000", 2024-09-16T14:49:33.139 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00078209799999999995 2024-09-16T14:49:33.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.140 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.140 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.999045+0000", 2024-09-16T14:49:33.140 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0049362879999999996 2024-09-16T14:49:33.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.140 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:33.140 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.003322+0000", 2024-09-16T14:49:33.140 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0042765879999999996 2024-09-16T14:49:33.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.140 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.140 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.003681+0000", 2024-09-16T14:49:33.140 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00035919199999999999 2024-09-16T14:49:33.141 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.141 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.141 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.141 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:590 3.4s0 3:255172f3:::smithi04040151-3:head [append 0~0] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:33.141 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:11.976073+0000", 2024-09-16T14:49:33.141 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 21.081428753000001, 2024-09-16T14:49:33.141 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0282184670000001, 2024-09-16T14:49:33.141 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.141 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:33.141 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:33.141 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:33.142 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:33.142 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 590 2024-09-16T14:49:33.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.142 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.142 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.142 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976073+0000", 2024-09-16T14:49:33.142 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.142 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.142 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976073+0000", 2024-09-16T14:49:33.142 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.143 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.143 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976072+0000", 2024-09-16T14:49:33.143 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.143 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.143 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976076+0000", 2024-09-16T14:49:33.143 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.985e-06 2024-09-16T14:49:33.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.143 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.143 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976084+0000", 2024-09-16T14:49:33.143 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0080000000000002e-06 2024-09-16T14:49:33.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.144 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.144 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976095+0000", 2024-09-16T14:49:33.144 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0413e-05 2024-09-16T14:49:33.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.144 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.144 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976150+0000", 2024-09-16T14:49:33.144 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.5588999999999999e-05 2024-09-16T14:49:33.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.144 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for peered", 2024-09-16T14:49:33.144 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:11.976151+0000", 2024-09-16T14:49:33.145 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8700000000000004e-07 2024-09-16T14:49:33.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.145 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.145 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.986165+0000", 2024-09-16T14:49:33.145 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.010013982 2024-09-16T14:49:33.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.145 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:33.145 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.986260+0000", 2024-09-16T14:49:33.145 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4545999999999999e-05 2024-09-16T14:49:33.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.145 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.994573+0000", 2024-09-16T14:49:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0083134779999999991 2024-09-16T14:49:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.994693+0000", 2024-09-16T14:49:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000119707 2024-09-16T14:49:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.995284+0000", 2024-09-16T14:49:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00059083400000000004 2024-09-16T14:49:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.147 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.147 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:12.999048+0000", 2024-09-16T14:49:33.147 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.003763878 2024-09-16T14:49:33.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.147 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:33.147 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.003931+0000", 2024-09-16T14:49:33.147 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0048833339999999996 2024-09-16T14:49:33.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.147 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.147 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:13.004291+0000", 2024-09-16T14:49:33.147 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00036028999999999998 2024-09-16T14:49:33.147 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.148 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.148 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.148 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.as2 77/72 ECSubWrite(tid=12, reqid=osd.6.0:12, at_version=77'32, trim_to=0'0, roll_forward_to=74'30))", 2024-09-16T14:49:33.148 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.903509+0000", 2024-09-16T14:49:33.148 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.1539927379999995, 2024-09-16T14:49:33.148 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010698673000000001, 2024-09-16T14:49:33.148 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.148 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.148 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.148 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.149 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903509+0000", 2024-09-16T14:49:33.149 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.149 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.149 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903509+0000", 2024-09-16T14:49:33.149 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.149 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.149 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903508+0000", 2024-09-16T14:49:33.149 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.150 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.150 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903532+0000", 2024-09-16T14:49:33.150 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3799999999999999e-05 2024-09-16T14:49:33.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.150 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.150 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903542+0000", 2024-09-16T14:49:33.150 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9459999999999993e-06 2024-09-16T14:49:33.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.150 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.150 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903550+0000", 2024-09-16T14:49:33.151 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7989999999999994e-06 2024-09-16T14:49:33.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.151 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.151 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903601+0000", 2024-09-16T14:49:33.151 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1479e-05 2024-09-16T14:49:33.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.151 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.151 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.903610+0000", 2024-09-16T14:49:33.151 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0669999999999998e-06 2024-09-16T14:49:33.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.151 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.152 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.914176+0000", 2024-09-16T14:49:33.152 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010566048999999999 2024-09-16T14:49:33.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.152 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.152 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.914207+0000", 2024-09-16T14:49:33.152 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0973000000000003e-05 2024-09-16T14:49:33.152 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.152 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.153 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.153 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.bs6 77/65 ECSubWrite(tid=308, reqid=osd.3.0:308, at_version=77'53, trim_to=0'0, roll_forward_to=74'51))", 2024-09-16T14:49:33.153 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.904842+0000", 2024-09-16T14:49:33.153 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.1526590720000005, 2024-09-16T14:49:33.153 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011875419, 2024-09-16T14:49:33.153 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.153 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.153 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.154 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.154 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904842+0000", 2024-09-16T14:49:33.154 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.154 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.154 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904842+0000", 2024-09-16T14:49:33.154 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.154 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.154 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904842+0000", 2024-09-16T14:49:33.154 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:33.154 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.155 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.155 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904866+0000", 2024-09-16T14:49:33.155 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3557000000000002e-05 2024-09-16T14:49:33.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.155 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.155 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904878+0000", 2024-09-16T14:49:33.155 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2320999999999999e-05 2024-09-16T14:49:33.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.155 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.155 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.155 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904892+0000", 2024-09-16T14:49:33.155 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4042e-05 2024-09-16T14:49:33.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.156 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.156 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.156 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904956+0000", 2024-09-16T14:49:33.156 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.4026e-05 2024-09-16T14:49:33.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.156 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.156 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.156 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.904966+0000", 2024-09-16T14:49:33.156 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0128e-05 2024-09-16T14:49:33.156 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.156 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.156 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.156 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916682+0000", 2024-09-16T14:49:33.157 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011715316999999999 2024-09-16T14:49:33.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.157 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.157 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916718+0000", 2024-09-16T14:49:33.157 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6233999999999999e-05 2024-09-16T14:49:33.157 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.157 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.157 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.157 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.157 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.157 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.6s1 77/27 ECSubWrite(tid=60, reqid=osd.0.0:59, at_version=0'0, trim_to=0'0, roll_forward_to=77'40))", 2024-09-16T14:49:33.157 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.905380+0000", 2024-09-16T14:49:33.157 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.1521210830000008, 2024-09-16T14:49:33.158 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011394315, 2024-09-16T14:49:33.158 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.158 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.158 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.158 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.158 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905380+0000", 2024-09-16T14:49:33.158 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.158 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.158 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.158 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905380+0000", 2024-09-16T14:49:33.158 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.158 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.159 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.159 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.159 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905380+0000", 2024-09-16T14:49:33.159 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:33.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.159 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.159 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.159 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905391+0000", 2024-09-16T14:49:33.159 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1868999999999999e-05 2024-09-16T14:49:33.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.159 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.159 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.159 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905404+0000", 2024-09-16T14:49:33.159 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2495e-05 2024-09-16T14:49:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905413+0000", 2024-09-16T14:49:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.02e-06 2024-09-16T14:49:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905463+0000", 2024-09-16T14:49:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.0283e-05 2024-09-16T14:49:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.160 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.905473+0000", 2024-09-16T14:49:33.161 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.6250000000000002e-06 2024-09-16T14:49:33.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.161 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.161 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916749+0000", 2024-09-16T14:49:33.161 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011275821 2024-09-16T14:49:33.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.161 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.161 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916775+0000", 2024-09-16T14:49:33.161 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6055999999999999e-05 2024-09-16T14:49:33.161 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.161 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.162 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.162 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.162 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s5 77/72 ECSubWrite(tid=142, reqid=osd.5.0:142, at_version=77'47, trim_to=0'0, roll_forward_to=74'45))", 2024-09-16T14:49:33.162 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.907289+0000", 2024-09-16T14:49:33.162 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.1502124190000007, 2024-09-16T14:49:33.162 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012031185, 2024-09-16T14:49:33.162 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.162 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.162 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.162 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.162 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.162 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907289+0000", 2024-09-16T14:49:33.162 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907289+0000", 2024-09-16T14:49:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907289+0000", 2024-09-16T14:49:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.163 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907322+0000", 2024-09-16T14:49:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3012999999999997e-05 2024-09-16T14:49:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907336+0000", 2024-09-16T14:49:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4603000000000001e-05 2024-09-16T14:49:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907346+0000", 2024-09-16T14:49:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4390000000000006e-06 2024-09-16T14:49:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.165 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.165 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907421+0000", 2024-09-16T14:49:33.165 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5488999999999995e-05 2024-09-16T14:49:33.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.165 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.165 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.165 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.907432+0000", 2024-09-16T14:49:33.165 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1034e-05 2024-09-16T14:49:33.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.165 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.165 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.165 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.919295+0000", 2024-09-16T14:49:33.165 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011863146999999999 2024-09-16T14:49:33.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.166 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.166 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.166 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.919320+0000", 2024-09-16T14:49:33.166 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4791999999999999e-05 2024-09-16T14:49:33.166 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.166 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.166 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.166 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.166 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.9s5 77/27 ECSubWrite(tid=82, reqid=osd.8.0:82, at_version=77'59, trim_to=0'0, roll_forward_to=74'57))", 2024-09-16T14:49:33.166 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.910048+0000", 2024-09-16T14:49:33.166 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 9.147453252, 2024-09-16T14:49:33.166 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0093052840000000005, 2024-09-16T14:49:33.166 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.167 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.167 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.167 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.167 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910048+0000", 2024-09-16T14:49:33.167 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.167 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.167 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910048+0000", 2024-09-16T14:49:33.167 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.167 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.167 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.168 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910048+0000", 2024-09-16T14:49:33.168 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.168 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.168 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910076+0000", 2024-09-16T14:49:33.168 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7858e-05 2024-09-16T14:49:33.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.168 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.168 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910089+0000", 2024-09-16T14:49:33.168 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3129000000000001e-05 2024-09-16T14:49:33.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.169 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.169 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910096+0000", 2024-09-16T14:49:33.169 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.8009999999999998e-06 2024-09-16T14:49:33.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.169 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.169 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.169 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910185+0000", 2024-09-16T14:49:33.169 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.9568999999999996e-05 2024-09-16T14:49:33.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.169 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.169 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.169 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.910194+0000", 2024-09-16T14:49:33.169 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7390000000000005e-06 2024-09-16T14:49:33.169 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.170 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.170 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.919338+0000", 2024-09-16T14:49:33.170 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0091444439999999998 2024-09-16T14:49:33.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.170 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.170 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.919354+0000", 2024-09-16T14:49:33.170 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5258e-05 2024-09-16T14:49:33.170 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.170 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.170 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.170 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.170 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.171 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:795 3.4s0 3:2854f0ec:::smithi04040151-42:head [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-16T14:49:33.171 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:32.887674+0000", 2024-09-16T14:49:33.171 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.16982771399999999, 2024-09-16T14:49:33.171 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0096733510000000002, 2024-09-16T14:49:33.171 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.171 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:33.171 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:33.171 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:33.171 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:33.171 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 795 2024-09-16T14:49:33.171 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.171 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.171 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.171 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.887674+0000", 2024-09-16T14:49:33.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.172 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.887674+0000", 2024-09-16T14:49:33.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.172 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.172 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.887673+0000", 2024-09-16T14:49:33.172 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.172 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.172 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.173 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.173 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.887679+0000", 2024-09-16T14:49:33.173 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1500000000000004e-06 2024-09-16T14:49:33.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.173 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.173 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.887690+0000", 2024-09-16T14:49:33.173 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0730000000000001e-05 2024-09-16T14:49:33.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.173 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.173 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.173 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.887703+0000", 2024-09-16T14:49:33.173 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3229e-05 2024-09-16T14:49:33.173 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.174 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.174 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.174 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.887751+0000", 2024-09-16T14:49:33.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7212000000000001e-05 2024-09-16T14:49:33.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.174 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.174 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:33.174 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.887758+0000", 2024-09-16T14:49:33.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.323e-06 2024-09-16T14:49:33.174 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.174 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.174 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.174 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.896062+0000", 2024-09-16T14:49:33.174 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0083044179999999992 2024-09-16T14:49:33.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.175 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:33.175 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.896161+0000", 2024-09-16T14:49:33.175 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8722999999999997e-05 2024-09-16T14:49:33.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.175 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.175 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.175 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.897347+0000", 2024-09-16T14:49:33.175 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0011861280000000001 2024-09-16T14:49:33.175 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.175 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.175 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.175 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.176 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.176 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:33.177 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:49:33.178 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:33.176+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:33.278 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:33.276+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:33.285 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:33.285 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:33.285 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:33.285 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:33.294 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:49:33.379 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:33.377+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:33.400 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:33.400 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:33.400 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:33.401 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:33.401 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:33.409 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:49:33.481 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:33.480+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:33.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:49:33.557 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=128, reqid=client.4500.0:369, at_version=42'38, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 44.595677477000002, 2024-09-16T14:49:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0060794769999999998, 2024-09-16T14:49:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.558 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959977+0000", 2024-09-16T14:49:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.959976+0000", 2024-09-16T14:49:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:33.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960436+0000", 2024-09-16T14:49:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00046022400000000002 2024-09-16T14:49:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960455+0000", 2024-09-16T14:49:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8162999999999999e-05 2024-09-16T14:49:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960468+0000", 2024-09-16T14:49:33.560 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3135999999999999e-05 2024-09-16T14:49:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960527+0000", 2024-09-16T14:49:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.8859e-05 2024-09-16T14:49:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.960533+0000", 2024-09-16T14:49:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:49:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.561 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.966018+0000", 2024-09-16T14:49:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0054845069999999996 2024-09-16T14:49:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.966057+0000", 2024-09-16T14:49:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.8927999999999997e-05 2024-09-16T14:49:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=129, reqid=client.4500.0:370, at_version=42'39, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:33.562 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 44.594421719000003, 2024-09-16T14:49:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0074279639999999996, 2024-09-16T14:49:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961233+0000", 2024-09-16T14:49:33.563 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.961232+0000", 2024-09-16T14:49:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962118+0000", 2024-09-16T14:49:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00088592399999999996 2024-09-16T14:49:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.564 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962132+0000", 2024-09-16T14:49:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4127e-05 2024-09-16T14:49:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962142+0000", 2024-09-16T14:49:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0181e-05 2024-09-16T14:49:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962192+0000", 2024-09-16T14:49:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9826e-05 2024-09-16T14:49:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.565 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962202+0000", 2024-09-16T14:49:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8400000000000007e-06 2024-09-16T14:49:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968622+0000", 2024-09-16T14:49:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00642021 2024-09-16T14:49:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968661+0000", 2024-09-16T14:49:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.8670000000000001e-05 2024-09-16T14:49:33.566 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 42/27 ECSubWrite(tid=130, reqid=client.4500.0:371, at_version=42'40, trim_to=0'0, roll_forward_to=37'35))", 2024-09-16T14:49:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 44.593086464000002, 2024-09-16T14:49:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0061144900000000002, 2024-09-16T14:49:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.567 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962568+0000", 2024-09-16T14:49:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962567+0000", 2024-09-16T14:49:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.568 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962595+0000", 2024-09-16T14:49:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7824999999999999e-05 2024-09-16T14:49:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962611+0000", 2024-09-16T14:49:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:49:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.962620+0000", 2024-09-16T14:49:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8259999999999999e-06 2024-09-16T14:49:33.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.963285+0000", 2024-09-16T14:49:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00066560100000000004 2024-09-16T14:49:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.963295+0000", 2024-09-16T14:49:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0036e-05 2024-09-16T14:49:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968667+0000", 2024-09-16T14:49:33.570 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0053715960000000002 2024-09-16T14:49:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:48.968683+0000", 2024-09-16T14:49:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5522000000000001e-05 2024-09-16T14:49:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 43/27 ECSubWrite(tid=133, reqid=osd.11.0:133, at_version=43'42, trim_to=0'0, roll_forward_to=42'40))", 2024-09-16T14:49:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 43.701878669999999, 2024-09-16T14:49:33.571 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073093740000000004, 2024-09-16T14:49:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.853776+0000", 2024-09-16T14:49:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854515+0000", 2024-09-16T14:49:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00073950999999999997 2024-09-16T14:49:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854527+0000", 2024-09-16T14:49:33.573 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2012e-05 2024-09-16T14:49:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854535+0000", 2024-09-16T14:49:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4569999999999999e-06 2024-09-16T14:49:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854952+0000", 2024-09-16T14:49:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00041779399999999998 2024-09-16T14:49:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.574 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.854961+0000", 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7749999999999998e-06 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.861052+0000", 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006090608 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:49.861085+0000", 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3442999999999998e-05 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.576 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.576 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 51/27 ECSubWrite(tid=169, reqid=osd.11.0:169, at_version=51'52, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:33.576 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:33.576 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 35.674468748999999, 2024-09-16T14:49:33.576 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006781848, 2024-09-16T14:49:33.576 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.576 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.576 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.576 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.576 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.576 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:33.576 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.576 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881186+0000", 2024-09-16T14:49:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881185+0000", 2024-09-16T14:49:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881208+0000", 2024-09-16T14:49:33.577 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2823000000000002e-05 2024-09-16T14:49:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881218+0000", 2024-09-16T14:49:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3929999999999997e-06 2024-09-16T14:49:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881225+0000", 2024-09-16T14:49:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1629999999999999e-06 2024-09-16T14:49:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.578 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881279+0000", 2024-09-16T14:49:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4147000000000002e-05 2024-09-16T14:49:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881288+0000", 2024-09-16T14:49:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4630000000000007e-06 2024-09-16T14:49:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.887941+0000", 2024-09-16T14:49:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0066530210000000003 2024-09-16T14:49:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.579 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.887968+0000", 2024-09-16T14:49:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6228999999999999e-05 2024-09-16T14:49:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 51/27 ECSubWrite(tid=173, reqid=osd.11.0:173, at_version=51'54, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 35.674317522000003, 2024-09-16T14:49:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0080949799999999999, 2024-09-16T14:49:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.580 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881337+0000", 2024-09-16T14:49:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881336+0000", 2024-09-16T14:49:33.581 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.582 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.582 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.582 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881356+0000", 2024-09-16T14:49:33.582 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9334000000000002e-05 2024-09-16T14:49:33.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.582 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.582 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.582 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881366+0000", 2024-09-16T14:49:33.582 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0742e-05 2024-09-16T14:49:33.582 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.582 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.582 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.582 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.881374+0000", 2024-09-16T14:49:33.583 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5020000000000003e-06 2024-09-16T14:49:33.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.583 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.583 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.583 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.882087+0000", 2024-09-16T14:49:33.583 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00071271300000000002 2024-09-16T14:49:33.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.583 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.583 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.583 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.882096+0000", 2024-09-16T14:49:33.583 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4550000000000001e-06 2024-09-16T14:49:33.583 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.583 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.583 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.584 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889403+0000", 2024-09-16T14:49:33.584 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0073067749999999997 2024-09-16T14:49:33.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.584 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.584 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.584 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.889432+0000", 2024-09-16T14:49:33.584 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9065999999999999e-05 2024-09-16T14:49:33.584 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.584 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.584 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.584 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.584 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.584 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=235, reqid=osd.7.0:235, at_version=61'21, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:33.584 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:33.585 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.624128645999999, 2024-09-16T14:49:33.585 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0080251469999999998, 2024-09-16T14:49:33.585 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.585 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.585 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.585 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.585 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.585 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:33.585 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.585 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.585 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.585 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.585 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931526+0000", 2024-09-16T14:49:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931525+0000", 2024-09-16T14:49:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931556+0000", 2024-09-16T14:49:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0713000000000002e-05 2024-09-16T14:49:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.586 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931577+0000", 2024-09-16T14:49:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0879000000000001e-05 2024-09-16T14:49:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931599+0000", 2024-09-16T14:49:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2107999999999999e-05 2024-09-16T14:49:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932715+0000", 2024-09-16T14:49:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0011159119999999999 2024-09-16T14:49:33.587 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932725+0000", 2024-09-16T14:49:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8239999999999995e-06 2024-09-16T14:49:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939520+0000", 2024-09-16T14:49:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00679556 2024-09-16T14:49:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.588 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939551+0000", 2024-09-16T14:49:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0582e-05 2024-09-16T14:49:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 61/27 ECSubWrite(tid=63, reqid=osd.1.0:63, at_version=61'50, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:33.589 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.624007855999999, 2024-09-16T14:49:33.590 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090916260000000002, 2024-09-16T14:49:33.590 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.590 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.590 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.590 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.590 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.590 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:33.590 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931647+0000", 2024-09-16T14:49:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931646+0000", 2024-09-16T14:49:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.591 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931665+0000", 2024-09-16T14:49:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8459999999999999e-05 2024-09-16T14:49:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931677+0000", 2024-09-16T14:49:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2150000000000001e-05 2024-09-16T14:49:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931684+0000", 2024-09-16T14:49:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3599999999999998e-06 2024-09-16T14:49:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.592 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931754+0000", 2024-09-16T14:49:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.9733999999999993e-05 2024-09-16T14:49:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.931766+0000", 2024-09-16T14:49:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1873e-05 2024-09-16T14:49:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940712+0000", 2024-09-16T14:49:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0089461669999999997 2024-09-16T14:49:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940738+0000", 2024-09-16T14:49:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6092999999999999e-05 2024-09-16T14:49:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.623607842999998, 2024-09-16T14:49:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010994122, 2024-09-16T14:49:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.594 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.595 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932047+0000", 2024-09-16T14:49:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932066+0000", 2024-09-16T14:49:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9802000000000001e-05 2024-09-16T14:49:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932076+0000", 2024-09-16T14:49:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8470000000000008e-06 2024-09-16T14:49:33.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932086+0000", 2024-09-16T14:49:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7010000000000008e-06 2024-09-16T14:49:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933307+0000", 2024-09-16T14:49:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0012207450000000001 2024-09-16T14:49:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933319+0000", 2024-09-16T14:49:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2473000000000001e-05 2024-09-16T14:49:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943013+0000", 2024-09-16T14:49:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0096943700000000008 2024-09-16T14:49:33.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.599 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.599 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943041+0000", 2024-09-16T14:49:33.599 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7376999999999999e-05 2024-09-16T14:49:33.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.599 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.622483015, 2024-09-16T14:49:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0099360779999999992, 2024-09-16T14:49:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.600 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933172+0000", 2024-09-16T14:49:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933171+0000", 2024-09-16T14:49:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933205+0000", 2024-09-16T14:49:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.3630999999999997e-05 2024-09-16T14:49:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933214+0000", 2024-09-16T14:49:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0620000000000007e-06 2024-09-16T14:49:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.602 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933222+0000", 2024-09-16T14:49:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933271+0000", 2024-09-16T14:49:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.9251e-05 2024-09-16T14:49:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933280+0000", 2024-09-16T14:49:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.5419999999999994e-06 2024-09-16T14:49:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.603 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943089+0000", 2024-09-16T14:49:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0098082529999999994 2024-09-16T14:49:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.943108+0000", 2024-09-16T14:49:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8916e-05 2024-09-16T14:49:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s2 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.622344485999999, 2024-09-16T14:49:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01234993, 2024-09-16T14:49:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.605 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933310+0000", 2024-09-16T14:49:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.606 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933339+0000", 2024-09-16T14:49:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9107000000000001e-05 2024-09-16T14:49:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933347+0000", 2024-09-16T14:49:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6219999999999998e-06 2024-09-16T14:49:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933354+0000", 2024-09-16T14:49:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.72e-06 2024-09-16T14:49:33.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933438+0000", 2024-09-16T14:49:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.3666999999999997e-05 2024-09-16T14:49:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933453+0000", 2024-09-16T14:49:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4814000000000001e-05 2024-09-16T14:49:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.609 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.609 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945638+0000", 2024-09-16T14:49:33.609 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012185367000000001 2024-09-16T14:49:33.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.609 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.609 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945660+0000", 2024-09-16T14:49:33.609 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2033999999999999e-05 2024-09-16T14:49:33.609 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.609 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.609 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.610 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s5 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:33.610 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:33.610 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.622299365, 2024-09-16T14:49:33.610 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012249365, 2024-09-16T14:49:33.610 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.610 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.610 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.610 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.610 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:33.610 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933355+0000", 2024-09-16T14:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933354+0000", 2024-09-16T14:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933377+0000", 2024-09-16T14:49:33.611 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3087000000000001e-05 2024-09-16T14:49:33.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.612 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.612 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933393+0000", 2024-09-16T14:49:33.612 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5231e-05 2024-09-16T14:49:33.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.612 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.612 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933400+0000", 2024-09-16T14:49:33.612 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.6190000000000002e-06 2024-09-16T14:49:33.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.612 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.613 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933470+0000", 2024-09-16T14:49:33.613 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.936e-05 2024-09-16T14:49:33.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.613 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.613 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933479+0000", 2024-09-16T14:49:33.613 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2609999999999999e-06 2024-09-16T14:49:33.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.613 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.613 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945575+0000", 2024-09-16T14:49:33.613 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012095896 2024-09-16T14:49:33.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.614 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.614 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945605+0000", 2024-09-16T14:49:33.614 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9765e-05 2024-09-16T14:49:33.614 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.614 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.614 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.614 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:33.614 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:33.614 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.622257071, 2024-09-16T14:49:33.614 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013525466, 2024-09-16T14:49:33.615 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.615 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.615 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.615 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.615 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:33.615 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.615 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.615 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:33.615 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933397+0000", 2024-09-16T14:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933451+0000", 2024-09-16T14:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.4162000000000001e-05 2024-09-16T14:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933459+0000", 2024-09-16T14:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1110000000000008e-06 2024-09-16T14:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.5050000000000004e-06 2024-09-16T14:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.936478+0000", 2024-09-16T14:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0030122790000000001 2024-09-16T14:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.936488+0000", 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0197000000000001e-05 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946886+0000", 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010397912 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946923+0000", 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6699999999999998e-05 2024-09-16T14:49:33.618 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.621099610000002, 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0083397039999999999, 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:33.619 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.620 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.620 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934555+0000", 2024-09-16T14:49:33.620 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.620 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.620 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934554+0000", 2024-09-16T14:49:33.620 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:33.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.621 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.621 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935547+0000", 2024-09-16T14:49:33.621 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00099309300000000001 2024-09-16T14:49:33.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.621 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.621 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935565+0000", 2024-09-16T14:49:33.621 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8335999999999999e-05 2024-09-16T14:49:33.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.621 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.621 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935578+0000", 2024-09-16T14:49:33.622 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2831e-05 2024-09-16T14:49:33.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.622 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.622 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935653+0000", 2024-09-16T14:49:33.622 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.4411000000000003e-05 2024-09-16T14:49:33.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.622 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.622 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935663+0000", 2024-09-16T14:49:33.622 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0869999999999999e-05 2024-09-16T14:49:33.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.623 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.623 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942652+0000", 2024-09-16T14:49:33.623 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.006988129 2024-09-16T14:49:33.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.623 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.623 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942895+0000", 2024-09-16T14:49:33.623 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00024308899999999999 2024-09-16T14:49:33.623 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.623 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.623 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.624 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s2 61/53 ECSubWrite(tid=96, reqid=osd.5.0:96, at_version=61'38, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:33.624 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:33.624 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.617760730000001, 2024-09-16T14:49:33.624 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090892060000000007, 2024-09-16T14:49:33.624 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.624 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.624 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.624 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.624 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:33.624 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.624 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.625 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.625 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937894+0000", 2024-09-16T14:49:33.625 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.625 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.625 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937893+0000", 2024-09-16T14:49:33.625 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:33.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.625 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.625 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937915+0000", 2024-09-16T14:49:33.625 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2566e-05 2024-09-16T14:49:33.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.626 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.626 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937929+0000", 2024-09-16T14:49:33.626 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3314e-05 2024-09-16T14:49:33.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.626 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.626 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.937938+0000", 2024-09-16T14:49:33.626 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.2050000000000008e-06 2024-09-16T14:49:33.626 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.626 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.627 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940150+0000", 2024-09-16T14:49:33.627 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0022119040000000002 2024-09-16T14:49:33.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.627 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.628 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.940158+0000", 2024-09-16T14:49:33.628 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.2930000000000001e-06 2024-09-16T14:49:33.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.628 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.628 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.628 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946972+0000", 2024-09-16T14:49:33.628 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0068139150000000002 2024-09-16T14:49:33.628 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.628 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.628 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.628 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946983+0000", 2024-09-16T14:49:33.628 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0907000000000001e-05 2024-09-16T14:49:33.628 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.629 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.629 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.629 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.629 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.8s2 61/27 ECSubWrite(tid=226, reqid=osd.11.0:226, at_version=61'82, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:33.629 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:33.629 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.617238114999999, 2024-09-16T14:49:33.629 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085452879999999998, 2024-09-16T14:49:33.629 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.629 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.629 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.629 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.630 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.630 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:33.630 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.630 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.630 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:33.630 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.630 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.630 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938416+0000", 2024-09-16T14:49:33.630 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:33.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.631 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.631 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.631 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938432+0000", 2024-09-16T14:49:33.631 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5931e-05 2024-09-16T14:49:33.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.631 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.631 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.631 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938444+0000", 2024-09-16T14:49:33.631 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1933000000000001e-05 2024-09-16T14:49:33.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.631 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.631 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.631 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938449+0000", 2024-09-16T14:49:33.632 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2070000000000003e-06 2024-09-16T14:49:33.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.632 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.632 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.632 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938492+0000", 2024-09-16T14:49:33.632 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.2827999999999997e-05 2024-09-16T14:49:33.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.632 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.632 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.632 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.938500+0000", 2024-09-16T14:49:33.632 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7600000000000008e-06 2024-09-16T14:49:33.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.632 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.632 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.633 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946932+0000", 2024-09-16T14:49:33.633 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0084318099999999997 2024-09-16T14:49:33.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.633 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.633 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.633 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946962+0000", 2024-09-16T14:49:33.633 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9603000000000001e-05 2024-09-16T14:49:33.633 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.633 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.633 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.633 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.634 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs3 61/27 ECSubWrite(tid=238, reqid=client.4500.0:536, at_version=61'23, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:33.634 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:33.634 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 25.616415143000001, 2024-09-16T14:49:33.634 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0064046049999999998, 2024-09-16T14:49:33.634 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.634 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.634 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.634 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.634 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:33.634 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.634 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.635 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939239+0000", 2024-09-16T14:49:33.635 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.635 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.635 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939238+0000", 2024-09-16T14:49:33.635 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:33.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.635 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.635 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939537+0000", 2024-09-16T14:49:33.635 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029911299999999999 2024-09-16T14:49:33.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.636 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.636 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939551+0000", 2024-09-16T14:49:33.636 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.4003e-05 2024-09-16T14:49:33.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.636 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.636 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939561+0000", 2024-09-16T14:49:33.636 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4269999999999997e-06 2024-09-16T14:49:33.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.636 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.636 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939777+0000", 2024-09-16T14:49:33.637 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00021644599999999999 2024-09-16T14:49:33.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.637 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.637 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.939790+0000", 2024-09-16T14:49:33.637 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2381999999999999e-05 2024-09-16T14:49:33.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.637 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.637 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945601+0000", 2024-09-16T14:49:33.637 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0058117139999999999 2024-09-16T14:49:33.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.637 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.638 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945644+0000", 2024-09-16T14:49:33.638 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.265e-05 2024-09-16T14:49:33.638 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.638 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.638 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.638 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.638 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 75/65 ECSubWrite(tid=135, reqid=client.4500.0:690, at_version=75'49, trim_to=0'0, roll_forward_to=73'47))", 2024-09-16T14:49:33.638 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:21.922611+0000", 2024-09-16T14:49:33.638 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 11.633043269, 2024-09-16T14:49:33.638 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0090307089999999996, 2024-09-16T14:49:33.638 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.638 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.638 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.639 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.639 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922611+0000", 2024-09-16T14:49:33.639 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.639 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.639 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922611+0000", 2024-09-16T14:49:33.639 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.639 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.639 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922611+0000", 2024-09-16T14:49:33.640 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:33.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.640 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.640 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922966+0000", 2024-09-16T14:49:33.640 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00035514900000000002 2024-09-16T14:49:33.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.640 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.640 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922984+0000", 2024-09-16T14:49:33.640 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7747000000000001e-05 2024-09-16T14:49:33.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.640 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.641 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.922993+0000", 2024-09-16T14:49:33.641 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8850000000000005e-06 2024-09-16T14:49:33.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.641 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.641 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.923096+0000", 2024-09-16T14:49:33.641 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000103626 2024-09-16T14:49:33.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.641 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.641 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.923107+0000", 2024-09-16T14:49:33.641 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0370999999999999e-05 2024-09-16T14:49:33.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.642 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.642 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.931610+0000", 2024-09-16T14:49:33.642 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0085030420000000006 2024-09-16T14:49:33.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.642 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.642 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.931642+0000", 2024-09-16T14:49:33.642 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.235e-05 2024-09-16T14:49:33.642 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.642 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.642 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.643 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 75/65 ECSubWrite(tid=136, reqid=client.4500.0:691, at_version=75'50, trim_to=0'0, roll_forward_to=73'47))", 2024-09-16T14:49:33.643 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:21.926141+0000", 2024-09-16T14:49:33.643 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 11.629513489000001, 2024-09-16T14:49:33.643 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0063006299999999998, 2024-09-16T14:49:33.643 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.643 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.643 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.643 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.643 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926141+0000", 2024-09-16T14:49:33.643 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.643 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926141+0000", 2024-09-16T14:49:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926140+0000", 2024-09-16T14:49:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926912+0000", 2024-09-16T14:49:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00077166799999999999 2024-09-16T14:49:33.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.645 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.645 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926932+0000", 2024-09-16T14:49:33.645 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.0273000000000001e-05 2024-09-16T14:49:33.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.645 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.645 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.926943+0000", 2024-09-16T14:49:33.645 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0934000000000001e-05 2024-09-16T14:49:33.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.645 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.645 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.927015+0000", 2024-09-16T14:49:33.645 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2012000000000001e-05 2024-09-16T14:49:33.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.646 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.646 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.927027+0000", 2024-09-16T14:49:33.646 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2490000000000001e-05 2024-09-16T14:49:33.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.646 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.646 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.932418+0000", 2024-09-16T14:49:33.646 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0053903349999999996 2024-09-16T14:49:33.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.646 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.647 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.932442+0000", 2024-09-16T14:49:33.647 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.4043e-05 2024-09-16T14:49:33.647 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.647 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.647 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.647 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs5 86/65 ECSubWrite(tid=181, reqid=client.4500.0:787, at_version=86'66, trim_to=0'0, roll_forward_to=86'65))", 2024-09-16T14:49:33.647 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:32.876281+0000", 2024-09-16T14:49:33.647 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 0.67937314699999996, 2024-09-16T14:49:33.647 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0063244859999999998, 2024-09-16T14:49:33.647 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:33.647 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:33.648 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:33.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.648 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:33.648 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.876281+0000", 2024-09-16T14:49:33.648 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.648 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:33.648 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.876281+0000", 2024-09-16T14:49:33.648 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:33.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.648 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:33.649 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.876281+0000", 2024-09-16T14:49:33.649 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:33.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.649 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.649 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:33.649 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.876502+0000", 2024-09-16T14:49:33.649 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.000221034 2024-09-16T14:49:33.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.649 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.649 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:33.649 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.876524+0000", 2024-09-16T14:49:33.650 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2137000000000002e-05 2024-09-16T14:49:33.650 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.650 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:33.650 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.876532+0000", 2024-09-16T14:49:33.650 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5720000000000003e-06 2024-09-16T14:49:33.650 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.650 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:33.650 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.876606+0000", 2024-09-16T14:49:33.650 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.3856999999999994e-05 2024-09-16T14:49:33.650 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.650 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:33.651 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.876615+0000", 2024-09-16T14:49:33.651 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.986e-06 2024-09-16T14:49:33.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.651 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.651 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:33.651 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.882580+0000", 2024-09-16T14:49:33.651 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0059651629999999999 2024-09-16T14:49:33.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:33.651 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:33.651 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:33.651 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:32.882606+0000", 2024-09-16T14:49:33.651 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5460999999999999e-05 2024-09-16T14:49:33.651 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.651 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.652 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.652 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:33.652 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:33.652 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:33.654 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:49:33.654 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:33.654 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:33.655 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:33.651+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:33.666 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:33.666 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:33.682 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:33.682 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:33.698 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:33.699 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:33.715 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:33.715 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:33.732 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:33.732 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:33.749 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:33.749 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:33.754 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:33.753+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:33.766 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:33.766 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:33.773 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:33.773 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:33.782 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:49:33.783 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:33.783 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:33.800 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:33.801 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:33.817 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:33.817 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:33.834 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:33.834 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:33.862 INFO:tasks.rados.rados.0.smithi040.stdout:327: done (2 left) 2024-09-16T14:49:33.862 INFO:tasks.rados.rados.0.smithi040.stdout:328: done (1 left) 2024-09-16T14:49:33.862 INFO:tasks.rados.rados.0.smithi040.stdout:329: done (0 left) 2024-09-16T14:49:33.862 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 32 current snap is 31 2024-09-16T14:49:33.862 INFO:tasks.rados.rados.0.smithi040.stdout:330: seq_num 121 ranges {3178496=491520,3670016=802816,4472832=720896,5193728=32768} 2024-09-16T14:49:33.863 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:33.860+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:33.873 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:49:33.881 INFO:tasks.rados.rados.0.smithi040.stdout:330: writing smithi04040151-32 from 3178496 to 3670016 tid 1 2024-09-16T14:49:33.886 INFO:tasks.rados.rados.0.smithi040.stdout:330: writing smithi04040151-32 from 3670016 to 4472832 tid 2 2024-09-16T14:49:33.890 INFO:tasks.rados.rados.0.smithi040.stdout:330: writing smithi04040151-32 from 4472832 to 5193728 tid 3 2024-09-16T14:49:33.890 INFO:tasks.rados.rados.0.smithi040.stdout:330: writing smithi04040151-32 from 5193728 to 5226496 tid 4 2024-09-16T14:49:33.890 INFO:tasks.rados.rados.0.smithi040.stdout:331: delete oid 41 current snap is 31 2024-09-16T14:49:33.892 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:33.892 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:33.896 INFO:tasks.rados.rados.0.smithi040.stdout:330: finishing write tid 1 to smithi04040151-32 2024-09-16T14:49:33.896 INFO:tasks.rados.rados.0.smithi040.stdout:330: finishing write tid 2 to smithi04040151-32 2024-09-16T14:49:33.897 INFO:tasks.rados.rados.0.smithi040.stdout:331: done (1 left) 2024-09-16T14:49:33.897 INFO:tasks.rados.rados.0.smithi040.stdout:332: copy_from oid 7 from oid 30 current snap is 31 2024-09-16T14:49:33.897 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 31 current snap is 31 2024-09-16T14:49:33.897 INFO:tasks.rados.rados.0.smithi040.stdout:333: seq_num 123 ranges {6389760=524288,6914048=573440,7487488=688128,8175616=294912} 2024-09-16T14:49:33.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:49:33.936 INFO:tasks.rados.rados.0.smithi040.stdout:333: writing smithi04040151-31 from 6389760 to 6914048 tid 1 2024-09-16T14:49:33.939 INFO:tasks.rados.rados.0.smithi040.stdout:333: writing smithi04040151-31 from 6914048 to 7487488 tid 2 2024-09-16T14:49:33.945 INFO:tasks.rados.rados.0.smithi040.stdout:333: writing smithi04040151-31 from 7487488 to 8175616 tid 3 2024-09-16T14:49:33.947 INFO:tasks.rados.rados.0.smithi040.stdout:333: writing smithi04040151-31 from 8175616 to 8470528 tid 4 2024-09-16T14:49:33.948 INFO:tasks.rados.rados.0.smithi040.stdout:334: copy_from oid 28 from oid 43 current snap is 31 2024-09-16T14:49:33.948 INFO:tasks.rados.rados.0.smithi040.stdout:330: finishing write tid 3 to smithi04040151-32 2024-09-16T14:49:33.948 INFO:tasks.rados.rados.0.smithi040.stdout:330: finishing write tid 4 to smithi04040151-32 2024-09-16T14:49:33.948 INFO:tasks.rados.rados.0.smithi040.stdout:330: finishing write tid 5 to smithi04040151-32 2024-09-16T14:49:33.948 INFO:tasks.rados.rados.0.smithi040.stdout:332: finishing copy_from to smithi04040151-7 2024-09-16T14:49:33.948 INFO:tasks.rados.rados.0.smithi040.stdout:332: got expected ENOENT (src dne) 2024-09-16T14:49:33.948 INFO:tasks.rados.rados.0.smithi040.stdout:332: finishing copy_from racing read to smithi04040151-7 2024-09-16T14:49:33.948 INFO:tasks.rados.rados.0.smithi040.stdout:330: finishing write tid 6 to smithi04040151-32 2024-09-16T14:49:33.948 INFO:tasks.rados.rados.0.smithi040.stdout:330: oid 32 updating version 0 to 68 2024-09-16T14:49:33.948 INFO:tasks.rados.rados.0.smithi040.stdout:330: oid 32 version 68 is already newer than 67 2024-09-16T14:49:33.948 INFO:tasks.rados.rados.0.smithi040.stdout:330: oid 32 updating version 68 to 69 2024-09-16T14:49:33.948 INFO:tasks.rados.rados.0.smithi040.stdout:330: oid 32 updating version 69 to 71 2024-09-16T14:49:33.949 INFO:tasks.rados.rados.0.smithi040.stdout:330: oid 32 version 71 is already newer than 70 2024-09-16T14:49:33.949 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 32 v 71 (ObjNum 121 snap 31 seq_num 121) dirty exists 2024-09-16T14:49:33.949 INFO:tasks.rados.rados.0.smithi040.stdout:330: left oid 32 (ObjNum 121 snap 31 seq_num 121) 2024-09-16T14:49:33.949 INFO:tasks.rados.rados.0.smithi040.stdout:333: finishing write tid 1 to smithi04040151-31 2024-09-16T14:49:33.949 INFO:tasks.rados.rados.0.smithi040.stdout:333: finishing write tid 2 to smithi04040151-31 2024-09-16T14:49:33.949 INFO:tasks.rados.rados.0.smithi040.stdout:330: done (3 left) 2024-09-16T14:49:33.949 INFO:tasks.rados.rados.0.smithi040.stdout:332: done (2 left) 2024-09-16T14:49:33.949 INFO:tasks.rados.rados.0.smithi040.stdout:335: rollback oid 10 current snap is 31 2024-09-16T14:49:33.949 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 10 to 29 2024-09-16T14:49:33.949 INFO:tasks.rados.rados.0.smithi040.stdout:336: rollback oid 34 current snap is 31 2024-09-16T14:49:33.949 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 34 to 29 2024-09-16T14:49:33.949 INFO:tasks.rados.rados.0.smithi040.stdout:337: snap_create 2024-09-16T14:49:33.950 INFO:tasks.rados.rados.0.smithi040.stdout:335: finishing rollback tid 1 to smithi04040151-10 2024-09-16T14:49:33.950 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 10 v 0 (ObjNum 113 snap 0 seq_num 1812442544) dirty dne 2024-09-16T14:49:33.955 INFO:tasks.rados.rados.0.smithi040.stdout:333: finishing write tid 3 to smithi04040151-31 2024-09-16T14:49:33.955 INFO:tasks.rados.rados.0.smithi040.stdout:333: finishing write tid 4 to smithi04040151-31 2024-09-16T14:49:33.955 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:33.953+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:33.957 INFO:tasks.rados.rados.0.smithi040.stdout:333: finishing write tid 5 to smithi04040151-31 2024-09-16T14:49:33.958 INFO:tasks.rados.rados.0.smithi040.stdout:336: finishing rollback tid 0 to smithi04040151-34 2024-09-16T14:49:33.959 INFO:tasks.rados.rados.0.smithi040.stdout:333: finishing write tid 6 to smithi04040151-31 2024-09-16T14:49:33.959 INFO:tasks.rados.rados.0.smithi040.stdout:333: oid 31 updating version 0 to 132 2024-09-16T14:49:33.959 INFO:tasks.rados.rados.0.smithi040.stdout:333: oid 31 version 132 is already newer than 130 2024-09-16T14:49:33.959 INFO:tasks.rados.rados.0.smithi040.stdout:333: oid 31 version 132 is already newer than 131 2024-09-16T14:49:33.959 INFO:tasks.rados.rados.0.smithi040.stdout:333: oid 31 version 132 is already newer than 128 2024-09-16T14:49:33.959 INFO:tasks.rados.rados.0.smithi040.stdout:333: oid 31 version 132 is already newer than 129 2024-09-16T14:49:33.959 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 31 v 132 (ObjNum 123 snap 31 seq_num 123) dirty exists 2024-09-16T14:49:33.959 INFO:tasks.rados.rados.0.smithi040.stdout:333: left oid 31 (ObjNum 123 snap 31 seq_num 123) 2024-09-16T14:49:33.962 INFO:tasks.rados.rados.0.smithi040.stdout:334: finishing copy_from racing read to smithi04040151-28 2024-09-16T14:49:33.962 INFO:tasks.rados.rados.0.smithi040.stdout:336: finishing rollback tid 1 to smithi04040151-34 2024-09-16T14:49:33.962 INFO:tasks.rados.rados.0.smithi040.stdout:336: finishing rollback tid 2 to smithi04040151-34 2024-09-16T14:49:33.962 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 34 v 136 (ObjNum 113 snap 26 seq_num 113) dirty exists 2024-09-16T14:49:33.965 INFO:tasks.rados.rados.0.smithi040.stdout:334: finishing copy_from to smithi04040151-28 2024-09-16T14:49:33.965 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 28 v 63 (ObjNum 62 snap 3 seq_num 62) dirty exists 2024-09-16T14:49:34.056 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:34.056 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:34.057 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:34.055+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:34.065 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:49:34.157 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:34.155+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:34.174 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:34.174 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:34.182 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:49:34.258 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:34.256+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:34.325 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:34.325 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:34.333 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:49:34.359 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:34.357+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:34.436 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:34.436 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:34.444 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:49:34.488 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:34.486+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:34.547 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:34.547 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:34.555 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:49:34.597 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:34.596+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:34.662 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:34.660+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:34.698 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:34.698 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:34.706 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:49:34.723 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 6] 2024-09-16T14:49:34.723 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:49:34.723 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2024-09-16T14:49:34.724 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:49:34.763 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:34.762+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:34.809 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:34.809 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:34.818 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:49:34.864 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:34.862+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:34.939 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is set 2024-09-16T14:49:34.940 INFO:tasks.rados.rados.0.smithi040.stdout:333: done (4 left) 2024-09-16T14:49:34.940 INFO:tasks.rados.rados.0.smithi040.stdout:334: done (3 left) 2024-09-16T14:49:34.940 INFO:tasks.rados.rados.0.smithi040.stdout:335: done (2 left) 2024-09-16T14:49:34.940 INFO:tasks.rados.rados.0.smithi040.stdout:336: done (1 left) 2024-09-16T14:49:34.940 INFO:tasks.rados.rados.0.smithi040.stdout:337: done (0 left) 2024-09-16T14:49:34.940 INFO:tasks.rados.rados.0.smithi040.stdout:338: snap_remove snap 29 2024-09-16T14:49:34.961 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:34.961 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:34.965 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:34.964+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:34.969 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:49:35.025 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:49:35.026 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:49:35.036 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-16T14:49:34.937675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":84542003,"num_objects":71,"num_object_clones":21,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":492,"num_read_kb":102103,"num_write":992,"num_write_kb":106588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":36454400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1141,"ondisk_log_size":1141,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":591160,"kb_used_data":176012,"kb_used_omap":19,"kb_used_meta":414956,"kb_avail":1131870920,"statfs":{"total":1159641169920,"available":1159035822080,"internally_reserved":0,"allocated":180236288,"data_stored":224329594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19502,"internal_metadata":424915922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6389760,"num_objects":7,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":13,"num_read_kb":12120,"num_write":82,"num_write_kb":8368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5046272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.082505"},"pg_stats":[{"pgid":"3.f","version":"82'42","reported_seq":230,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.864227+0000","last_change":"2024-09-16T14:49:29.864227+0000","last_active":"2024-09-16T14:49:29.864227+0000","last_peered":"2024-09-16T14:49:29.864227+0000","last_clean":"2024-09-16T14:49:29.864227+0000","last_became_active":"2024-09-16T14:48:35.075111+0000","last_became_peered":"2024-09-16T14:48:35.075111+0000","last_unstale":"2024-09-16T14:49:29.864227+0000","last_undegraded":"2024-09-16T14:49:29.864227+0000","last_fullsized":"2024-09-16T14:49:29.864227+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:58.979883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057476499999999998,"stat_sum":{"num_bytes":5144576,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":10586,"num_write":36,"num_write_kb":5424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"3.e","version":"83'60","reported_seq":250,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T14:49:30.864442+0000","last_change":"2024-09-16T14:49:29.900724+0000","last_active":"2024-09-16T14:49:30.864442+0000","last_peered":"2024-09-16T14:49:30.864442+0000","last_clean":"2024-09-16T14:49:30.864442+0000","last_became_active":"2024-09-16T14:48:35.075626+0000","last_became_peered":"2024-09-16T14:48:35.075626+0000","last_unstale":"2024-09-16T14:49:30.864442+0000","last_undegraded":"2024-09-16T14:49:30.864442+0000","last_fullsized":"2024-09-16T14:49:30.864442+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.908348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.040031999999999998,"stat_sum":{"num_bytes":6520832,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4438,"num_write":60,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"3.d","version":"83'69","reported_seq":313,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T14:49:30.862018+0000","last_change":"2024-09-16T14:49:29.897913+0000","last_active":"2024-09-16T14:49:30.862018+0000","last_peered":"2024-09-16T14:49:30.862018+0000","last_clean":"2024-09-16T14:49:30.862018+0000","last_became_active":"2024-09-16T14:49:12.979812+0000","last_became_peered":"2024-09-16T14:49:12.979812+0000","last_unstale":"2024-09-16T14:49:30.862018+0000","last_undegraded":"2024-09-16T14:49:30.862018+0000","last_fullsized":"2024-09-16T14:49:30.862018+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:42.685792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.034030511999999999,"stat_sum":{"num_bytes":8142848,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":6205,"num_write":57,"num_write_kb":7952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4489216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"3.c","version":"84'62","reported_seq":293,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T14:49:30.868266+0000","last_change":"2024-09-16T14:49:29.895152+0000","last_active":"2024-09-16T14:49:30.868266+0000","last_peered":"2024-09-16T14:49:30.868266+0000","last_clean":"2024-09-16T14:49:30.868266+0000","last_became_active":"2024-09-16T14:49:12.979797+0000","last_became_peered":"2024-09-16T14:49:12.979797+0000","last_unstale":"2024-09-16T14:49:30.868266+0000","last_undegraded":"2024-09-16T14:49:30.868266+0000","last_fullsized":"2024-09-16T14:49:30.868266+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.201481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036638197999999997,"stat_sum":{"num_bytes":5570560,"num_objects":6,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":28,"num_read_kb":9848,"num_write":25,"num_write_kb":7952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3440640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"3.b","version":"83'73","reported_seq":280,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.899572+0000","last_change":"2024-09-16T14:49:29.899572+0000","last_active":"2024-09-16T14:49:29.899572+0000","last_peered":"2024-09-16T14:49:29.899572+0000","last_clean":"2024-09-16T14:49:29.899572+0000","last_became_active":"2024-09-16T14:49:12.972366+0000","last_became_peered":"2024-09-16T14:49:12.972366+0000","last_unstale":"2024-09-16T14:49:29.899572+0000","last_undegraded":"2024-09-16T14:49:29.899572+0000","last_fullsized":"2024-09-16T14:49:29.899572+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:03.456665+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.041180202999999999,"stat_sum":{"num_bytes":1949696,"num_objects":5,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":16,"num_read_kb":4902,"num_write":44,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1785856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,1,6],"acting":[3,0,4,8,11,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"3.a","version":"83'52","reported_seq":134,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T14:49:32.497416+0000","last_change":"2024-09-16T14:49:29.870865+0000","last_active":"2024-09-16T14:49:32.497416+0000","last_peered":"2024-09-16T14:49:32.497416+0000","last_clean":"2024-09-16T14:49:32.497416+0000","last_became_active":"2024-09-16T14:49:19.868841+0000","last_became_peered":"2024-09-16T14:49:19.868841+0000","last_unstale":"2024-09-16T14:49:32.497416+0000","last_undegraded":"2024-09-16T14:49:32.497416+0000","last_fullsized":"2024-09-16T14:49:32.497416+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:45.972141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012921518999999999,"stat_sum":{"num_bytes":8601600,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":14,"num_read_kb":1062,"num_write":41,"num_write_kb":9184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":9224192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,10,8,9,3,7],"acting":[6,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"3.9","version":"82'65","reported_seq":265,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.864211+0000","last_change":"2024-09-16T14:49:29.864211+0000","last_active":"2024-09-16T14:49:29.864211+0000","last_peered":"2024-09-16T14:49:29.864211+0000","last_clean":"2024-09-16T14:49:29.864211+0000","last_became_active":"2024-09-16T14:48:35.075076+0000","last_became_peered":"2024-09-16T14:48:35.075076+0000","last_unstale":"2024-09-16T14:49:29.864211+0000","last_undegraded":"2024-09-16T14:49:29.864211+0000","last_fullsized":"2024-09-16T14:49:29.864211+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:46.349013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055506900000000005,"stat_sum":{"num_bytes":6144000,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":27,"num_read_kb":2555,"num_write":28,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"3.8","version":"79'93","reported_seq":285,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T14:49:30.864256+0000","last_change":"2024-09-16T14:49:29.889233+0000","last_active":"2024-09-16T14:49:30.864256+0000","last_peered":"2024-09-16T14:49:30.864256+0000","last_clean":"2024-09-16T14:49:30.864256+0000","last_became_active":"2024-09-16T14:48:34.881078+0000","last_became_peered":"2024-09-16T14:48:34.881078+0000","last_unstale":"2024-09-16T14:49:30.864256+0000","last_undegraded":"2024-09-16T14:49:30.864256+0000","last_fullsized":"2024-09-16T14:49:30.864256+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:03.772938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051012500000000005,"stat_sum":{"num_bytes":3899392,"num_objects":4,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":37,"num_read_kb":15126,"num_write":92,"num_write_kb":6592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2,4,3,5,10,8],"acting":[11,1,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"1.0","version":"20'137","reported_seq":247,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.863362+0000","last_change":"2024-09-16T14:48:22.359960+0000","last_active":"2024-09-16T14:49:29.863362+0000","last_peered":"2024-09-16T14:49:29.863362+0000","last_clean":"2024-09-16T14:49:29.863362+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:49:29.863362+0000","last_undegraded":"2024-09-16T14:49:29.863362+0000","last_fullsized":"2024-09-16T14:49:29.863362+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:48:21.118926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:37.878583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":78,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.857688+0000","last_change":"2024-09-16T14:48:26.159458+0000","last_active":"2024-09-16T14:49:29.857688+0000","last_peered":"2024-09-16T14:49:29.857688+0000","last_clean":"2024-09-16T14:49:29.857688+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:49:29.857688+0000","last_undegraded":"2024-09-16T14:49:29.857688+0000","last_fullsized":"2024-09-16T14:49:29.857688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:33.086957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"84'102","reported_seq":330,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T14:49:30.859602+0000","last_change":"2024-09-16T14:49:29.858215+0000","last_active":"2024-09-16T14:49:30.859602+0000","last_peered":"2024-09-16T14:49:30.859602+0000","last_clean":"2024-09-16T14:49:30.859602+0000","last_became_active":"2024-09-16T14:49:12.975112+0000","last_became_peered":"2024-09-16T14:49:12.975112+0000","last_unstale":"2024-09-16T14:49:30.859602+0000","last_undegraded":"2024-09-16T14:49:30.859602+0000","last_fullsized":"2024-09-16T14:49:30.859602+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:42.640118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052472300000000005,"stat_sum":{"num_bytes":6176768,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":31,"num_read_kb":6825,"num_write":59,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4259840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":77,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.863168+0000","last_change":"2024-09-16T14:48:26.568303+0000","last_active":"2024-09-16T14:49:29.863168+0000","last_peered":"2024-09-16T14:49:29.863168+0000","last_clean":"2024-09-16T14:49:29.863168+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:49:29.863168+0000","last_undegraded":"2024-09-16T14:49:29.863168+0000","last_fullsized":"2024-09-16T14:49:29.863168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:35.481460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.858828+0000","last_change":"2024-09-16T14:49:29.858828+0000","last_active":"2024-09-16T14:49:29.858828+0000","last_peered":"2024-09-16T14:49:29.858828+0000","last_clean":"2024-09-16T14:49:29.858828+0000","last_became_active":"2024-09-16T14:49:12.975153+0000","last_became_peered":"2024-09-16T14:49:12.975153+0000","last_unstale":"2024-09-16T14:49:29.858828+0000","last_undegraded":"2024-09-16T14:49:29.858828+0000","last_fullsized":"2024-09-16T14:49:29.858828+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:35.471823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048261400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.854743+0000","last_change":"2024-09-16T14:49:27.155463+0000","last_active":"2024-09-16T14:49:29.854743+0000","last_peered":"2024-09-16T14:49:29.854743+0000","last_clean":"2024-09-16T14:49:29.854743+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:49:29.854743+0000","last_undegraded":"2024-09-16T14:49:29.854743+0000","last_fullsized":"2024-09-16T14:49:29.854743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:27.155307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:27.155307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:32.986356+0000","scrub_duration":0.001601157,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"82'43","reported_seq":254,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-16T14:49:30.864286+0000","last_change":"2024-09-16T14:49:29.860546+0000","last_active":"2024-09-16T14:49:30.864286+0000","last_peered":"2024-09-16T14:49:30.864286+0000","last_clean":"2024-09-16T14:49:30.864286+0000","last_became_active":"2024-09-16T14:49:12.974901+0000","last_became_peered":"2024-09-16T14:49:12.974901+0000","last_unstale":"2024-09-16T14:49:30.864286+0000","last_undegraded":"2024-09-16T14:49:30.864286+0000","last_fullsized":"2024-09-16T14:49:30.864286+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.845606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091190200000000003,"stat_sum":{"num_bytes":5767168,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":1111,"num_write":21,"num_write_kb":6000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1294336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":90,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.858093+0000","last_change":"2024-09-16T14:49:28.351058+0000","last_active":"2024-09-16T14:49:29.858093+0000","last_peered":"2024-09-16T14:49:29.858093+0000","last_clean":"2024-09-16T14:49:29.858093+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:49:29.858093+0000","last_undegraded":"2024-09-16T14:49:29.858093+0000","last_fullsized":"2024-09-16T14:49:29.858093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-16T14:49:28.350938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:28.350938+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:42.292353+0000","scrub_duration":0.0018950309999999999,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"83'37","reported_seq":260,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.866067+0000","last_change":"2024-09-16T14:49:29.866067+0000","last_active":"2024-09-16T14:49:29.866067+0000","last_peered":"2024-09-16T14:49:29.866067+0000","last_clean":"2024-09-16T14:49:29.866067+0000","last_became_active":"2024-09-16T14:49:12.979630+0000","last_became_peered":"2024-09-16T14:49:12.979630+0000","last_unstale":"2024-09-16T14:49:29.866067+0000","last_undegraded":"2024-09-16T14:49:29.866067+0000","last_fullsized":"2024-09-16T14:49:29.866067+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:51.166853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010745802,"stat_sum":{"num_bytes":1736704,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":858,"num_write":34,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":81920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"3.5","version":"83'61","reported_seq":361,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.897610+0000","last_change":"2024-09-16T14:49:29.897610+0000","last_active":"2024-09-16T14:49:29.897610+0000","last_peered":"2024-09-16T14:49:29.897610+0000","last_clean":"2024-09-16T14:49:29.897610+0000","last_became_active":"2024-09-16T14:49:28.941909+0000","last_became_peered":"2024-09-16T14:49:28.941909+0000","last_unstale":"2024-09-16T14:49:29.897610+0000","last_undegraded":"2024-09-16T14:49:29.897610+0000","last_fullsized":"2024-09-16T14:49:29.897610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:03.755958+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.038966897,"stat_sum":{"num_bytes":4374528,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":3942,"num_write":19,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":8486912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,1,11,10],"acting":[5,3,2,6,7,1,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.867813+0000","last_change":"2024-09-16T14:48:26.161822+0000","last_active":"2024-09-16T14:49:29.867813+0000","last_peered":"2024-09-16T14:49:29.867813+0000","last_clean":"2024-09-16T14:49:29.867813+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:49:29.867813+0000","last_undegraded":"2024-09-16T14:49:29.867813+0000","last_fullsized":"2024-09-16T14:49:29.867813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:48.998147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"83'70","reported_seq":293,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.896861+0000","last_change":"2024-09-16T14:49:29.896861+0000","last_active":"2024-09-16T14:49:29.896861+0000","last_peered":"2024-09-16T14:49:29.896861+0000","last_clean":"2024-09-16T14:49:29.896861+0000","last_became_active":"2024-09-16T14:49:12.982763+0000","last_became_peered":"2024-09-16T14:49:12.982763+0000","last_unstale":"2024-09-16T14:49:29.896861+0000","last_undegraded":"2024-09-16T14:49:29.896861+0000","last_fullsized":"2024-09-16T14:49:29.896861+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:00.568854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028867647999999999,"stat_sum":{"num_bytes":3391488,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":5003,"num_write":80,"num_write_kb":5024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":3391488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,9,5,10,2,6,0],"acting":[1,11,9,5,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":77,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.863829+0000","last_change":"2024-09-16T14:48:26.156486+0000","last_active":"2024-09-16T14:49:29.863829+0000","last_peered":"2024-09-16T14:49:29.863829+0000","last_clean":"2024-09-16T14:49:29.863829+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:49:29.863829+0000","last_undegraded":"2024-09-16T14:49:29.863829+0000","last_fullsized":"2024-09-16T14:49:29.863829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.015966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.7","version":"83'126","reported_seq":323,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.907810+0000","last_change":"2024-09-16T14:49:29.907810+0000","last_active":"2024-09-16T14:49:29.907810+0000","last_peered":"2024-09-16T14:49:29.907810+0000","last_clean":"2024-09-16T14:49:29.907810+0000","last_became_active":"2024-09-16T14:48:35.075213+0000","last_became_peered":"2024-09-16T14:48:35.075213+0000","last_unstale":"2024-09-16T14:49:29.907810+0000","last_undegraded":"2024-09-16T14:49:29.907810+0000","last_fullsized":"2024-09-16T14:49:29.907810+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:38.895447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.050017613000000002,"stat_sum":{"num_bytes":12730368,"num_objects":9,"num_object_clones":4,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":52,"num_read_kb":28661,"num_write":151,"num_write_kb":13872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.867733+0000","last_change":"2024-09-16T14:48:26.161840+0000","last_active":"2024-09-16T14:49:29.867733+0000","last_peered":"2024-09-16T14:49:29.867733+0000","last_clean":"2024-09-16T14:49:29.867733+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:49:29.867733+0000","last_undegraded":"2024-09-16T14:49:29.867733+0000","last_fullsized":"2024-09-16T14:49:29.867733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:39.436189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"83'46","reported_seq":241,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-16T14:49:29.870623+0000","last_change":"2024-09-16T14:49:29.870623+0000","last_active":"2024-09-16T14:49:29.870623+0000","last_peered":"2024-09-16T14:49:29.870623+0000","last_clean":"2024-09-16T14:49:29.870623+0000","last_became_active":"2024-09-16T14:48:35.076301+0000","last_became_peered":"2024-09-16T14:48:35.076301+0000","last_unstale":"2024-09-16T14:49:29.870623+0000","last_undegraded":"2024-09-16T14:49:29.870623+0000","last_fullsized":"2024-09-16T14:49:29.870623+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:50.342709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012679107,"stat_sum":{"num_bytes":3801088,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":872,"num_write":62,"num_write_kb":5392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,5],"acting":[0,1,4,7,8,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2e","length":"2"},{"start":"31","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-16T14:49:32.497471+0000","last_change":"2024-09-16T14:49:12.961994+0000","last_active":"2024-09-16T14:49:32.497471+0000","last_peered":"2024-09-16T14:49:32.497471+0000","last_clean":"2024-09-16T14:49:32.497471+0000","last_became_active":"2024-09-16T14:49:12.961697+0000","last_became_peered":"2024-09-16T14:49:12.961697+0000","last_unstale":"2024-09-16T14:49:32.497471+0000","last_undegraded":"2024-09-16T14:49:32.497471+0000","last_fullsized":"2024-09-16T14:49:32.497471+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:46.679090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":83951616,"num_objects":67,"num_object_clones":21,"num_object_copies":536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":366,"num_read_kb":101994,"num_write":809,"num_write_kb":103648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":36454400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":168574976,"data_stored":218324992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1001,"ondisk_log_size":1001,"up":128,"acting":128,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411344,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66760,"kb_used_data":17596,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94305080,"statfs":{"total":96636764160,"available":96568401920,"internally_reserved":0,"allocated":18018304,"data_stored":23038847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50330053},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.62},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.497}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.47299999999999998}]},{"osd":2,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.44500000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.70999999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67100000000000004}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.66100000000000003}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59699999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.753}]}]},{"osd":10,"up_from":18,"seq":77309411346,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66740,"kb_used_data":17572,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94305100,"statfs":{"total":96636764160,"available":96568422400,"internally_reserved":0,"allocated":17993728,"data_stored":22676899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.434}]},{"osd":2,"last update":"Mon Sep 16 14:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.499}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.374},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.38500000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51800000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.42599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.45700000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.47899999999999998}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.52900000000000003}]}]},{"osd":6,"up_from":41,"seq":176093659147,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39016,"kb_used_data":12124,"kb_used_omap":1,"kb_used_meta":26878,"kb_avail":94332824,"statfs":{"total":96636764160,"available":96596811776,"internally_reserved":0,"allocated":12414976,"data_stored":14903752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1965,"internal_metadata":27523155},"hb_peers":[1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]}]},{"osd":1,"up_from":18,"seq":77309411346,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65592,"kb_used_data":16424,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94306248,"statfs":{"total":96636764160,"available":96569597952,"internally_reserved":0,"allocated":16818176,"data_stored":21318002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.39600000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.0590000000000002,"5min":4.0590000000000002,"15min":4.0590000000000002},"last":0.437}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.313},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.377}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":3.9620000000000002,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.68899999999999995}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.65000000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66500000000000004}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":11,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":4.6589999999999998,"5min":4.6589999999999998,"15min":4.6589999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.57899999999999996}]}]},{"osd":0,"up_from":18,"seq":77309411344,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45440,"kb_used_data":14320,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94326400,"statfs":{"total":96636764160,"available":96590233600,"internally_reserved":0,"allocated":14663680,"data_stored":19568991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.28399999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.30299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53200000000000003}]},{"osd":5,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.67500000000000004}]},{"osd":8,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.44}]},{"osd":9,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.50900000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.495}]}]},{"osd":5,"up_from":18,"seq":77309411345,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44040,"kb_used_data":12920,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327800,"statfs":{"total":96636764160,"available":96591667200,"internally_reserved":0,"allocated":13230080,"data_stored":15288690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.45400000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.502},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.252},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.34599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.432}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.63},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52500000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.67400000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.58199999999999996}]}]},{"osd":9,"up_from":17,"seq":73014444049,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49876,"kb_used_data":18760,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94321964,"statfs":{"total":96636764160,"available":96585691136,"internally_reserved":0,"allocated":19210240,"data_stored":24369398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64500000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.59399999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.621},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.52300000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.46899999999999997}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.69499999999999995}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.53600000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.57699999999999996}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.35599999999999998}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.42599999999999999}]},{"osd":11,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.499},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.56499999999999995}]}]},{"osd":8,"up_from":17,"seq":73014444049,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39696,"kb_used_data":12672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94332144,"statfs":{"total":96636764160,"available":96596115456,"internally_reserved":0,"allocated":12976128,"data_stored":15296863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.45200000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.77300000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.72999999999999998}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.79100000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.749}]}]},{"osd":7,"up_from":17,"seq":73014444049,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46416,"kb_used_data":15292,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94325424,"statfs":{"total":96636764160,"available":96589234176,"internally_reserved":0,"allocated":15659008,"data_stored":19536662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31848908},"hb_peers":[0,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5620000000000003,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.32300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.36199999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.33300000000000002}]},{"osd":4,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.311},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.376}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.496},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.59999999999999998}]},{"osd":9,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.622},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57799999999999996}]},{"osd":10,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.66900000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.63600000000000001}]}]},{"osd":4,"up_from":17,"seq":73014444048,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47888,"kb_used_data":16772,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94323952,"statfs":{"total":96636764160,"available":96587726848,"internally_reserved":0,"allocated":17174528,"data_stored":21530870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.45900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.44900000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.44}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.28699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.42599999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.60199999999999998}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.68000000000000005}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.627}]},{"osd":11,"last update":"Mon Sep 16 14:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.58699999999999997}]}]},{"osd":3,"up_from":17,"seq":73014444050,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46452,"kb_used_data":15336,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94325388,"statfs":{"total":96636764160,"available":96589197312,"internally_reserved":0,"allocated":15704064,"data_stored":19683574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.315},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.45800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.35299999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.44700000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.498},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.54100000000000004}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.629}]},{"osd":9,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.64800000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.64000000000000001}]},{"osd":11,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.68799999999999994}]}]},{"osd":2,"up_from":17,"seq":73014444049,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":6224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":6373376,"data_stored":7117046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.41399999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.433}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.504},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60099999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.624},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53600000000000003}]},{"osd":8,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.65300000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.52100000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.63700000000000001}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13688832,"data_stored":18972672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15839232,"data_stored":20721664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5595136,"data_stored":6713344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14925824,"data_stored":19582976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16211968,"data_stored":21245952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12251136,"data_stored":14692352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":11931648,"data_stored":15011840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14163968,"data_stored":18419712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12001280,"data_stored":14700544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":18432000,"data_stored":24268800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":17022976,"data_stored":22081536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16510976,"data_stored":21913600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:49:35.040 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2024-09-16T14:49:35.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pg_num 26 2024-09-16T14:49:35.065 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:35.065+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:35.072 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:35.072 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:35.081 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:49:35.167 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:35.166+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:35.223 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:35.223 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:35.231 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:49:35.268 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:35.266+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:35.344 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:35.345 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:35.354 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:49:35.369 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:35.368+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:35.458 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:35.458 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:35.466 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:49:35.470 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:35.468+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:35.568 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:35.568 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:35.570 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:35.569+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:35.576 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:49:35.671 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:35.670+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:35.718 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:35.718 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:35.727 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:49:35.772 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:35.771+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:35.846 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:49:35.868 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:35.868 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:35.876 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:49:35.890 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:35.888+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:35.941 INFO:teuthology.orchestra.run.smithi040.stderr:set pool 3 pg_num to 26 2024-09-16T14:49:35.941 INFO:tasks.rados.rados.0.smithi040.stdout:338: done (0 left) 2024-09-16T14:49:35.941 INFO:tasks.rados.rados.0.smithi040.stdout:339: snap_remove snap 20 2024-09-16T14:49:35.977 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:35.977 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:35.978 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:35.975+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:35.979 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:35.979 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:35.988 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:49:35.993 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:35.993 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:36.009 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:36.009 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:36.025 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:36.025 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:36.042 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:36.043 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:36.058 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:36.059 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:36.074 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:36.073+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:36.083 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:36.083 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:36.099 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:36.099 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:36.101 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:36.101 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:36.108 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:49:36.117 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:36.118 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:36.134 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:36.134 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:36.150 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:36.150 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:36.166 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:36.167 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:36.175 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:36.174+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:36.250 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:36.250 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:36.251 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:36.251 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:36.259 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:49:36.276 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:36.274+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:36.362 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:36.362 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:36.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:36.362 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:36.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.362 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 34/27 ECSubWrite(tid=80, reqid=osd.3.0:80, at_version=34'20, trim_to=0'0, roll_forward_to=33'16))", 2024-09-16T14:49:36.362 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:49:36.362 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.450332326999998, 2024-09-16T14:49:36.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055013639999999999, 2024-09-16T14:49:36.362 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.362 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.363 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:49:36.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909415+0000", 2024-09-16T14:49:36.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909414+0000", 2024-09-16T14:49:36.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:36.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909442+0000", 2024-09-16T14:49:36.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7662999999999999e-05 2024-09-16T14:49:36.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909452+0000", 2024-09-16T14:49:36.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0773e-05 2024-09-16T14:49:36.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909463+0000", 2024-09-16T14:49:36.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0144e-05 2024-09-16T14:49:36.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909522+0000", 2024-09-16T14:49:36.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9651000000000002e-05 2024-09-16T14:49:36.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909531+0000", 2024-09-16T14:49:36.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8529999999999998e-06 2024-09-16T14:49:36.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914874+0000", 2024-09-16T14:49:36.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053432829999999999 2024-09-16T14:49:36.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914916+0000", 2024-09-16T14:49:36.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1928000000000002e-05 2024-09-16T14:49:36.366 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.366 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.366 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.367 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:49:36.367 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:36.367 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.450263517000003, 2024-09-16T14:49:36.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079855010000000008, 2024-09-16T14:49:36.367 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.367 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.367 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:36.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:36.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909483+0000", 2024-09-16T14:49:36.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909521+0000", 2024-09-16T14:49:36.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7185999999999999e-05 2024-09-16T14:49:36.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909543+0000", 2024-09-16T14:49:36.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2802999999999998e-05 2024-09-16T14:49:36.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909551+0000", 2024-09-16T14:49:36.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5039999999999996e-06 2024-09-16T14:49:36.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909927+0000", 2024-09-16T14:49:36.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037637400000000001 2024-09-16T14:49:36.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:49:36.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2040000000000002e-06 2024-09-16T14:49:36.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917442+0000", 2024-09-16T14:49:36.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007508659 2024-09-16T14:49:36.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917469+0000", 2024-09-16T14:49:36.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7197999999999999e-05 2024-09-16T14:49:36.372 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.372 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.372 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.372 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:49:36.372 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:36.372 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.446191171000002, 2024-09-16T14:49:36.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067701159999999996, 2024-09-16T14:49:36.372 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.372 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.372 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:36.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:36.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:36.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913579+0000", 2024-09-16T14:49:36.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2829000000000001e-05 2024-09-16T14:49:36.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913591+0000", 2024-09-16T14:49:36.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2264e-05 2024-09-16T14:49:36.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913600+0000", 2024-09-16T14:49:36.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9959999999999999e-06 2024-09-16T14:49:36.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913648+0000", 2024-09-16T14:49:36.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7759000000000001e-05 2024-09-16T14:49:36.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913657+0000", 2024-09-16T14:49:36.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6709999999999994e-06 2024-09-16T14:49:36.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920296+0000", 2024-09-16T14:49:36.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066385120000000001 2024-09-16T14:49:36.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920326+0000", 2024-09-16T14:49:36.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0405000000000001e-05 2024-09-16T14:49:36.377 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.377 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.377 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.377 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 34/27 ECSubWrite(tid=74, reqid=client.4500.0:229, at_version=34'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:49:36.377 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:36.377 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.445375212999998, 2024-09-16T14:49:36.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060056090000000003, 2024-09-16T14:49:36.377 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.377 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.378 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:36.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:36.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:36.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914468+0000", 2024-09-16T14:49:36.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6398999999999996e-05 2024-09-16T14:49:36.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914483+0000", 2024-09-16T14:49:36.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5238e-05 2024-09-16T14:49:36.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914494+0000", 2024-09-16T14:49:36.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0192e-05 2024-09-16T14:49:36.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914622+0000", 2024-09-16T14:49:36.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012855599999999999 2024-09-16T14:49:36.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914635+0000", 2024-09-16T14:49:36.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2510000000000001e-05 2024-09-16T14:49:36.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920339+0000", 2024-09-16T14:49:36.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057041549999999998 2024-09-16T14:49:36.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920378+0000", 2024-09-16T14:49:36.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8918999999999998e-05 2024-09-16T14:49:36.381 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.381 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.381 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.382 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:49:36.382 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:36.382 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.443362436999998, 2024-09-16T14:49:36.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069011319999999999, 2024-09-16T14:49:36.382 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.382 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.382 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:36.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:36.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:36.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:49:36.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916410+0000", 2024-09-16T14:49:36.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5343000000000001e-05 2024-09-16T14:49:36.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916418+0000", 2024-09-16T14:49:36.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2360000000000004e-06 2024-09-16T14:49:36.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916428+0000", 2024-09-16T14:49:36.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6779999999999995e-06 2024-09-16T14:49:36.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916492+0000", 2024-09-16T14:49:36.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4219000000000003e-05 2024-09-16T14:49:36.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916502+0000", 2024-09-16T14:49:36.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6690000000000001e-06 2024-09-16T14:49:36.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923254+0000", 2024-09-16T14:49:36.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067518070000000003 2024-09-16T14:49:36.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923286+0000", 2024-09-16T14:49:36.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2367999999999999e-05 2024-09-16T14:49:36.386 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.386 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.386 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.386 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s7 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:36.386 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:36.386 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.438058052999999, 2024-09-16T14:49:36.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006972692, 2024-09-16T14:49:36.386 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.386 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.387 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:36.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:36.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:36.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921709+0000", 2024-09-16T14:49:36.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0411000000000002e-05 2024-09-16T14:49:36.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921719+0000", 2024-09-16T14:49:36.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2650000000000002e-06 2024-09-16T14:49:36.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921726+0000", 2024-09-16T14:49:36.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8770000000000004e-06 2024-09-16T14:49:36.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921773+0000", 2024-09-16T14:49:36.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7435999999999998e-05 2024-09-16T14:49:36.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921782+0000", 2024-09-16T14:49:36.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6039999999999993e-06 2024-09-16T14:49:36.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928635+0000", 2024-09-16T14:49:36.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068533129999999998 2024-09-16T14:49:36.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928662+0000", 2024-09-16T14:49:36.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7089000000000001e-05 2024-09-16T14:49:36.391 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.391 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.391 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.391 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:36.391 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:36.391 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.437561565000003, 2024-09-16T14:49:36.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064783530000000001, 2024-09-16T14:49:36.391 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.391 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.391 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:36.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:36.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922185+0000", 2024-09-16T14:49:36.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:36.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.393 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922205+0000", 2024-09-16T14:49:36.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9871e-05 2024-09-16T14:49:36.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.393 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922219+0000", 2024-09-16T14:49:36.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4266e-05 2024-09-16T14:49:36.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.393 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922229+0000", 2024-09-16T14:49:36.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5619999999999993e-06 2024-09-16T14:49:36.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922270+0000", 2024-09-16T14:49:36.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0917000000000001e-05 2024-09-16T14:49:36.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922278+0000", 2024-09-16T14:49:36.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3890000000000005e-06 2024-09-16T14:49:36.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928630+0000", 2024-09-16T14:49:36.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063524439999999996 2024-09-16T14:49:36.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928664+0000", 2024-09-16T14:49:36.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3651e-05 2024-09-16T14:49:36.395 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.395 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.395 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.395 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s5 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:49:36.396 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:36.396 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.436113161000002, 2024-09-16T14:49:36.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069615850000000002, 2024-09-16T14:49:36.396 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.396 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.396 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:36.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:36.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923633+0000", 2024-09-16T14:49:36.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:36.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.397 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.397 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923644+0000", 2024-09-16T14:49:36.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.08e-05 2024-09-16T14:49:36.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923657+0000", 2024-09-16T14:49:36.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2836999999999999e-05 2024-09-16T14:49:36.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923664+0000", 2024-09-16T14:49:36.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1470000000000004e-06 2024-09-16T14:49:36.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923885+0000", 2024-09-16T14:49:36.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022155800000000001 2024-09-16T14:49:36.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923894+0000", 2024-09-16T14:49:36.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9260000000000006e-06 2024-09-16T14:49:36.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930571+0000", 2024-09-16T14:49:36.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066770420000000002 2024-09-16T14:49:36.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930596+0000", 2024-09-16T14:49:36.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4383e-05 2024-09-16T14:49:36.400 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.400 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.400 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.400 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as3 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:36.400 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:36.401 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.435366924999997, 2024-09-16T14:49:36.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062471519999999997, 2024-09-16T14:49:36.401 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.401 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.401 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:36.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:36.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:36.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924405+0000", 2024-09-16T14:49:36.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5147000000000002e-05 2024-09-16T14:49:36.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924414+0000", 2024-09-16T14:49:36.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4810000000000006e-06 2024-09-16T14:49:36.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924421+0000", 2024-09-16T14:49:36.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0899999999999999e-06 2024-09-16T14:49:36.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924468+0000", 2024-09-16T14:49:36.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.727e-05 2024-09-16T14:49:36.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924477+0000", 2024-09-16T14:49:36.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7360000000000007e-06 2024-09-16T14:49:36.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.404 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.404 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930601+0000", 2024-09-16T14:49:36.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061242529999999996 2024-09-16T14:49:36.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930628+0000", 2024-09-16T14:49:36.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6517e-05 2024-09-16T14:49:36.405 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.405 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.405 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.405 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:36.405 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:36.405 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.435116721, 2024-09-16T14:49:36.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006019328, 2024-09-16T14:49:36.406 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.406 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.406 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:36.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:36.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924630+0000", 2024-09-16T14:49:36.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924650+0000", 2024-09-16T14:49:36.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0024e-05 2024-09-16T14:49:36.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924660+0000", 2024-09-16T14:49:36.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9219999999999992e-06 2024-09-16T14:49:36.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924668+0000", 2024-09-16T14:49:36.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6820000000000003e-06 2024-09-16T14:49:36.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924711+0000", 2024-09-16T14:49:36.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3009000000000001e-05 2024-09-16T14:49:36.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924720+0000", 2024-09-16T14:49:36.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9290000000000004e-06 2024-09-16T14:49:36.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930634+0000", 2024-09-16T14:49:36.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059144810000000001 2024-09-16T14:49:36.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930650+0000", 2024-09-16T14:49:36.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5688999999999998e-05 2024-09-16T14:49:36.410 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.410 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.411 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.411 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.8s6 51/27 ECSubWrite(tid=173, reqid=osd.11.0:173, at_version=51'54, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:36.411 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.881283+0000", 2024-09-16T14:49:36.411 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.478464633000002, 2024-09-16T14:49:36.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057129700000000004, 2024-09-16T14:49:36.411 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.411 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.411 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881283+0000", 2024-09-16T14:49:36.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881283+0000", 2024-09-16T14:49:36.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881282+0000", 2024-09-16T14:49:36.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881333+0000", 2024-09-16T14:49:36.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0413e-05 2024-09-16T14:49:36.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881345+0000", 2024-09-16T14:49:36.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1933000000000001e-05 2024-09-16T14:49:36.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881353+0000", 2024-09-16T14:49:36.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0469999999999994e-06 2024-09-16T14:49:36.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882512+0000", 2024-09-16T14:49:36.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001159806 2024-09-16T14:49:36.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882521+0000", 2024-09-16T14:49:36.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9639999999999992e-06 2024-09-16T14:49:36.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.886958+0000", 2024-09-16T14:49:36.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044365170000000001 2024-09-16T14:49:36.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.886996+0000", 2024-09-16T14:49:36.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7808999999999997e-05 2024-09-16T14:49:36.415 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.415 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.415 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.415 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:36.415 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:36.415 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.426747601999999, 2024-09-16T14:49:36.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062137989999999999, 2024-09-16T14:49:36.416 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.416 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.416 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:36.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:36.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.932999+0000", 2024-09-16T14:49:36.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:36.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933022+0000", 2024-09-16T14:49:36.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3518000000000001e-05 2024-09-16T14:49:36.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933035+0000", 2024-09-16T14:49:36.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2565000000000001e-05 2024-09-16T14:49:36.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933044+0000", 2024-09-16T14:49:36.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3009999999999995e-06 2024-09-16T14:49:36.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934016+0000", 2024-09-16T14:49:36.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00097159700000000004 2024-09-16T14:49:36.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934022+0000", 2024-09-16T14:49:36.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4869999999999999e-06 2024-09-16T14:49:36.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939202+0000", 2024-09-16T14:49:36.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051795820000000003 2024-09-16T14:49:36.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939214+0000", 2024-09-16T14:49:36.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1663e-05 2024-09-16T14:49:36.421 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.421 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.421 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.421 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.cs3 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:36.421 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:36.421 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.426281067000001, 2024-09-16T14:49:36.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057306709999999997, 2024-09-16T14:49:36.421 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.421 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.421 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:36.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:36.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.422 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.422 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.422 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933465+0000", 2024-09-16T14:49:36.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:36.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933489+0000", 2024-09-16T14:49:36.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3940000000000001e-05 2024-09-16T14:49:36.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933501+0000", 2024-09-16T14:49:36.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1932e-05 2024-09-16T14:49:36.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933510+0000", 2024-09-16T14:49:36.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5820000000000002e-06 2024-09-16T14:49:36.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936680+0000", 2024-09-16T14:49:36.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003169893 2024-09-16T14:49:36.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936689+0000", 2024-09-16T14:49:36.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5810000000000003e-06 2024-09-16T14:49:36.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939176+0000", 2024-09-16T14:49:36.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002486238 2024-09-16T14:49:36.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939197+0000", 2024-09-16T14:49:36.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1325e-05 2024-09-16T14:49:36.425 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.425 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.425 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.426 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.9s7 62/27 ECSubWrite(tid=55, reqid=osd.8.0:55, at_version=62'40, trim_to=0'0, roll_forward_to=61'38))", 2024-09-16T14:49:36.426 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:08.938874+0000", 2024-09-16T14:49:36.426 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.420872964000001, 2024-09-16T14:49:36.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057068529999999996, 2024-09-16T14:49:36.426 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.426 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.426 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.938874+0000", 2024-09-16T14:49:36.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.938874+0000", 2024-09-16T14:49:36.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.938874+0000", 2024-09-16T14:49:36.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.938901+0000", 2024-09-16T14:49:36.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7032000000000001e-05 2024-09-16T14:49:36.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.938928+0000", 2024-09-16T14:49:36.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7487e-05 2024-09-16T14:49:36.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.938943+0000", 2024-09-16T14:49:36.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4188999999999999e-05 2024-09-16T14:49:36.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.939120+0000", 2024-09-16T14:49:36.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017783499999999999 2024-09-16T14:49:36.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.939133+0000", 2024-09-16T14:49:36.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2994e-05 2024-09-16T14:49:36.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.944549+0000", 2024-09-16T14:49:36.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054160670000000001 2024-09-16T14:49:36.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.430 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.430 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.944581+0000", 2024-09-16T14:49:36.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.1761000000000003e-05 2024-09-16T14:49:36.430 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.430 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.430 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.430 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 62/27 ECSubWrite(tid=244, reqid=osd.7.0:244, at_version=62'27, trim_to=0'0, roll_forward_to=61'25))", 2024-09-16T14:49:36.430 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:36.430 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.41691806, 2024-09-16T14:49:36.430 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007111081, 2024-09-16T14:49:36.430 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.431 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.431 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:36.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.431 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.431 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:36.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:36.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942855+0000", 2024-09-16T14:49:36.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5999999999999998e-05 2024-09-16T14:49:36.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942867+0000", 2024-09-16T14:49:36.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2481e-05 2024-09-16T14:49:36.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942879+0000", 2024-09-16T14:49:36.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1838000000000001e-05 2024-09-16T14:49:36.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942938+0000", 2024-09-16T14:49:36.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8282000000000002e-05 2024-09-16T14:49:36.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942948+0000", 2024-09-16T14:49:36.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0497e-05 2024-09-16T14:49:36.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949905+0000", 2024-09-16T14:49:36.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069572940000000002 2024-09-16T14:49:36.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949940+0000", 2024-09-16T14:49:36.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4965999999999997e-05 2024-09-16T14:49:36.434 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.435 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.435 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.435 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 62/27 ECSubWrite(tid=234, reqid=osd.11.0:234, at_version=62'39, trim_to=0'0, roll_forward_to=61'37))", 2024-09-16T14:49:36.435 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:36.435 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.416484575999998, 2024-09-16T14:49:36.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066952440000000004, 2024-09-16T14:49:36.435 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.435 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.435 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:36.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:36.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943262+0000", 2024-09-16T14:49:36.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943285+0000", 2024-09-16T14:49:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2650000000000002e-05 2024-09-16T14:49:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943301+0000", 2024-09-16T14:49:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5398000000000001e-05 2024-09-16T14:49:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943309+0000", 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9269999999999994e-06 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943373+0000", 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3194999999999994e-05 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943383+0000", 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0778000000000001e-05 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.438 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949934+0000", 2024-09-16T14:49:36.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065509790000000002 2024-09-16T14:49:36.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949958+0000", 2024-09-16T14:49:36.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3598999999999999e-05 2024-09-16T14:49:36.439 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.439 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.439 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.439 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 76/27 ECSubWrite(tid=303, reqid=client.4500.0:699, at_version=76'98, trim_to=0'0, roll_forward_to=76'95))", 2024-09-16T14:49:36.440 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:36.440 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.449714030999999, 2024-09-16T14:49:36.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006028032, 2024-09-16T14:49:36.440 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.440 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.441 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:36.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:36.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910032+0000", 2024-09-16T14:49:36.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:36.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910053+0000", 2024-09-16T14:49:36.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0665e-05 2024-09-16T14:49:36.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910067+0000", 2024-09-16T14:49:36.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4416000000000001e-05 2024-09-16T14:49:36.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910077+0000", 2024-09-16T14:49:36.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4509999999999998e-06 2024-09-16T14:49:36.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.444 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910367+0000", 2024-09-16T14:49:36.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00029006799999999999 2024-09-16T14:49:36.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.444 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.444 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.444 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910378+0000", 2024-09-16T14:49:36.444 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0628000000000001e-05 2024-09-16T14:49:36.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.916036+0000", 2024-09-16T14:49:36.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0056580270000000004 2024-09-16T14:49:36.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.445 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.445 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.445 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.916061+0000", 2024-09-16T14:49:36.445 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5658000000000001e-05 2024-09-16T14:49:36.445 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.445 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.445 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.446 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 77/27 ECSubWrite(tid=277, reqid=osd.11.0:277, at_version=77'54, trim_to=0'0, roll_forward_to=75'52))", 2024-09-16T14:49:36.446 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:36.446 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.449226662999999, 2024-09-16T14:49:36.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064553950000000001, 2024-09-16T14:49:36.446 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.446 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.446 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.446 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.446 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.446 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:36.446 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910520+0000", 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910554+0000", 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3111999999999997e-05 2024-09-16T14:49:36.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910578+0000", 2024-09-16T14:49:36.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4757999999999999e-05 2024-09-16T14:49:36.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910590+0000", 2024-09-16T14:49:36.448 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2156e-05 2024-09-16T14:49:36.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.448 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.448 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.448 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910646+0000", 2024-09-16T14:49:36.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5979000000000001e-05 2024-09-16T14:49:36.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910656+0000", 2024-09-16T14:49:36.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9410000000000002e-06 2024-09-16T14:49:36.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.449 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916943+0000", 2024-09-16T14:49:36.449 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062870260000000002 2024-09-16T14:49:36.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.449 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.449 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916976+0000", 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2641999999999997e-05 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 77/65 ECSubWrite(tid=309, reqid=client.4500.0:710, at_version=77'97, trim_to=0'0, roll_forward_to=73'95))", 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.911636+0000", 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.448111011, 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054858010000000002, 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.911636+0000", 2024-09-16T14:49:36.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.911636+0000", 2024-09-16T14:49:36.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.911635+0000", 2024-09-16T14:49:36.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:36.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.912338+0000", 2024-09-16T14:49:36.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00070288100000000004 2024-09-16T14:49:36.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.912358+0000", 2024-09-16T14:49:36.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9704e-05 2024-09-16T14:49:36.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.912368+0000", 2024-09-16T14:49:36.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9280000000000005e-06 2024-09-16T14:49:36.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.912447+0000", 2024-09-16T14:49:36.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9617999999999996e-05 2024-09-16T14:49:36.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.912458+0000", 2024-09-16T14:49:36.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0409e-05 2024-09-16T14:49:36.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917096+0000", 2024-09-16T14:49:36.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046382410000000004 2024-09-16T14:49:36.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917122+0000", 2024-09-16T14:49:36.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5961000000000001e-05 2024-09-16T14:49:36.454 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.454 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.454 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.455 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as3 77/72 ECSubWrite(tid=14, reqid=client.4500.0:702, at_version=77'33, trim_to=0'0, roll_forward_to=77'32))", 2024-09-16T14:49:36.455 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.917638+0000", 2024-09-16T14:49:36.455 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.442109228, 2024-09-16T14:49:36.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055533229999999998, 2024-09-16T14:49:36.455 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.455 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.455 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917638+0000", 2024-09-16T14:49:36.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917638+0000", 2024-09-16T14:49:36.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917637+0000", 2024-09-16T14:49:36.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:36.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918467+0000", 2024-09-16T14:49:36.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083006699999999998 2024-09-16T14:49:36.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918484+0000", 2024-09-16T14:49:36.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6569e-05 2024-09-16T14:49:36.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918493+0000", 2024-09-16T14:49:36.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.115e-06 2024-09-16T14:49:36.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918547+0000", 2024-09-16T14:49:36.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4063000000000001e-05 2024-09-16T14:49:36.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918558+0000", 2024-09-16T14:49:36.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1428e-05 2024-09-16T14:49:36.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923159+0000", 2024-09-16T14:49:36.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046010590000000002 2024-09-16T14:49:36.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923191+0000", 2024-09-16T14:49:36.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2107999999999998e-05 2024-09-16T14:49:36.459 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.459 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.459 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.459 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.459 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.459 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:36.461 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:49:36.461 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:36.459+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:36.562 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:36.560+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:36.564 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:36.564 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:36.564 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:36.564 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:36.573 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:49:36.662 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:36.661+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:36.673 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:36.673 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:36.673 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:36.673 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:36.674 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:36.682 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:49:36.766 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:36.765+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:36.825 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:36.826 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:36.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:36.826 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:36.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.826 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:229 3.es0 3:7ecfd044:::smithi04040151-35:head [copy-from ver 4] snapc 4=[4,2] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:49:36.826 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:36.826 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.919429612000002, 2024-09-16T14:49:36.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016506461, 2024-09-16T14:49:36.826 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.826 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:36.826 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.827 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.827 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.827 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 229 2024-09-16T14:49:36.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.827 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:36.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:36.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904168+0000", 2024-09-16T14:49:36.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:36.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904187+0000", 2024-09-16T14:49:36.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9449999999999998e-05 2024-09-16T14:49:36.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904206+0000", 2024-09-16T14:49:36.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8698e-05 2024-09-16T14:49:36.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904215+0000", 2024-09-16T14:49:36.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:49:36.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904276+0000", 2024-09-16T14:49:36.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1130999999999995e-05 2024-09-16T14:49:36.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:36.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904284+0000", 2024-09-16T14:49:36.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0609999999999996e-06 2024-09-16T14:49:36.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906272+0000", 2024-09-16T14:49:36.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019884210000000002 2024-09-16T14:49:36.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906391+0000", 2024-09-16T14:49:36.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000118843 2024-09-16T14:49:36.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.912595+0000", 2024-09-16T14:49:36.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062038190000000002 2024-09-16T14:49:36.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917584+0000", 2024-09-16T14:49:36.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049889069999999999 2024-09-16T14:49:36.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:36.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920592+0000", 2024-09-16T14:49:36.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030081639999999998 2024-09-16T14:49:36.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920676+0000", 2024-09-16T14:49:36.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3974999999999995e-05 2024-09-16T14:49:36.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.833 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.833 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:230 3.es0 3:7ecfd044:::smithi04040151-35:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:49:36.833 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:49:36.834 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 55.919368128999999, 2024-09-16T14:49:36.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0096834569999999995, 2024-09-16T14:49:36.834 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.834 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:36.834 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.834 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.834 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.834 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 230 2024-09-16T14:49:36.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.834 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:49:36.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:49:36.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904231+0000", 2024-09-16T14:49:36.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904235+0000", 2024-09-16T14:49:36.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4059999999999998e-06 2024-09-16T14:49:36.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904244+0000", 2024-09-16T14:49:36.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3139999999999993e-06 2024-09-16T14:49:36.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904250+0000", 2024-09-16T14:49:36.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0719999999999996e-06 2024-09-16T14:49:36.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904338+0000", 2024-09-16T14:49:36.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7923000000000005e-05 2024-09-16T14:49:36.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:36.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904344+0000", 2024-09-16T14:49:36.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8669999999999999e-06 2024-09-16T14:49:36.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906575+0000", 2024-09-16T14:49:36.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002230994 2024-09-16T14:49:36.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:36.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906632+0000", 2024-09-16T14:49:36.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7357e-05 2024-09-16T14:49:36.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913716+0000", 2024-09-16T14:49:36.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070835070000000002 2024-09-16T14:49:36.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913822+0000", 2024-09-16T14:49:36.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010566299999999999 2024-09-16T14:49:36.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913915+0000", 2024-09-16T14:49:36.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3018999999999998e-05 2024-09-16T14:49:36.839 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.840 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.840 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.840 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs4 37/27 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:36.840 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:49:36.840 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.902831317999997, 2024-09-16T14:49:36.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0098104279999999995, 2024-09-16T14:49:36.840 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.840 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.840 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:49:36.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:49:36.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920768+0000", 2024-09-16T14:49:36.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920785+0000", 2024-09-16T14:49:36.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7218e-05 2024-09-16T14:49:36.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920793+0000", 2024-09-16T14:49:36.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9300000000000003e-06 2024-09-16T14:49:36.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920800+0000", 2024-09-16T14:49:36.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1860000000000003e-06 2024-09-16T14:49:36.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923692+0000", 2024-09-16T14:49:36.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028921329999999999 2024-09-16T14:49:36.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923701+0000", 2024-09-16T14:49:36.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6109999999999994e-06 2024-09-16T14:49:36.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930554+0000", 2024-09-16T14:49:36.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068534019999999998 2024-09-16T14:49:36.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930578+0000", 2024-09-16T14:49:36.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4232000000000001e-05 2024-09-16T14:49:36.845 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.845 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.845 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.846 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:298 3.es0 3:7ecfd044:::smithi04040151-35:head [append 0~720896] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:36.846 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:36.846 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.894765796999998, 2024-09-16T14:49:36.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014187808, 2024-09-16T14:49:36.846 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.846 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:36.846 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.846 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.846 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.846 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 298 2024-09-16T14:49:36.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.846 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:36.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:36.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928832+0000", 2024-09-16T14:49:36.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:36.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929746+0000", 2024-09-16T14:49:36.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00091434199999999995 2024-09-16T14:49:36.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929760+0000", 2024-09-16T14:49:36.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3519e-05 2024-09-16T14:49:36.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929767+0000", 2024-09-16T14:49:36.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3180000000000001e-06 2024-09-16T14:49:36.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929849+0000", 2024-09-16T14:49:36.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2020000000000004e-05 2024-09-16T14:49:36.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:36.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929854+0000", 2024-09-16T14:49:36.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1209999999999998e-06 2024-09-16T14:49:36.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931714+0000", 2024-09-16T14:49:36.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001859989 2024-09-16T14:49:36.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931845+0000", 2024-09-16T14:49:36.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130901 2024-09-16T14:49:36.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933089+0000", 2024-09-16T14:49:36.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001243535 2024-09-16T14:49:36.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936370+0000", 2024-09-16T14:49:36.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032817660000000002 2024-09-16T14:49:36.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:36.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942945+0000", 2024-09-16T14:49:36.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065741560000000003 2024-09-16T14:49:36.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943021+0000", 2024-09-16T14:49:36.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6734000000000001e-05 2024-09-16T14:49:36.852 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.852 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.852 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.852 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:299 3.es0 3:7ecfd044:::smithi04040151-35:head [append 0~376832] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:36.852 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:36.852 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.892475996999998, 2024-09-16T14:49:36.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012149627, 2024-09-16T14:49:36.852 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.852 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:36.852 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.853 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 299 2024-09-16T14:49:36.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.853 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:36.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:36.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931122+0000", 2024-09-16T14:49:36.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:36.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931722+0000", 2024-09-16T14:49:36.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059944100000000004 2024-09-16T14:49:36.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931732+0000", 2024-09-16T14:49:36.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9369999999999999e-06 2024-09-16T14:49:36.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931739+0000", 2024-09-16T14:49:36.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2819999999999999e-06 2024-09-16T14:49:36.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933835+0000", 2024-09-16T14:49:36.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020961140000000001 2024-09-16T14:49:36.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933929+0000", 2024-09-16T14:49:36.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3683000000000004e-05 2024-09-16T14:49:36.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.934917+0000", 2024-09-16T14:49:36.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00098803200000000002 2024-09-16T14:49:36.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.939558+0000", 2024-09-16T14:49:36.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00464175 2024-09-16T14:49:36.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:36.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943205+0000", 2024-09-16T14:49:36.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036465820000000002 2024-09-16T14:49:36.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943273+0000", 2024-09-16T14:49:36.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7917999999999996e-05 2024-09-16T14:49:36.857 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.857 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.857 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.858 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:300 3.es0 3:7ecfd044:::smithi04040151-35:head [setxattr _header (56) in=7b] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:36.858 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:36.858 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.891849903000001, 2024-09-16T14:49:36.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011740911, 2024-09-16T14:49:36.858 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.858 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:36.858 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.858 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.858 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.858 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 300 2024-09-16T14:49:36.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.859 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:36.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:36.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:36.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:36.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931753+0000", 2024-09-16T14:49:36.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0380000000000001e-06 2024-09-16T14:49:36.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931759+0000", 2024-09-16T14:49:36.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3430000000000001e-06 2024-09-16T14:49:36.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931765+0000", 2024-09-16T14:49:36.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-16T14:49:36.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.935623+0000", 2024-09-16T14:49:36.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038583049999999998 2024-09-16T14:49:36.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.935717+0000", 2024-09-16T14:49:36.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4587999999999996e-05 2024-09-16T14:49:36.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936150+0000", 2024-09-16T14:49:36.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043270599999999999 2024-09-16T14:49:36.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.939576+0000", 2024-09-16T14:49:36.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034257620000000002 2024-09-16T14:49:36.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:36.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943399+0000", 2024-09-16T14:49:36.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003823145 2024-09-16T14:49:36.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943490+0000", 2024-09-16T14:49:36.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1241000000000003e-05 2024-09-16T14:49:36.863 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.863 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.863 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.864 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:301 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:36.864 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:36.864 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 51.891828603, 2024-09-16T14:49:36.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014215955000000001, 2024-09-16T14:49:36.864 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.864 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:36.864 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.864 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.864 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.864 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 301 2024-09-16T14:49:36.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.864 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:36.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:36.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931770+0000", 2024-09-16T14:49:36.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931775+0000", 2024-09-16T14:49:36.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4530000000000004e-06 2024-09-16T14:49:36.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931782+0000", 2024-09-16T14:49:36.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0550000000000002e-06 2024-09-16T14:49:36.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931787+0000", 2024-09-16T14:49:36.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2310000000000004e-06 2024-09-16T14:49:36.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936401+0000", 2024-09-16T14:49:36.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046143419999999996 2024-09-16T14:49:36.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:36.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936482+0000", 2024-09-16T14:49:36.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0407000000000005e-05 2024-09-16T14:49:36.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943851+0000", 2024-09-16T14:49:36.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073695000000000002 2024-09-16T14:49:36.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943951+0000", 2024-09-16T14:49:36.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9327000000000006e-05 2024-09-16T14:49:36.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.945987+0000", 2024-09-16T14:49:36.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002036057 2024-09-16T14:49:36.869 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.869 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.869 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.869 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:369 3.8s0 3:1f96bdfc:::smithi04040151-37:head [append 0~475136] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:36.869 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:36.869 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 47.865886609999997, 2024-09-16T14:49:36.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010979078999999999, 2024-09-16T14:49:36.869 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.869 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:36.869 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.869 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.870 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.870 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 369 2024-09-16T14:49:36.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.870 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:36.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:36.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957711+0000", 2024-09-16T14:49:36.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:36.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958323+0000", 2024-09-16T14:49:36.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000612439 2024-09-16T14:49:36.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958339+0000", 2024-09-16T14:49:36.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5985999999999999e-05 2024-09-16T14:49:36.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958348+0000", 2024-09-16T14:49:36.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8699999999999998e-06 2024-09-16T14:49:36.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958393+0000", 2024-09-16T14:49:36.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4981000000000002e-05 2024-09-16T14:49:36.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958478+0000", 2024-09-16T14:49:36.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4845999999999994e-05 2024-09-16T14:49:36.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.959659+0000", 2024-09-16T14:49:36.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011808999999999999 2024-09-16T14:49:36.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962687+0000", 2024-09-16T14:49:36.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030278330000000002 2024-09-16T14:49:36.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:36.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968651+0000", 2024-09-16T14:49:36.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005964764 2024-09-16T14:49:36.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968692+0000", 2024-09-16T14:49:36.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0374999999999999e-05 2024-09-16T14:49:36.875 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.875 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.875 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.875 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:372 3.8s0 3:1f96bdfc:::smithi04040151-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:36.875 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:49:36.875 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 47.863158274, 2024-09-16T14:49:36.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010116534999999999, 2024-09-16T14:49:36.876 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.876 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:36.876 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.876 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.876 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.876 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 372 2024-09-16T14:49:36.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.876 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:49:36.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:49:36.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960441+0000", 2024-09-16T14:49:36.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960445+0000", 2024-09-16T14:49:36.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0579999999999999e-06 2024-09-16T14:49:36.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960452+0000", 2024-09-16T14:49:36.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5360000000000004e-06 2024-09-16T14:49:36.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.960458+0000", 2024-09-16T14:49:36.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3260000000000003e-06 2024-09-16T14:49:36.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962902+0000", 2024-09-16T14:49:36.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002444313 2024-09-16T14:49:36.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:36.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962973+0000", 2024-09-16T14:49:36.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1179000000000004e-05 2024-09-16T14:49:36.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.969538+0000", 2024-09-16T14:49:36.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006565169 2024-09-16T14:49:36.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.969635+0000", 2024-09-16T14:49:36.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6427000000000004e-05 2024-09-16T14:49:36.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.970558+0000", 2024-09-16T14:49:36.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00092299600000000004 2024-09-16T14:49:36.880 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.880 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.880 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.880 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:387 3.8s0 3:1f96bdfc:::smithi04040151-37:head [read 0~2686976,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:36.881 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:49:36.881 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.95913625, 2024-09-16T14:49:36.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010154198999999999, 2024-09-16T14:49:36.881 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.881 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:36.881 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.881 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.881 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.881 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 387 2024-09-16T14:49:36.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.881 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:49:36.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:49:36.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:49:36.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864465+0000", 2024-09-16T14:49:36.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.774e-06 2024-09-16T14:49:36.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864472+0000", 2024-09-16T14:49:36.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0040000000000002e-06 2024-09-16T14:49:36.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864477+0000", 2024-09-16T14:49:36.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0779999999999997e-06 2024-09-16T14:49:36.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.866679+0000", 2024-09-16T14:49:36.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00220228 2024-09-16T14:49:36.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.866770+0000", 2024-09-16T14:49:36.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0952999999999995e-05 2024-09-16T14:49:36.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874617+0000", 2024-09-16T14:49:36.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007847527 2024-09-16T14:49:36.885 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.885 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.885 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.885 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:398 3.as0 3:572aff57:::smithi04040151-20:head [append 0~507904] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:36.885 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:36.885 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.944521844, 2024-09-16T14:49:36.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010447988, 2024-09-16T14:49:36.885 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.885 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:36.885 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.886 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 398 2024-09-16T14:49:36.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.886 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:36.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:36.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879076+0000", 2024-09-16T14:49:36.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:36.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881851+0000", 2024-09-16T14:49:36.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0027753420000000001 2024-09-16T14:49:36.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881863+0000", 2024-09-16T14:49:36.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2203e-05 2024-09-16T14:49:36.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881871+0000", 2024-09-16T14:49:36.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4059999999999999e-06 2024-09-16T14:49:36.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881924+0000", 2024-09-16T14:49:36.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2932000000000001e-05 2024-09-16T14:49:36.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882118+0000", 2024-09-16T14:49:36.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019413500000000001 2024-09-16T14:49:36.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884114+0000", 2024-09-16T14:49:36.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001996154 2024-09-16T14:49:36.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889295+0000", 2024-09-16T14:49:36.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051809899999999999 2024-09-16T14:49:36.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:36.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889480+0000", 2024-09-16T14:49:36.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018519700000000001 2024-09-16T14:49:36.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889525+0000", 2024-09-16T14:49:36.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5399999999999999e-05 2024-09-16T14:49:36.890 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.890 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.890 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.891 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:400 3.as0 3:572aff57:::smithi04040151-20:head [append 0~262144] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:36.891 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:36.891 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.940012813999999, 2024-09-16T14:49:36.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010619327, 2024-09-16T14:49:36.891 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.891 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:36.891 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.891 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.892 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.892 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 400 2024-09-16T14:49:36.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.892 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:36.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:36.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883585+0000", 2024-09-16T14:49:36.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:36.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883868+0000", 2024-09-16T14:49:36.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028385599999999997 2024-09-16T14:49:36.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883880+0000", 2024-09-16T14:49:36.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1181e-05 2024-09-16T14:49:36.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883888+0000", 2024-09-16T14:49:36.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.926e-06 2024-09-16T14:49:36.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887739+0000", 2024-09-16T14:49:36.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003851658 2024-09-16T14:49:36.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887849+0000", 2024-09-16T14:49:36.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109616 2024-09-16T14:49:36.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.888684+0000", 2024-09-16T14:49:36.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083518000000000004 2024-09-16T14:49:36.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894010+0000", 2024-09-16T14:49:36.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053260670000000003 2024-09-16T14:49:36.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:36.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894143+0000", 2024-09-16T14:49:36.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000133077 2024-09-16T14:49:36.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894206+0000", 2024-09-16T14:49:36.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2658999999999998e-05 2024-09-16T14:49:36.896 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.896 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.896 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.897 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:401 3.as0 3:572aff57:::smithi04040151-20:head [setxattr _header (57) in=7b] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:36.897 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:36.897 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.93970289, 2024-09-16T14:49:36.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010524774000000001, 2024-09-16T14:49:36.897 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.897 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:36.897 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.897 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.897 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.897 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 401 2024-09-16T14:49:36.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.898 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:36.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:36.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:36.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883900+0000", 2024-09-16T14:49:36.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1400000000000002e-06 2024-09-16T14:49:36.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883908+0000", 2024-09-16T14:49:36.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2689999999999999e-06 2024-09-16T14:49:36.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883913+0000", 2024-09-16T14:49:36.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-16T14:49:36.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889560+0000", 2024-09-16T14:49:36.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005647165 2024-09-16T14:49:36.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889627+0000", 2024-09-16T14:49:36.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6951000000000003e-05 2024-09-16T14:49:36.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889898+0000", 2024-09-16T14:49:36.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270709 2024-09-16T14:49:36.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894012+0000", 2024-09-16T14:49:36.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041137810000000004 2024-09-16T14:49:36.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:36.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894344+0000", 2024-09-16T14:49:36.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000332121 2024-09-16T14:49:36.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894421+0000", 2024-09-16T14:49:36.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7406999999999999e-05 2024-09-16T14:49:36.902 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.902 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.902 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.902 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:402 3.as0 3:572aff57:::smithi04040151-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:36.903 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:36.903 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.939681014000001, 2024-09-16T14:49:36.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011864743000000001, 2024-09-16T14:49:36.903 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.903 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:36.903 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.903 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.903 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.903 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 402 2024-09-16T14:49:36.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.903 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:36.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:36.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:36.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883921+0000", 2024-09-16T14:49:36.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7520000000000001e-06 2024-09-16T14:49:36.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883927+0000", 2024-09-16T14:49:36.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4679999999999997e-06 2024-09-16T14:49:36.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883932+0000", 2024-09-16T14:49:36.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1440000000000002e-06 2024-09-16T14:49:36.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890477+0000", 2024-09-16T14:49:36.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065446430000000002 2024-09-16T14:49:36.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:36.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890586+0000", 2024-09-16T14:49:36.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010909 2024-09-16T14:49:36.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894750+0000", 2024-09-16T14:49:36.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00416359 2024-09-16T14:49:36.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894834+0000", 2024-09-16T14:49:36.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4596000000000001e-05 2024-09-16T14:49:36.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895783+0000", 2024-09-16T14:49:36.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00094892000000000002 2024-09-16T14:49:36.908 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.908 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.908 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.908 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:443 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:36.908 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:36.908 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.947470441, 2024-09-16T14:49:36.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.013766494000000001, 2024-09-16T14:49:36.908 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.909 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:36.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.909 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.911 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.912 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 443 2024-09-16T14:49:36.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.912 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:36.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:36.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876127+0000", 2024-09-16T14:49:36.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:36.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876138+0000", 2024-09-16T14:49:36.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1344e-05 2024-09-16T14:49:36.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876159+0000", 2024-09-16T14:49:36.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1157000000000001e-05 2024-09-16T14:49:36.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876170+0000", 2024-09-16T14:49:36.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0651e-05 2024-09-16T14:49:36.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876234+0000", 2024-09-16T14:49:36.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4114000000000003e-05 2024-09-16T14:49:36.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:36.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876243+0000", 2024-09-16T14:49:36.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0049999999999993e-06 2024-09-16T14:49:36.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879445+0000", 2024-09-16T14:49:36.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032019269999999998 2024-09-16T14:49:36.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879550+0000", 2024-09-16T14:49:36.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010536500000000001 2024-09-16T14:49:36.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889895+0000", 2024-09-16T14:49:36.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010345043 2024-09-16T14:49:36.916 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.916 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.916 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.917 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:444 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:36.917 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:36.917 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.947380733999999, 2024-09-16T14:49:36.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014716241, 2024-09-16T14:49:36.917 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.917 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:36.917 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.917 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.917 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.917 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 444 2024-09-16T14:49:36.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.917 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:36.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:36.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876218+0000", 2024-09-16T14:49:36.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:36.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876224+0000", 2024-09-16T14:49:36.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:49:36.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876235+0000", 2024-09-16T14:49:36.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.025e-05 2024-09-16T14:49:36.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876242+0000", 2024-09-16T14:49:36.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8790000000000003e-06 2024-09-16T14:49:36.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876315+0000", 2024-09-16T14:49:36.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2260000000000003e-05 2024-09-16T14:49:36.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:36.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876320+0000", 2024-09-16T14:49:36.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5690000000000004e-06 2024-09-16T14:49:36.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880078+0000", 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0037576430000000002 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880168+0000", 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0353999999999996e-05 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.890935+0000", 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010766556 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.923 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:445 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:36.923 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:36.923 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.947336751999998, 2024-09-16T14:49:36.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016529716999999999, 2024-09-16T14:49:36.923 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.923 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:36.923 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.923 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.923 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.923 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 445 2024-09-16T14:49:36.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.923 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:36.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:36.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876262+0000", 2024-09-16T14:49:36.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:36.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876269+0000", 2024-09-16T14:49:36.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9469999999999997e-06 2024-09-16T14:49:36.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876276+0000", 2024-09-16T14:49:36.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5950000000000004e-06 2024-09-16T14:49:36.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876281+0000", 2024-09-16T14:49:36.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:49:36.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876343+0000", 2024-09-16T14:49:36.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1685000000000003e-05 2024-09-16T14:49:36.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:36.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876348+0000", 2024-09-16T14:49:36.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7269999999999998e-06 2024-09-16T14:49:36.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881091+0000", 2024-09-16T14:49:36.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047434599999999997 2024-09-16T14:49:36.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881181+0000", 2024-09-16T14:49:36.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9838000000000003e-05 2024-09-16T14:49:36.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.892792+0000", 2024-09-16T14:49:36.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011611316 2024-09-16T14:49:36.928 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.928 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.928 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.928 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s1 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:36.928 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:36.928 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.890526502, 2024-09-16T14:49:36.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011783765999999999, 2024-09-16T14:49:36.928 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.928 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:36.928 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:36.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:36.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933072+0000", 2024-09-16T14:49:36.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:36.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933092+0000", 2024-09-16T14:49:36.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0000000000000002e-05 2024-09-16T14:49:36.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933119+0000", 2024-09-16T14:49:36.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6877e-05 2024-09-16T14:49:36.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933127+0000", 2024-09-16T14:49:36.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4600000000000003e-06 2024-09-16T14:49:36.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.937087+0000", 2024-09-16T14:49:36.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0039599129999999998 2024-09-16T14:49:36.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.937096+0000", 2024-09-16T14:49:36.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8910000000000001e-06 2024-09-16T14:49:36.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944834+0000", 2024-09-16T14:49:36.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077376809999999997 2024-09-16T14:49:36.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944857+0000", 2024-09-16T14:49:36.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2785999999999998e-05 2024-09-16T14:49:36.932 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.932 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.932 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.932 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:591 3.0s0 3:060eef9f:::smithi04040151-16:head [copy-from ver 27] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:36.932 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:36.933 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.847727494000001, 2024-09-16T14:49:36.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0279670919999999, 2024-09-16T14:49:36.933 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.933 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:36.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.933 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.933 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 591 2024-09-16T14:49:36.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.933 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:36.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:36.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975870+0000", 2024-09-16T14:49:36.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:36.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975878+0000", 2024-09-16T14:49:36.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2619999999999998e-06 2024-09-16T14:49:36.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975902+0000", 2024-09-16T14:49:36.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4036000000000002e-05 2024-09-16T14:49:36.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975912+0000", 2024-09-16T14:49:36.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-16T14:49:36.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975986+0000", 2024-09-16T14:49:36.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3548999999999997e-05 2024-09-16T14:49:36.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:36.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975990+0000", 2024-09-16T14:49:36.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7670000000000001e-06 2024-09-16T14:49:36.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.975658+0000", 2024-09-16T14:49:36.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99966882800000001 2024-09-16T14:49:36.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for missing object", 2024-09-16T14:49:36.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.975885+0000", 2024-09-16T14:49:36.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000226682 2024-09-16T14:49:36.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.982217+0000", 2024-09-16T14:49:36.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063321530000000001 2024-09-16T14:49:36.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.982350+0000", 2024-09-16T14:49:36.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013316700000000001 2024-09-16T14:49:36.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:36.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.999798+0000", 2024-09-16T14:49:36.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017447608 2024-09-16T14:49:36.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:36.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001884+0000", 2024-09-16T14:49:36.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020859889999999999 2024-09-16T14:49:36.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:36.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.003629+0000", 2024-09-16T14:49:36.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001745106 2024-09-16T14:49:36.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.003839+0000", 2024-09-16T14:49:36.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020980099999999999 2024-09-16T14:49:36.939 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.940 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.940 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.940 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.940 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.940 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:592 3.0s0 3:060eef9f:::smithi04040151-16:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:36.940 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:36.940 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.847670569999998, 2024-09-16T14:49:36.940 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0256284010000001, 2024-09-16T14:49:36.940 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:36.940 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:36.940 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:36.940 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:36.941 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:36.941 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 592 2024-09-16T14:49:36.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.941 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:36.941 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.941 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:36.941 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:36.941 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:36.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:36.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:36.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:36.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975928+0000", 2024-09-16T14:49:36.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:36.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.942 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.942 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:36.942 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975936+0000", 2024-09-16T14:49:36.942 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6090000000000003e-06 2024-09-16T14:49:36.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:36.943 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975943+0000", 2024-09-16T14:49:36.943 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:36.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.943 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.943 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:36.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975949+0000", 2024-09-16T14:49:36.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8300000000000001e-06 2024-09-16T14:49:36.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.976030+0000", 2024-09-16T14:49:36.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1267000000000006e-05 2024-09-16T14:49:36.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.944 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.944 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:36.944 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.976033+0000", 2024-09-16T14:49:36.944 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7590000000000002e-06 2024-09-16T14:49:36.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.977041+0000", 2024-09-16T14:49:36.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0010081639999999 2024-09-16T14:49:36.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for missing object", 2024-09-16T14:49:36.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.977086+0000", 2024-09-16T14:49:36.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4879999999999997e-05 2024-09-16T14:49:36.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.945 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.945 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.945 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.983015+0000", 2024-09-16T14:49:36.945 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059287480000000002 2024-09-16T14:49:36.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:36.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.983090+0000", 2024-09-16T14:49:36.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5081999999999997e-05 2024-09-16T14:49:36.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:36.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001193+0000", 2024-09-16T14:49:36.946 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.018102713999999999 2024-09-16T14:49:36.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.946 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.946 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:36.946 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001306+0000", 2024-09-16T14:49:36.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011330799999999999 2024-09-16T14:49:36.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:36.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:36.947 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:36.947 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001557+0000", 2024-09-16T14:49:36.947 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025100900000000001 2024-09-16T14:49:36.947 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.947 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.947 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.947 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:36.947 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:36.947 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:36.949 INFO:tasks.rados.rados.0.smithi040.stdout:339: done (0 left) 2024-09-16T14:49:36.949 INFO:tasks.rados.rados.0.smithi040.stdout:340: copy_from oid 27 from oid 35 current snap is 32 2024-09-16T14:49:36.949 INFO:tasks.rados.rados.0.smithi040.stdout:341: snap_remove snap 30 2024-09-16T14:49:36.949 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:49:36.954 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:49:36.955 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:36.951+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:37.050 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:37.049+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:37.071 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:37.071 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:37.071 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:37.071 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:37.080 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:49:37.164 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:37.163+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:37.222 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:37.222 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:37.222 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:37.223 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:37.223 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:37.231 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:49:37.272 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:37.271+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:37.353 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:37.351+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:37.375 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:37.375 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:37.376 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:37.376 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:37.376 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.376 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds3 66/65 ECSubWrite(tid=256, reqid=client.4500.0:601, at_version=66'53, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:37.376 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.993987+0000", 2024-09-16T14:49:37.377 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.378720933, 2024-09-16T14:49:37.377 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036421859999999999, 2024-09-16T14:49:37.377 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.377 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.377 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.377 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.377 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.378 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.993987+0000", 2024-09-16T14:49:37.378 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.378 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.378 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.378 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.378 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.993987+0000", 2024-09-16T14:49:37.379 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.379 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.379 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.379 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.379 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.993987+0000", 2024-09-16T14:49:37.379 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.379 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.380 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.380 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.380 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994130+0000", 2024-09-16T14:49:37.380 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014385799999999999 2024-09-16T14:49:37.380 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.380 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.381 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.381 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994146+0000", 2024-09-16T14:49:37.381 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.588e-05 2024-09-16T14:49:37.381 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.381 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.381 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.382 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994166+0000", 2024-09-16T14:49:37.382 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9582999999999999e-05 2024-09-16T14:49:37.382 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.382 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.382 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.382 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994229+0000", 2024-09-16T14:49:37.382 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3495000000000001e-05 2024-09-16T14:49:37.383 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.383 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.383 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.383 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.994239+0000", 2024-09-16T14:49:37.383 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2080000000000006e-06 2024-09-16T14:49:37.383 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.383 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.384 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.384 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.997601+0000", 2024-09-16T14:49:37.384 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0033624900000000001 2024-09-16T14:49:37.384 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.384 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.384 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.384 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.997630+0000", 2024-09-16T14:49:37.384 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8530999999999998e-05 2024-09-16T14:49:37.385 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.385 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.385 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.385 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.385 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.385 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 66/65 ECSubWrite(tid=38, reqid=client.4500.0:596, at_version=66'25, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:37.385 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.009720+0000", 2024-09-16T14:49:37.386 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.362988325, 2024-09-16T14:49:37.386 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053370680000000004, 2024-09-16T14:49:37.386 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.386 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.386 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.386 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.387 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.387 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009720+0000", 2024-09-16T14:49:37.387 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.387 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.387 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.387 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.388 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009720+0000", 2024-09-16T14:49:37.388 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.388 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.388 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.388 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.388 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009719+0000", 2024-09-16T14:49:37.389 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.389 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.389 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.389 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.389 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009878+0000", 2024-09-16T14:49:37.390 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015914100000000001 2024-09-16T14:49:37.390 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.390 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.390 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.390 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009902+0000", 2024-09-16T14:49:37.390 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3164999999999999e-05 2024-09-16T14:49:37.391 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.391 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.391 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.391 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009911+0000", 2024-09-16T14:49:37.391 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9429999999999995e-06 2024-09-16T14:49:37.391 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.391 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.392 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.392 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009974+0000", 2024-09-16T14:49:37.392 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2117000000000002e-05 2024-09-16T14:49:37.392 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.392 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.392 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.392 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009985+0000", 2024-09-16T14:49:37.392 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1539000000000001e-05 2024-09-16T14:49:37.393 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.393 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.393 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.393 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.015029+0000", 2024-09-16T14:49:37.393 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050442999999999998 2024-09-16T14:49:37.393 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.393 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.393 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.394 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.015057+0000", 2024-09-16T14:49:37.394 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7606999999999998e-05 2024-09-16T14:49:37.394 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.394 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.394 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.394 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.394 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.394 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 66/65 ECSubWrite(tid=39, reqid=client.4500.0:597, at_version=66'26, trim_to=0'0, roll_forward_to=66'23))", 2024-09-16T14:49:37.395 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:37.395 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.360319018999999, 2024-09-16T14:49:37.395 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0060634030000000002, 2024-09-16T14:49:37.395 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.395 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.395 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.395 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.396 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.396 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:37.396 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.396 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.396 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.396 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.397 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:37.397 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.397 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.397 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.397 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.397 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012388+0000", 2024-09-16T14:49:37.398 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.398 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.398 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.398 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.398 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012433+0000", 2024-09-16T14:49:37.398 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4588000000000001e-05 2024-09-16T14:49:37.399 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.399 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.399 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.399 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012604+0000", 2024-09-16T14:49:37.399 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017141300000000001 2024-09-16T14:49:37.399 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.400 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.400 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.400 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012617+0000", 2024-09-16T14:49:37.400 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2792999999999999e-05 2024-09-16T14:49:37.400 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.400 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.401 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.401 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012666+0000", 2024-09-16T14:49:37.401 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9079000000000002e-05 2024-09-16T14:49:37.401 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.401 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.401 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.402 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012678+0000", 2024-09-16T14:49:37.402 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1405e-05 2024-09-16T14:49:37.402 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.402 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.402 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.402 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018433+0000", 2024-09-16T14:49:37.403 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005755146 2024-09-16T14:49:37.403 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.403 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.403 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.403 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018453+0000", 2024-09-16T14:49:37.404 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9780999999999999e-05 2024-09-16T14:49:37.404 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.404 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.405 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.405 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.405 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.405 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 66/65 ECSubWrite(tid=40, reqid=client.4500.0:598, at_version=66'27, trim_to=0'0, roll_forward_to=66'23))", 2024-09-16T14:49:37.405 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.013745+0000", 2024-09-16T14:49:37.405 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.35896339, 2024-09-16T14:49:37.406 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0047198850000000001, 2024-09-16T14:49:37.406 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.406 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.406 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.406 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.406 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.407 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013745+0000", 2024-09-16T14:49:37.407 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.407 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.407 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.407 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.407 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013745+0000", 2024-09-16T14:49:37.408 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.408 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.408 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.408 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.408 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013744+0000", 2024-09-16T14:49:37.408 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.409 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.409 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.409 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.409 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013766+0000", 2024-09-16T14:49:37.409 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1721000000000001e-05 2024-09-16T14:49:37.410 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.410 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.410 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.410 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013788+0000", 2024-09-16T14:49:37.410 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1708e-05 2024-09-16T14:49:37.410 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.411 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.411 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.411 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013798+0000", 2024-09-16T14:49:37.411 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.026e-05 2024-09-16T14:49:37.411 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.411 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.411 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.412 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013847+0000", 2024-09-16T14:49:37.412 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9579000000000001e-05 2024-09-16T14:49:37.412 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.412 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.412 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.412 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013857+0000", 2024-09-16T14:49:37.413 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9939999999999995e-06 2024-09-16T14:49:37.413 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.413 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.413 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.413 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018456+0000", 2024-09-16T14:49:37.413 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.004598141 2024-09-16T14:49:37.414 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.414 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.414 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.414 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018465+0000", 2024-09-16T14:49:37.415 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2439999999999999e-06 2024-09-16T14:49:37.415 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.415 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.415 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.415 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.415 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.415 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:702 3.as0 3:572aff57:::smithi04040151-20:head [copy-from ver 4] snapc 2f=[2f,2e,29,28,25] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:37.416 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:37.416 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.472630736999999, 2024-09-16T14:49:37.416 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.032342711000000003, 2024-09-16T14:49:37.416 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.416 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:37.416 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:37.416 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:37.417 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:37.417 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 702 2024-09-16T14:49:37.417 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.417 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.417 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.417 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.417 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:37.418 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.418 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.418 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.418 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.418 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:37.418 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.419 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.419 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.419 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.419 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900076+0000", 2024-09-16T14:49:37.419 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:37.419 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.419 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900082+0000", 2024-09-16T14:49:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3640000000000004e-06 2024-09-16T14:49:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.420 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900094+0000", 2024-09-16T14:49:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1208e-05 2024-09-16T14:49:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.421 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900105+0000", 2024-09-16T14:49:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1782e-05 2024-09-16T14:49:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.422 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900161+0000", 2024-09-16T14:49:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5244999999999997e-05 2024-09-16T14:49:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900170+0000", 2024-09-16T14:49:37.423 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4409999999999999e-06 2024-09-16T14:49:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.901930+0000", 2024-09-16T14:49:37.424 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001760246 2024-09-16T14:49:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902046+0000", 2024-09-16T14:49:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000115882 2024-09-16T14:49:37.425 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.917553+0000", 2024-09-16T14:49:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015506967999999999 2024-09-16T14:49:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918732+0000", 2024-09-16T14:49:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001178733 2024-09-16T14:49:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.426 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932379+0000", 2024-09-16T14:49:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013647276999999999 2024-09-16T14:49:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932420+0000", 2024-09-16T14:49:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1066999999999998e-05 2024-09-16T14:49:37.427 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:703 3.as0 3:572aff57:::smithi04040151-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:37.428 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.472589339000001, 2024-09-16T14:49:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.018566787000000001, 2024-09-16T14:49:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:37.429 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 703 2024-09-16T14:49:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.430 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900118+0000", 2024-09-16T14:49:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.431 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900122+0000", 2024-09-16T14:49:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5669999999999999e-06 2024-09-16T14:49:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.432 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.433 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.433 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900129+0000", 2024-09-16T14:49:37.433 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0360000000000001e-06 2024-09-16T14:49:37.433 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.433 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.433 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.434 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900134+0000", 2024-09-16T14:49:37.434 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.916e-06 2024-09-16T14:49:37.434 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.434 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.434 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.434 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900217+0000", 2024-09-16T14:49:37.435 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3189000000000006e-05 2024-09-16T14:49:37.435 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.435 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.435 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:37.435 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900222+0000", 2024-09-16T14:49:37.435 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7430000000000002e-06 2024-09-16T14:49:37.436 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.436 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902209+0000", 2024-09-16T14:49:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0019871440000000001 2024-09-16T14:49:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:37.437 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902254+0000", 2024-09-16T14:49:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4969000000000003e-05 2024-09-16T14:49:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918411+0000", 2024-09-16T14:49:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.016157207999999999 2024-09-16T14:49:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918534+0000", 2024-09-16T14:49:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000123146 2024-09-16T14:49:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.438 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918686+0000", 2024-09-16T14:49:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015134799999999999 2024-09-16T14:49:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s2 77/65 ECSubWrite(tid=312, reqid=client.4500.0:711, at_version=77'98, trim_to=0'0, roll_forward_to=73'95))", 2024-09-16T14:49:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.915065+0000", 2024-09-16T14:49:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.457642878, 2024-09-16T14:49:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003580101, 2024-09-16T14:49:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.439 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915065+0000", 2024-09-16T14:49:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915065+0000", 2024-09-16T14:49:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.440 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915065+0000", 2024-09-16T14:49:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915202+0000", 2024-09-16T14:49:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00013698000000000001 2024-09-16T14:49:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915211+0000", 2024-09-16T14:49:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1940000000000004e-06 2024-09-16T14:49:37.441 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915218+0000", 2024-09-16T14:49:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2230000000000001e-06 2024-09-16T14:49:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915387+0000", 2024-09-16T14:49:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00016841100000000001 2024-09-16T14:49:37.442 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.915395+0000", 2024-09-16T14:49:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4530000000000002e-06 2024-09-16T14:49:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918628+0000", 2024-09-16T14:49:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0032327760000000001 2024-09-16T14:49:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.443 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918646+0000", 2024-09-16T14:49:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7750000000000001e-05 2024-09-16T14:49:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 80/65 ECSubWrite(tid=46, reqid=client.4500.0:745, at_version=80'31, trim_to=0'0, roll_forward_to=66'27))", 2024-09-16T14:49:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:26.992801+0000", 2024-09-16T14:49:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.379907013, 2024-09-16T14:49:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045201599999999996, 2024-09-16T14:49:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.444 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992801+0000", 2024-09-16T14:49:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992801+0000", 2024-09-16T14:49:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.445 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992801+0000", 2024-09-16T14:49:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992818+0000", 2024-09-16T14:49:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7876e-05 2024-09-16T14:49:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992829+0000", 2024-09-16T14:49:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0756e-05 2024-09-16T14:49:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.446 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992838+0000", 2024-09-16T14:49:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8969999999999997e-06 2024-09-16T14:49:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992889+0000", 2024-09-16T14:49:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.1236000000000002e-05 2024-09-16T14:49:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.992900+0000", 2024-09-16T14:49:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0183999999999999e-05 2024-09-16T14:49:37.447 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.997301+0000", 2024-09-16T14:49:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044018879999999996 2024-09-16T14:49:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:26.997321+0000", 2024-09-16T14:49:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0026000000000001e-05 2024-09-16T14:49:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.448 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s6 82/65 ECSubWrite(tid=85, reqid=osd.1.0:85, at_version=82'68, trim_to=0'0, roll_forward_to=81'66))", 2024-09-16T14:49:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.928836+0000", 2024-09-16T14:49:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 8.4438722229999996, 2024-09-16T14:49:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0035656770000000002, 2024-09-16T14:49:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928836+0000", 2024-09-16T14:49:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928836+0000", 2024-09-16T14:49:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928836+0000", 2024-09-16T14:49:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928858+0000", 2024-09-16T14:49:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2561e-05 2024-09-16T14:49:37.450 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928871+0000", 2024-09-16T14:49:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3054000000000001e-05 2024-09-16T14:49:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928880+0000", 2024-09-16T14:49:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.32e-06 2024-09-16T14:49:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928936+0000", 2024-09-16T14:49:37.451 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.6548999999999997e-05 2024-09-16T14:49:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928946+0000", 2024-09-16T14:49:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0064999999999999e-05 2024-09-16T14:49:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.932369+0000", 2024-09-16T14:49:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034226220000000002 2024-09-16T14:49:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.452 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.932402+0000", 2024-09-16T14:49:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2755000000000001e-05 2024-09-16T14:49:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.0s2 82/65 ECSubWrite(tid=289, reqid=osd.11.0:289, at_version=82'41, trim_to=0'0, roll_forward_to=79'39))", 2024-09-16T14:49:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.936772+0000", 2024-09-16T14:49:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 8.4359362949999994, 2024-09-16T14:49:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048157149999999999, 2024-09-16T14:49:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.453 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936772+0000", 2024-09-16T14:49:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936772+0000", 2024-09-16T14:49:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.454 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936772+0000", 2024-09-16T14:49:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936794+0000", 2024-09-16T14:49:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2670000000000001e-05 2024-09-16T14:49:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936811+0000", 2024-09-16T14:49:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6971000000000001e-05 2024-09-16T14:49:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.455 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936820+0000", 2024-09-16T14:49:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6600000000000001e-06 2024-09-16T14:49:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936895+0000", 2024-09-16T14:49:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4531999999999997e-05 2024-09-16T14:49:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936904+0000", 2024-09-16T14:49:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1900000000000001e-06 2024-09-16T14:49:37.456 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.941567+0000", 2024-09-16T14:49:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0046633009999999999 2024-09-16T14:49:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.941588+0000", 2024-09-16T14:49:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0682e-05 2024-09-16T14:49:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.457 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds3 82/65 ECSubWrite(tid=296, reqid=osd.7.0:295, at_version=0'0, trim_to=0'0, roll_forward_to=82'67))", 2024-09-16T14:49:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.940379+0000", 2024-09-16T14:49:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 8.4323291200000003, 2024-09-16T14:49:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003481591, 2024-09-16T14:49:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940379+0000", 2024-09-16T14:49:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.458 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940379+0000", 2024-09-16T14:49:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940378+0000", 2024-09-16T14:49:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940390+0000", 2024-09-16T14:49:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.24e-05 2024-09-16T14:49:37.459 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940405+0000", 2024-09-16T14:49:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4786999999999999e-05 2024-09-16T14:49:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940413+0000", 2024-09-16T14:49:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8539999999999997e-06 2024-09-16T14:49:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.460 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940465+0000", 2024-09-16T14:49:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.1403999999999998e-05 2024-09-16T14:49:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.940474+0000", 2024-09-16T14:49:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6500000000000008e-06 2024-09-16T14:49:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.943834+0000", 2024-09-16T14:49:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003359534 2024-09-16T14:49:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.461 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.943861+0000", 2024-09-16T14:49:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7072000000000001e-05 2024-09-16T14:49:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:767 3.as0 3:572aff57:::smithi04040151-20:head [copy-from ver 63] snapc 31=[2e,28] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:49:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:29.077215+0000", 2024-09-16T14:49:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 8.2954930269999991, 2024-09-16T14:49:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.021272019999999999, 2024-09-16T14:49:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:37.462 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 767 2024-09-16T14:49:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077215+0000", 2024-09-16T14:49:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077215+0000", 2024-09-16T14:49:37.463 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077213+0000", 2024-09-16T14:49:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077223+0000", 2024-09-16T14:49:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9890000000000004e-06 2024-09-16T14:49:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.464 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077242+0000", 2024-09-16T14:49:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8286000000000001e-05 2024-09-16T14:49:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077253+0000", 2024-09-16T14:49:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1807999999999999e-05 2024-09-16T14:49:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077321+0000", 2024-09-16T14:49:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7131999999999994e-05 2024-09-16T14:49:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.465 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077431+0000", 2024-09-16T14:49:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0001102 2024-09-16T14:49:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.090085+0000", 2024-09-16T14:49:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012654349 2024-09-16T14:49:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092962+0000", 2024-09-16T14:49:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002877133 2024-09-16T14:49:37.466 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.098442+0000", 2024-09-16T14:49:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0054794149999999996 2024-09-16T14:49:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.098487+0000", 2024-09-16T14:49:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5587000000000002e-05 2024-09-16T14:49:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.467 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:768 3.as0 3:572aff57:::smithi04040151-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:49:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 8.2954418870000008, 2024-09-16T14:49:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015338218000000001, 2024-09-16T14:49:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 768 2024-09-16T14:49:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.468 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.469 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077268+0000", 2024-09-16T14:49:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.266e-06 2024-09-16T14:49:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077275+0000", 2024-09-16T14:49:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6200000000000001e-06 2024-09-16T14:49:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.470 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077278+0000", 2024-09-16T14:49:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.055e-06 2024-09-16T14:49:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077612+0000", 2024-09-16T14:49:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000334321 2024-09-16T14:49:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077660+0000", 2024-09-16T14:49:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8161999999999997e-05 2024-09-16T14:49:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.471 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092440+0000", 2024-09-16T14:49:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014779825999999999 2024-09-16T14:49:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092530+0000", 2024-09-16T14:49:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9232000000000002e-05 2024-09-16T14:49:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092605+0000", 2024-09-16T14:49:37.472 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5086999999999994e-05 2024-09-16T14:49:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds3 86/65 ECSubWrite(tid=302, reqid=client.4500.0:792, at_version=86'71, trim_to=0'0, roll_forward_to=83'69))", 2024-09-16T14:49:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:32.881497+0000", 2024-09-16T14:49:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 4.4912115960000003, 2024-09-16T14:49:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0055907470000000001, 2024-09-16T14:49:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.473 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881497+0000", 2024-09-16T14:49:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881497+0000", 2024-09-16T14:49:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881496+0000", 2024-09-16T14:49:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.474 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881544+0000", 2024-09-16T14:49:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7973000000000003e-05 2024-09-16T14:49:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881555+0000", 2024-09-16T14:49:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1405e-05 2024-09-16T14:49:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881564+0000", 2024-09-16T14:49:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7329999999999992e-06 2024-09-16T14:49:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881635+0000", 2024-09-16T14:49:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.1067000000000002e-05 2024-09-16T14:49:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881647+0000", 2024-09-16T14:49:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1239e-05 2024-09-16T14:49:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.476 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.887060+0000", 2024-09-16T14:49:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005413691 2024-09-16T14:49:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.887087+0000", 2024-09-16T14:49:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7234e-05 2024-09-16T14:49:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.cs2 90/65 ECSubWrite(tid=189, reqid=osd.5.0:189, at_version=90'73, trim_to=0'0, roll_forward_to=89'71))", 2024-09-16T14:49:37.477 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.947178+0000", 2024-09-16T14:49:37.478 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.425530293, 2024-09-16T14:49:37.478 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014311760999999999, 2024-09-16T14:49:37.478 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.478 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.478 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.478 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.478 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.478 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947178+0000", 2024-09-16T14:49:37.478 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.478 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.478 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.478 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.478 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947178+0000", 2024-09-16T14:49:37.478 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.479 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.479 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.479 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.479 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947177+0000", 2024-09-16T14:49:37.479 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.479 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.479 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.479 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.479 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947210+0000", 2024-09-16T14:49:37.479 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2759999999999998e-05 2024-09-16T14:49:37.479 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.479 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.479 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.479 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947224+0000", 2024-09-16T14:49:37.480 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3577000000000001e-05 2024-09-16T14:49:37.480 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.480 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.480 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.480 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947233+0000", 2024-09-16T14:49:37.480 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8169999999999994e-06 2024-09-16T14:49:37.480 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.480 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.480 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.480 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947304+0000", 2024-09-16T14:49:37.480 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0809000000000006e-05 2024-09-16T14:49:37.480 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.480 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.481 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.481 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947317+0000", 2024-09-16T14:49:37.481 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.292e-05 2024-09-16T14:49:37.481 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.481 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.481 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.481 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.961464+0000", 2024-09-16T14:49:37.481 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014146532 2024-09-16T14:49:37.481 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.481 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.481 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.481 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.961490+0000", 2024-09-16T14:49:37.481 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6012000000000001e-05 2024-09-16T14:49:37.482 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.482 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.482 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.482 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.482 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.482 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs7 90/65 ECSubWrite(tid=379, reqid=osd.3.0:379, at_version=90'75, trim_to=0'0, roll_forward_to=83'73))", 2024-09-16T14:49:37.482 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.947459+0000", 2024-09-16T14:49:37.482 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.42524974500000001, 2024-09-16T14:49:37.482 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0096387420000000005, 2024-09-16T14:49:37.482 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.482 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.482 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.482 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.482 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.483 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947459+0000", 2024-09-16T14:49:37.483 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.483 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.483 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.483 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.483 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947459+0000", 2024-09-16T14:49:37.483 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.483 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.483 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.483 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.483 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947458+0000", 2024-09-16T14:49:37.483 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.483 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.483 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.484 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.484 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947486+0000", 2024-09-16T14:49:37.484 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8123000000000002e-05 2024-09-16T14:49:37.484 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.484 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.484 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.484 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947496+0000", 2024-09-16T14:49:37.484 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7650000000000005e-06 2024-09-16T14:49:37.484 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.484 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.484 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.484 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947504+0000", 2024-09-16T14:49:37.484 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9719999999999992e-06 2024-09-16T14:49:37.484 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.485 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.485 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.485 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947574+0000", 2024-09-16T14:49:37.485 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0259999999999995e-05 2024-09-16T14:49:37.485 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.485 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.485 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.485 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947584+0000", 2024-09-16T14:49:37.485 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7440000000000002e-06 2024-09-16T14:49:37.485 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.485 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.485 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.485 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957064+0000", 2024-09-16T14:49:37.486 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0094797530000000005 2024-09-16T14:49:37.486 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.486 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.486 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.486 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957097+0000", 2024-09-16T14:49:37.486 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3615999999999998e-05 2024-09-16T14:49:37.486 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.486 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.486 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.486 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.486 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.486 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs7 90/65 ECSubWrite(tid=380, reqid=osd.3.0:380, at_version=90'77, trim_to=0'0, roll_forward_to=83'73))", 2024-09-16T14:49:37.486 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.948118+0000", 2024-09-16T14:49:37.486 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.42458989899999999, 2024-09-16T14:49:37.487 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014633293, 2024-09-16T14:49:37.487 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.487 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.487 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.487 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.487 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.487 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948118+0000", 2024-09-16T14:49:37.487 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.487 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.487 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.487 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.487 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948118+0000", 2024-09-16T14:49:37.487 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.488 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.488 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.488 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.488 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948118+0000", 2024-09-16T14:49:37.488 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.488 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.488 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.488 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.488 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948141+0000", 2024-09-16T14:49:37.488 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3300999999999999e-05 2024-09-16T14:49:37.488 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.488 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.488 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.488 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948158+0000", 2024-09-16T14:49:37.489 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6999000000000001e-05 2024-09-16T14:49:37.489 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.489 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.489 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.489 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948166+0000", 2024-09-16T14:49:37.489 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.551e-06 2024-09-16T14:49:37.489 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.489 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.489 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.489 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948701+0000", 2024-09-16T14:49:37.489 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00053460599999999997 2024-09-16T14:49:37.489 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.489 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.490 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.490 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948711+0000", 2024-09-16T14:49:37.490 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6889999999999999e-06 2024-09-16T14:49:37.490 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.490 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.490 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.490 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.962726+0000", 2024-09-16T14:49:37.490 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014015272 2024-09-16T14:49:37.490 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.490 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.490 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.490 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.962752+0000", 2024-09-16T14:49:37.490 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5701999999999999e-05 2024-09-16T14:49:37.491 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.491 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.491 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.491 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.491 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.491 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.cs2 90/65 ECSubWrite(tid=190, reqid=osd.5.0:190, at_version=90'75, trim_to=0'0, roll_forward_to=89'71))", 2024-09-16T14:49:37.491 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.957165+0000", 2024-09-16T14:49:37.491 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.41554376399999998, 2024-09-16T14:49:37.491 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0094526560000000003, 2024-09-16T14:49:37.491 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.491 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.491 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.491 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.491 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.492 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957165+0000", 2024-09-16T14:49:37.492 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.492 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.492 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.492 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.492 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957165+0000", 2024-09-16T14:49:37.492 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.492 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.492 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.492 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.492 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957164+0000", 2024-09-16T14:49:37.492 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.492 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.493 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.493 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.493 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957190+0000", 2024-09-16T14:49:37.493 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6451000000000001e-05 2024-09-16T14:49:37.493 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.493 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.493 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.493 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957203+0000", 2024-09-16T14:49:37.493 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.237e-05 2024-09-16T14:49:37.493 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.493 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.493 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.493 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957211+0000", 2024-09-16T14:49:37.493 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8789999999999992e-06 2024-09-16T14:49:37.494 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.494 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.494 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.494 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.961640+0000", 2024-09-16T14:49:37.494 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044288629999999999 2024-09-16T14:49:37.494 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.494 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.494 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.494 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.961649+0000", 2024-09-16T14:49:37.494 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9660000000000002e-06 2024-09-16T14:49:37.494 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.494 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.494 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.494 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.966600+0000", 2024-09-16T14:49:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049509059999999997 2024-09-16T14:49:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.966617+0000", 2024-09-16T14:49:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7010000000000001e-05 2024-09-16T14:49:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds3 90/65 ECSubWrite(tid=311, reqid=osd.7.0:311, at_version=90'75, trim_to=0'0, roll_forward_to=89'73))", 2024-09-16T14:49:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.958625+0000", 2024-09-16T14:49:37.495 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.41408302899999999, 2024-09-16T14:49:37.496 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0061203280000000004, 2024-09-16T14:49:37.496 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.496 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.496 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.496 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.496 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.496 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958625+0000", 2024-09-16T14:49:37.496 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.496 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.496 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.496 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.496 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958625+0000", 2024-09-16T14:49:37.496 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.497 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.497 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.497 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.497 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958625+0000", 2024-09-16T14:49:37.497 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:37.497 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.497 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.497 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.497 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958647+0000", 2024-09-16T14:49:37.497 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2073999999999999e-05 2024-09-16T14:49:37.497 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.497 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.497 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.497 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958660+0000", 2024-09-16T14:49:37.498 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2831e-05 2024-09-16T14:49:37.498 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.498 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.498 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.498 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958673+0000", 2024-09-16T14:49:37.498 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3169999999999999e-05 2024-09-16T14:49:37.498 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.498 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.498 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.498 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958738+0000", 2024-09-16T14:49:37.498 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.4768000000000001e-05 2024-09-16T14:49:37.498 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.498 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.498 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.499 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958754+0000", 2024-09-16T14:49:37.499 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6388999999999998e-05 2024-09-16T14:49:37.499 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.499 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.499 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.499 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.964722+0000", 2024-09-16T14:49:37.499 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0059672579999999996 2024-09-16T14:49:37.499 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.499 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.499 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.499 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.964746+0000", 2024-09-16T14:49:37.499 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4070000000000002e-05 2024-09-16T14:49:37.499 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.499 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.500 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.500 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.500 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.500 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds3 90/65 ECSubWrite(tid=312, reqid=osd.7.0:312, at_version=90'77, trim_to=0'0, roll_forward_to=89'73))", 2024-09-16T14:49:37.500 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.962249+0000", 2024-09-16T14:49:37.500 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.410458975, 2024-09-16T14:49:37.500 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0043313650000000002, 2024-09-16T14:49:37.500 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:37.500 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.500 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:37.500 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.500 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:37.500 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.962249+0000", 2024-09-16T14:49:37.501 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.501 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.501 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.501 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:37.501 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.962249+0000", 2024-09-16T14:49:37.501 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:37.501 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.501 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.501 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:37.501 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.962249+0000", 2024-09-16T14:49:37.501 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.501 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.501 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.501 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:37.502 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.962271+0000", 2024-09-16T14:49:37.502 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2152999999999999e-05 2024-09-16T14:49:37.502 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.502 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.502 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:37.502 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.962285+0000", 2024-09-16T14:49:37.502 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3923999999999999e-05 2024-09-16T14:49:37.502 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.502 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.502 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.502 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.962293+0000", 2024-09-16T14:49:37.502 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8459999999999997e-06 2024-09-16T14:49:37.502 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.502 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.962402+0000", 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010868 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.962411+0000", 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.986e-06 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.966556+0000", 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0041446879999999997 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:37.503 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:37.504 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:37.504 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.966581+0000", 2024-09-16T14:49:37.504 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4864000000000001e-05 2024-09-16T14:49:37.504 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.504 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.504 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.504 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:37.504 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:37.504 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:37.506 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:37.503+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:37.506 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:49:37.606 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:37.605+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:37.614 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:37.614 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:37.615 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:37.615 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:37.623 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:49:37.707 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:37.705+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:37.729 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:37.729 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:37.730 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:37.730 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:37.730 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:37.738 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:49:37.807 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:37.805+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:37.846 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:37.846 INFO:teuthology.orchestra.run.smithi040.stdout: "size": 20, 2024-09-16T14:49:37.846 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 600, 2024-09-16T14:49:37.846 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [ 2024-09-16T14:49:37.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.846 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:454 3.6s0 3:608d2d89:::smithi04040151-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:37.846 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:37.846 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 39.914283537000003, 2024-09-16T14:49:37.846 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013776338000000001, 2024-09-16T14:49:37.846 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.847 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:37.847 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:37.847 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:37.847 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:37.847 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 454 2024-09-16T14:49:37.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.847 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.847 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.847 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:37.847 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.848 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.848 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:37.848 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.848 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.848 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929433+0000", 2024-09-16T14:49:37.848 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.848 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.848 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929436+0000", 2024-09-16T14:49:37.848 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7939999999999998e-06 2024-09-16T14:49:37.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.849 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.849 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929444+0000", 2024-09-16T14:49:37.849 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.4749999999999993e-06 2024-09-16T14:49:37.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.849 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.849 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.929449+0000", 2024-09-16T14:49:37.849 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2750000000000003e-06 2024-09-16T14:49:37.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.849 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.850 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.933117+0000", 2024-09-16T14:49:37.850 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0036674250000000002 2024-09-16T14:49:37.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.850 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:37.850 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.933207+0000", 2024-09-16T14:49:37.850 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0358000000000005e-05 2024-09-16T14:49:37.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.850 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.850 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.937436+0000", 2024-09-16T14:49:37.850 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0042287089999999998 2024-09-16T14:49:37.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.851 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:37.851 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.937535+0000", 2024-09-16T14:49:37.851 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9501999999999997e-05 2024-09-16T14:49:37.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.851 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.851 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:57.943210+0000", 2024-09-16T14:49:37.851 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0056742429999999998 2024-09-16T14:49:37.851 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.851 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.851 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.851 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 52/27 ECSubWrite(tid=182, reqid=osd.11.0:182, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:37.852 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:37.852 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 38.956471790999998, 2024-09-16T14:49:37.852 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014202568, 2024-09-16T14:49:37.852 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.852 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.852 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.852 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.852 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.852 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:37.852 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887245+0000", 2024-09-16T14:49:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.853 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887302+0000", 2024-09-16T14:49:37.854 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.7673999999999997e-05 2024-09-16T14:49:37.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.854 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.854 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.854 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887313+0000", 2024-09-16T14:49:37.854 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0712e-05 2024-09-16T14:49:37.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.854 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.854 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.854 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887321+0000", 2024-09-16T14:49:37.854 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.1329999999999999e-06 2024-09-16T14:49:37.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.854 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.854 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.855 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887397+0000", 2024-09-16T14:49:37.855 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5564999999999999e-05 2024-09-16T14:49:37.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.855 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.855 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887407+0000", 2024-09-16T14:49:37.855 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.9690000000000006e-06 2024-09-16T14:49:37.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.855 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.855 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901422+0000", 2024-09-16T14:49:37.855 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.01401468 2024-09-16T14:49:37.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.856 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.856 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901448+0000", 2024-09-16T14:49:37.856 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6191999999999999e-05 2024-09-16T14:49:37.856 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.856 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.856 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.856 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 52/27 ECSubWrite(tid=214, reqid=osd.7.0:214, at_version=52'35, trim_to=0'0, roll_forward_to=51'31))", 2024-09-16T14:49:37.856 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:37.856 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 38.956428592000002, 2024-09-16T14:49:37.856 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014116002000000001, 2024-09-16T14:49:37.857 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.857 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.857 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.857 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.857 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:37.857 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.857 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.857 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:37.857 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.858 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.858 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.858 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887288+0000", 2024-09-16T14:49:37.858 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.858 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.858 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.858 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.858 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887318+0000", 2024-09-16T14:49:37.858 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.0477999999999999e-05 2024-09-16T14:49:37.858 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.858 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.858 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.858 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887329+0000", 2024-09-16T14:49:37.858 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0845999999999999e-05 2024-09-16T14:49:37.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.859 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.859 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887337+0000", 2024-09-16T14:49:37.859 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7880000000000007e-06 2024-09-16T14:49:37.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.859 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.859 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887631+0000", 2024-09-16T14:49:37.859 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00029412399999999999 2024-09-16T14:49:37.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.859 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.859 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.887640+0000", 2024-09-16T14:49:37.860 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.0180000000000007e-06 2024-09-16T14:49:37.860 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.860 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.860 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.860 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901368+0000", 2024-09-16T14:49:37.860 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013728346000000001 2024-09-16T14:49:37.860 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.860 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.860 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.860 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.901404+0000", 2024-09-16T14:49:37.860 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.5964000000000003e-05 2024-09-16T14:49:37.860 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.860 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.861 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.861 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 52/27 ECSubWrite(tid=51, reqid=osd.1.0:50, at_version=0'0, trim_to=0'0, roll_forward_to=52'40))", 2024-09-16T14:49:37.861 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:37.861 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 38.954555624000001, 2024-09-16T14:49:37.861 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013036324, 2024-09-16T14:49:37.861 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.861 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.861 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.861 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.861 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:37.861 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.862 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.862 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.862 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889161+0000", 2024-09-16T14:49:37.862 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.862 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.862 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.862 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889160+0000", 2024-09-16T14:49:37.862 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.862 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.862 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.862 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889174+0000", 2024-09-16T14:49:37.863 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3416e-05 2024-09-16T14:49:37.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.863 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.863 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889186+0000", 2024-09-16T14:49:37.863 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2369e-05 2024-09-16T14:49:37.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.863 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.863 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889195+0000", 2024-09-16T14:49:37.863 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6540000000000005e-06 2024-09-16T14:49:37.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.863 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.864 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889243+0000", 2024-09-16T14:49:37.864 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.8455000000000003e-05 2024-09-16T14:49:37.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.864 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.864 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889253+0000", 2024-09-16T14:49:37.864 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.3700000000000001e-06 2024-09-16T14:49:37.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.864 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.864 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902168+0000", 2024-09-16T14:49:37.864 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012915311 2024-09-16T14:49:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902198+0000", 2024-09-16T14:49:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9641e-05 2024-09-16T14:49:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s2 52/27 ECSubWrite(tid=29, reqid=osd.4.0:29, at_version=52'20, trim_to=0'0, roll_forward_to=51'18))", 2024-09-16T14:49:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 38.953752449, 2024-09-16T14:49:37.865 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012371216000000001, 2024-09-16T14:49:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.866 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.867 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:37.867 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.867 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.867 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.867 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.867 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889964+0000", 2024-09-16T14:49:37.867 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.867 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.867 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.867 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.867 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.889990+0000", 2024-09-16T14:49:37.867 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5548999999999999e-05 2024-09-16T14:49:37.867 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890001+0000", 2024-09-16T14:49:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.098e-05 2024-09-16T14:49:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890009+0000", 2024-09-16T14:49:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5930000000000006e-06 2024-09-16T14:49:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890069+0000", 2024-09-16T14:49:37.868 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.9855999999999997e-05 2024-09-16T14:49:37.869 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.869 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.869 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.869 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.890082+0000", 2024-09-16T14:49:37.869 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3312999999999999e-05 2024-09-16T14:49:37.869 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.869 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902310+0000", 2024-09-16T14:49:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012227780000000001 2024-09-16T14:49:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:48:58.902336+0000", 2024-09-16T14:49:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.5652000000000001e-05 2024-09-16T14:49:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.870 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.3s2 61/27 ECSubWrite(tid=35, reqid=osd.4.0:34, at_version=0'0, trim_to=0'0, roll_forward_to=61'22))", 2024-09-16T14:49:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.914306947, 2024-09-16T14:49:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012712925, 2024-09-16T14:49:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.871 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.872 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.872 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929410+0000", 2024-09-16T14:49:37.872 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.872 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.872 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.872 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929409+0000", 2024-09-16T14:49:37.872 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.872 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.873 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.873 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929422+0000", 2024-09-16T14:49:37.873 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3497e-05 2024-09-16T14:49:37.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.873 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.873 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.873 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929434+0000", 2024-09-16T14:49:37.873 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.205e-05 2024-09-16T14:49:37.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.873 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.873 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.874 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929443+0000", 2024-09-16T14:49:37.874 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.8419999999999994e-06 2024-09-16T14:49:37.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.874 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.874 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.874 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929490+0000", 2024-09-16T14:49:37.874 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.6825e-05 2024-09-16T14:49:37.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.874 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.874 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.874 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.929500+0000", 2024-09-16T14:49:37.874 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8810000000000008e-06 2024-09-16T14:49:37.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.875 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.875 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.875 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942092+0000", 2024-09-16T14:49:37.875 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012591955 2024-09-16T14:49:37.875 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.875 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.875 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.875 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942123+0000", 2024-09-16T14:49:37.875 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.1015e-05 2024-09-16T14:49:37.875 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.875 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.875 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.876 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.876 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.876 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs6 61/27 ECSubWrite(tid=235, reqid=osd.7.0:235, at_version=61'21, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:37.876 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:37.876 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.912880036000001, 2024-09-16T14:49:37.876 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011696734, 2024-09-16T14:49:37.876 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.876 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.876 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.876 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.877 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.877 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:37.877 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.877 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.877 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.877 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:37.877 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.878 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.878 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.878 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930837+0000", 2024-09-16T14:49:37.878 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967296 2024-09-16T14:49:37.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.878 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.878 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.879 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930866+0000", 2024-09-16T14:49:37.879 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9138000000000001e-05 2024-09-16T14:49:37.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.879 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.879 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.879 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930874+0000", 2024-09-16T14:49:37.879 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.5890000000000003e-06 2024-09-16T14:49:37.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.879 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.879 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.879 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930882+0000", 2024-09-16T14:49:37.879 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.25e-06 2024-09-16T14:49:37.880 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.880 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.880 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.880 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930933+0000", 2024-09-16T14:49:37.880 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.1536999999999998e-05 2024-09-16T14:49:37.880 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.880 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.881 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.881 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.930941+0000", 2024-09-16T14:49:37.881 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.7510000000000008e-06 2024-09-16T14:49:37.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.881 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.881 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.881 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942507+0000", 2024-09-16T14:49:37.881 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011565765 2024-09-16T14:49:37.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.881 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.881 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.881 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.942534+0000", 2024-09-16T14:49:37.881 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6900999999999999e-05 2024-09-16T14:49:37.881 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.882 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.882 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.882 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 61/27 ECSubWrite(tid=63, reqid=osd.1.0:63, at_version=61'50, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:37.882 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:37.882 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.911343599999999, 2024-09-16T14:49:37.882 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014509462000000001, 2024-09-16T14:49:37.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.882 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.882 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.882 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.882 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:37.883 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.883 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.883 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.883 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:37.883 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.883 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.883 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.883 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932373+0000", 2024-09-16T14:49:37.883 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.883 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.884 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.884 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932395+0000", 2024-09-16T14:49:37.884 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2039999999999999e-05 2024-09-16T14:49:37.884 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.884 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.884 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.884 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932404+0000", 2024-09-16T14:49:37.884 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.4879999999999996e-06 2024-09-16T14:49:37.884 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.884 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.884 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.884 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932412+0000", 2024-09-16T14:49:37.885 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.61e-06 2024-09-16T14:49:37.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.885 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.885 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.885 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932577+0000", 2024-09-16T14:49:37.885 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00016469799999999999 2024-09-16T14:49:37.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.885 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.885 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.885 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932605+0000", 2024-09-16T14:49:37.886 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7963e-05 2024-09-16T14:49:37.886 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.886 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.886 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.886 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946863+0000", 2024-09-16T14:49:37.886 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014258600999999999 2024-09-16T14:49:37.886 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.886 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.886 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.886 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946883+0000", 2024-09-16T14:49:37.886 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9494e-05 2024-09-16T14:49:37.886 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.886 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.886 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.887 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.887 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.887 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs6 61/27 ECSubWrite(tid=93, reqid=osd.5.0:93, at_version=61'31, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:37.887 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:37.887 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.911260195000001, 2024-09-16T14:49:37.887 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014400098, 2024-09-16T14:49:37.887 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.887 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.887 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.887 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.887 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.887 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:37.887 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.887 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.888 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.888 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.888 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932457+0000", 2024-09-16T14:49:37.888 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.888 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.888 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.888 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.888 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932456+0000", 2024-09-16T14:49:37.888 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.888 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.888 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.888 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.888 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932475+0000", 2024-09-16T14:49:37.888 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.8695999999999999e-05 2024-09-16T14:49:37.889 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.889 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.889 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.889 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932485+0000", 2024-09-16T14:49:37.889 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7799999999999995e-06 2024-09-16T14:49:37.889 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.889 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.889 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.889 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932492+0000", 2024-09-16T14:49:37.889 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.5209999999999997e-06 2024-09-16T14:49:37.889 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.889 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.889 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.889 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932705+0000", 2024-09-16T14:49:37.890 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00021277700000000001 2024-09-16T14:49:37.890 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.890 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.890 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.890 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932721+0000", 2024-09-16T14:49:37.890 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6095000000000001e-05 2024-09-16T14:49:37.890 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.890 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.890 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.890 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946825+0000", 2024-09-16T14:49:37.890 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014103269999999999 2024-09-16T14:49:37.890 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.890 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.890 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.891 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946857+0000", 2024-09-16T14:49:37.891 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.2271999999999999e-05 2024-09-16T14:49:37.891 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.891 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.891 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.891 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.891 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.891 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.4s7 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:37.891 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:37.891 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.911063345999999, 2024-09-16T14:49:37.891 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015300085999999999, 2024-09-16T14:49:37.891 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.891 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.892 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.892 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.892 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.892 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:37.892 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.892 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.892 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.892 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.892 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932654+0000", 2024-09-16T14:49:37.892 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.892 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.892 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.892 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.892 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932653+0000", 2024-09-16T14:49:37.893 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.893 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.893 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.893 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.893 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932674+0000", 2024-09-16T14:49:37.893 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.1047e-05 2024-09-16T14:49:37.893 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.893 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.893 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.893 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932685+0000", 2024-09-16T14:49:37.893 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0733999999999999e-05 2024-09-16T14:49:37.893 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.893 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932693+0000", 2024-09-16T14:49:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.0819999999999999e-06 2024-09-16T14:49:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945297+0000", 2024-09-16T14:49:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012604198 2024-09-16T14:49:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945306+0000", 2024-09-16T14:49:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.1549999999999996e-06 2024-09-16T14:49:37.894 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.895 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.895 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.895 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947927+0000", 2024-09-16T14:49:37.895 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.002621295 2024-09-16T14:49:37.895 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.895 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.895 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.895 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947954+0000", 2024-09-16T14:49:37.895 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6219000000000001e-05 2024-09-16T14:49:37.895 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.895 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.895 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.895 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.895 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.896 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.5s3 61/53 ECSubWrite(tid=94, reqid=osd.5.0:94, at_version=61'36, trim_to=0'0, roll_forward_to=60'34))", 2024-09-16T14:49:37.896 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:37.896 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.910975617999998, 2024-09-16T14:49:37.896 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013712046, 2024-09-16T14:49:37.896 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.896 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.896 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.896 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.896 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.896 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:37.896 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.896 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.896 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.897 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.897 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:37.897 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.897 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.897 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.897 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932741+0000", 2024-09-16T14:49:37.897 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.897 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.897 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.897 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932760+0000", 2024-09-16T14:49:37.897 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9247999999999999e-05 2024-09-16T14:49:37.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.898 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.898 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.898 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932773+0000", 2024-09-16T14:49:37.898 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3081000000000001e-05 2024-09-16T14:49:37.898 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.898 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.898 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.898 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932783+0000", 2024-09-16T14:49:37.898 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.7850000000000003e-06 2024-09-16T14:49:37.898 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.898 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.898 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.898 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932830+0000", 2024-09-16T14:49:37.898 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.7608999999999998e-05 2024-09-16T14:49:37.899 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.899 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.899 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.932842+0000", 2024-09-16T14:49:37.899 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1082e-05 2024-09-16T14:49:37.899 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.899 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.899 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946430+0000", 2024-09-16T14:49:37.899 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013588141999999999 2024-09-16T14:49:37.899 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.899 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.900 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946453+0000", 2024-09-16T14:49:37.900 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.3714999999999999e-05 2024-09-16T14:49:37.900 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.900 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.900 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.900 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.900 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.900 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:37.900 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:37.900 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.910560675999999, 2024-09-16T14:49:37.900 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013268299000000001, 2024-09-16T14:49:37.900 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.900 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.901 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.901 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.901 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.901 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:37.901 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.901 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.901 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.901 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.901 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933156+0000", 2024-09-16T14:49:37.901 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.901 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.901 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.901 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.902 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933155+0000", 2024-09-16T14:49:37.902 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.902 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.902 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933182+0000", 2024-09-16T14:49:37.902 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.7090999999999999e-05 2024-09-16T14:49:37.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.902 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.902 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933202+0000", 2024-09-16T14:49:37.902 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9459000000000001e-05 2024-09-16T14:49:37.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.903 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.903 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933211+0000", 2024-09-16T14:49:37.903 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.397e-06 2024-09-16T14:49:37.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.903 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.903 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.903 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933284+0000", 2024-09-16T14:49:37.903 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.2711000000000002e-05 2024-09-16T14:49:37.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.903 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.903 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.903 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933294+0000", 2024-09-16T14:49:37.903 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0079e-05 2024-09-16T14:49:37.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.904 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.904 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946408+0000", 2024-09-16T14:49:37.904 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.013114097999999999 2024-09-16T14:49:37.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.904 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.904 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946425+0000", 2024-09-16T14:49:37.904 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6568000000000001e-05 2024-09-16T14:49:37.904 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.904 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.904 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.905 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.905 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.cs6 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:37.905 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:37.905 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.910343775000001, 2024-09-16T14:49:37.905 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.014598939999999999, 2024-09-16T14:49:37.905 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.905 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.905 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.905 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.905 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.905 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:37.905 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.905 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.906 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.906 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933373+0000", 2024-09-16T14:49:37.906 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.906 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.906 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.906 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933372+0000", 2024-09-16T14:49:37.906 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.907 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.907 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.907 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933398+0000", 2024-09-16T14:49:37.907 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.6129999999999999e-05 2024-09-16T14:49:37.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.907 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.907 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.907 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933414+0000", 2024-09-16T14:49:37.907 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5549999999999999e-05 2024-09-16T14:49:37.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.907 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.907 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.907 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.933422+0000", 2024-09-16T14:49:37.908 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8620000000000003e-06 2024-09-16T14:49:37.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.908 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.908 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945189+0000", 2024-09-16T14:49:37.908 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011767552000000001 2024-09-16T14:49:37.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.908 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.908 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.945200+0000", 2024-09-16T14:49:37.908 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0766e-05 2024-09-16T14:49:37.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.909 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.909 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.909 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947957+0000", 2024-09-16T14:49:37.909 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0027565319999999999 2024-09-16T14:49:37.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.909 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.909 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.909 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.947972+0000", 2024-09-16T14:49:37.909 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.5359999999999999e-05 2024-09-16T14:49:37.909 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.909 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.909 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.910 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.910 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.fs6 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:37.910 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:37.910 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.909337509, 2024-09-16T14:49:37.910 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010566212, 2024-09-16T14:49:37.910 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.910 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.910 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.910 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.910 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.910 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:37.910 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.910 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.911 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.911 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.911 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:37.911 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.911 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.911 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.911 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934379+0000", 2024-09-16T14:49:37.911 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.911 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.911 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.911 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934728+0000", 2024-09-16T14:49:37.912 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034932000000000002 2024-09-16T14:49:37.912 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.912 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.912 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.912 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934739+0000", 2024-09-16T14:49:37.912 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1318000000000001e-05 2024-09-16T14:49:37.912 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.912 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.912 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.912 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934752+0000", 2024-09-16T14:49:37.912 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2901999999999999e-05 2024-09-16T14:49:37.912 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.912 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.913 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.913 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934823+0000", 2024-09-16T14:49:37.913 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.1118000000000006e-05 2024-09-16T14:49:37.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.913 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.913 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.934834+0000", 2024-09-16T14:49:37.913 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0591000000000001e-05 2024-09-16T14:49:37.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.913 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.913 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.944917+0000", 2024-09-16T14:49:37.913 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010082906000000001 2024-09-16T14:49:37.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.914 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.914 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.914 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.944946+0000", 2024-09-16T14:49:37.914 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.8895999999999999e-05 2024-09-16T14:49:37.914 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.914 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.914 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.914 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.914 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.ds7 61/27 ECSubWrite(tid=237, reqid=osd.7.0:237, at_version=61'50, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:37.914 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:37.914 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.90813739, 2024-09-16T14:49:37.915 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011297481, 2024-09-16T14:49:37.915 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.915 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.915 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.915 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.915 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.915 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:37.915 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.915 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.915 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.915 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.915 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935580+0000", 2024-09-16T14:49:37.915 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.916 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.916 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.916 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935579+0000", 2024-09-16T14:49:37.916 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.916 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.916 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.916 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935608+0000", 2024-09-16T14:49:37.916 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.9578999999999999e-05 2024-09-16T14:49:37.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.916 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.917 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.917 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935625+0000", 2024-09-16T14:49:37.917 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7075000000000002e-05 2024-09-16T14:49:37.917 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.917 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.917 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.917 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935642+0000", 2024-09-16T14:49:37.917 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.6480000000000001e-05 2024-09-16T14:49:37.917 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.918 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.918 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.918 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935874+0000", 2024-09-16T14:49:37.918 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00023162399999999999 2024-09-16T14:49:37.918 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.918 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.918 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.918 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.935887+0000", 2024-09-16T14:49:37.918 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.3811e-05 2024-09-16T14:49:37.918 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.918 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.918 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.918 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946860+0000", 2024-09-16T14:49:37.918 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010972552999999999 2024-09-16T14:49:37.919 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.919 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.919 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.919 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.946877+0000", 2024-09-16T14:49:37.919 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.7132000000000001e-05 2024-09-16T14:49:37.919 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.919 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.919 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.919 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.919 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.919 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.es4 61/27 ECSubWrite(tid=231, reqid=osd.11.0:231, at_version=61'37, trim_to=0'0, roll_forward_to=60'33))", 2024-09-16T14:49:37.919 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:37.919 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.889888846000002, 2024-09-16T14:49:37.919 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012494261, 2024-09-16T14:49:37.920 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.920 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.920 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.920 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.920 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.920 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:37.920 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.920 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.920 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.920 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.920 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:37.920 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.920 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.921 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.921 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.921 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953828+0000", 2024-09-16T14:49:37.921 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.921 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.921 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.921 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.921 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953840+0000", 2024-09-16T14:49:37.921 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.1983e-05 2024-09-16T14:49:37.921 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.921 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.921 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.921 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953846+0000", 2024-09-16T14:49:37.921 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.319e-06 2024-09-16T14:49:37.922 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.922 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.922 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.922 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.953852+0000", 2024-09-16T14:49:37.922 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.823e-06 2024-09-16T14:49:37.922 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.922 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.922 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.922 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954193+0000", 2024-09-16T14:49:37.922 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034066199999999999 2024-09-16T14:49:37.923 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.923 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.923 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.923 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954203+0000", 2024-09-16T14:49:37.923 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0421999999999999e-05 2024-09-16T14:49:37.923 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.923 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.923 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.923 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966300+0000", 2024-09-16T14:49:37.923 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012096519 2024-09-16T14:49:37.923 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.923 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.923 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.924 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966322+0000", 2024-09-16T14:49:37.924 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2829000000000001e-05 2024-09-16T14:49:37.924 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.924 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.924 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.924 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.924 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.924 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 61/27 ECSubWrite(tid=244, reqid=osd.3.0:243, at_version=0'0, trim_to=0'0, roll_forward_to=61'81))", 2024-09-16T14:49:37.924 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:37.924 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 29.889438576, 2024-09-16T14:49:37.924 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.012038332000000001, 2024-09-16T14:49:37.924 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.924 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.924 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.925 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.925 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:37.925 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.925 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.925 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.925 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954278+0000", 2024-09-16T14:49:37.925 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.925 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.925 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.925 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954277+0000", 2024-09-16T14:49:37.926 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.999999 2024-09-16T14:49:37.926 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.926 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.926 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.926 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954290+0000", 2024-09-16T14:49:37.926 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2364000000000001e-05 2024-09-16T14:49:37.926 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.926 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.926 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.926 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954302+0000", 2024-09-16T14:49:37.926 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.2665e-05 2024-09-16T14:49:37.926 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.926 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.926 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954322+0000", 2024-09-16T14:49:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9429e-05 2024-09-16T14:49:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954374+0000", 2024-09-16T14:49:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 5.2182000000000003e-05 2024-09-16T14:49:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.954384+0000", 2024-09-16T14:49:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0022e-05 2024-09-16T14:49:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.928 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.928 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966295+0000", 2024-09-16T14:49:37.928 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.011910399 2024-09-16T14:49:37.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.928 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.928 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.928 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:07.966317+0000", 2024-09-16T14:49:37.928 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 2.2169999999999999e-05 2024-09-16T14:49:37.928 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.928 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.928 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.928 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.929 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:688 3.6s0 3:608d2d89:::smithi04040151-48:head [copy-from ver 4] snapc 2e=[2e,2b,29,28,25] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T14:49:37.929 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:21.913077+0000", 2024-09-16T14:49:37.929 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.930639964000001, 2024-09-16T14:49:37.929 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.023450634000000001, 2024-09-16T14:49:37.929 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.929 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:37.929 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:37.929 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:37.929 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:37.929 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 688 2024-09-16T14:49:37.929 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.929 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.929 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.929 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.930 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913077+0000", 2024-09-16T14:49:37.930 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.930 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.930 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.930 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.930 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913077+0000", 2024-09-16T14:49:37.930 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.930 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.930 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.930 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.930 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913075+0000", 2024-09-16T14:49:37.930 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:37.930 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.931 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.931 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.931 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913085+0000", 2024-09-16T14:49:37.931 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 9.8810000000000008e-06 2024-09-16T14:49:37.931 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.931 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.931 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.931 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913099+0000", 2024-09-16T14:49:37.931 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.382e-05 2024-09-16T14:49:37.931 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.931 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.931 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.931 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913108+0000", 2024-09-16T14:49:37.932 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.7390000000000005e-06 2024-09-16T14:49:37.932 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.932 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.932 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.932 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913174+0000", 2024-09-16T14:49:37.932 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.6598000000000002e-05 2024-09-16T14:49:37.932 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.932 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.932 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:37.932 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913288+0000", 2024-09-16T14:49:37.932 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011353000000000001 2024-09-16T14:49:37.932 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.932 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.932 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.933 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.929027+0000", 2024-09-16T14:49:37.933 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.015739758999999999 2024-09-16T14:49:37.933 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.933 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.933 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.933 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.931900+0000", 2024-09-16T14:49:37.933 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0028725310000000002 2024-09-16T14:49:37.933 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.933 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.933 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "commit_sent", 2024-09-16T14:49:37.933 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.936419+0000", 2024-09-16T14:49:37.933 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.0045194340000000001 2024-09-16T14:49:37.933 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.934 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.934 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.934 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.936528+0000", 2024-09-16T14:49:37.934 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00010822199999999999 2024-09-16T14:49:37.934 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.934 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.934 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.934 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.934 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.934 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "osd_op(client.4500.0:689 3.6s0 3:608d2d89:::smithi04040151-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-09-16T14:49:37.934 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:37.935 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 15.930591831999999, 2024-09-16T14:49:37.935 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.018229170999999999, 2024-09-16T14:49:37.936 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.936 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "started", 2024-09-16T14:49:37.936 INFO:teuthology.orchestra.run.smithi040.stdout: "client_info": { 2024-09-16T14:49:37.936 INFO:teuthology.orchestra.run.smithi040.stdout: "client": "client.4500", 2024-09-16T14:49:37.936 INFO:teuthology.orchestra.run.smithi040.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:37.936 INFO:teuthology.orchestra.run.smithi040.stdout: "tid": 689 2024-09-16T14:49:37.936 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.936 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.936 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.936 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.936 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:37.936 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.936 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.936 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.937 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.937 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:37.937 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.937 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.937 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.937 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.937 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913125+0000", 2024-09-16T14:49:37.937 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.937 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.937 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.937 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.937 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913128+0000", 2024-09-16T14:49:37.937 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.4580000000000002e-06 2024-09-16T14:49:37.937 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.938 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.938 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.938 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913137+0000", 2024-09-16T14:49:37.938 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6039999999999993e-06 2024-09-16T14:49:37.938 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.938 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.938 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.938 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913145+0000", 2024-09-16T14:49:37.938 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 8.6200000000000005e-06 2024-09-16T14:49:37.938 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.938 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.938 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.938 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913491+0000", 2024-09-16T14:49:37.939 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00034584200000000001 2024-09-16T14:49:37.939 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.939 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.939 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:37.939 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.913536+0000", 2024-09-16T14:49:37.939 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4.5269000000000003e-05 2024-09-16T14:49:37.939 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.939 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.939 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.939 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.930844+0000", 2024-09-16T14:49:37.939 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.017307802000000001 2024-09-16T14:49:37.939 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.939 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.939 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "started", 2024-09-16T14:49:37.940 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.930954+0000", 2024-09-16T14:49:37.940 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00011033500000000001 2024-09-16T14:49:37.940 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.940 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.940 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.940 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:21.931354+0000", 2024-09-16T14:49:37.940 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.00039983500000000002 2024-09-16T14:49:37.940 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.940 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.940 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.940 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.940 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.940 INFO:teuthology.orchestra.run.smithi040.stdout: "description": "MOSDECSubOpWrite(3.7s3 77/27 ECSubWrite(tid=311, reqid=osd.3.0:311, at_version=77'100, trim_to=0'0, roll_forward_to=76'98))", 2024-09-16T14:49:37.940 INFO:teuthology.orchestra.run.smithi040.stdout: "initiated_at": "2024-09-16T14:49:23.915984+0000", 2024-09-16T14:49:37.941 INFO:teuthology.orchestra.run.smithi040.stdout: "age": 13.92773276, 2024-09-16T14:49:37.941 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010184956, 2024-09-16T14:49:37.941 INFO:teuthology.orchestra.run.smithi040.stdout: "type_data": { 2024-09-16T14:49:37.941 INFO:teuthology.orchestra.run.smithi040.stdout: "flag_point": "reached pg", 2024-09-16T14:49:37.941 INFO:teuthology.orchestra.run.smithi040.stdout: "events": [ 2024-09-16T14:49:37.941 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.941 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "initiated", 2024-09-16T14:49:37.941 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.915984+0000", 2024-09-16T14:49:37.941 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.941 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.941 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.941 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "throttled", 2024-09-16T14:49:37.941 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.915984+0000", 2024-09-16T14:49:37.941 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0 2024-09-16T14:49:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "header_read", 2024-09-16T14:49:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.915984+0000", 2024-09-16T14:49:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "all_read", 2024-09-16T14:49:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916003+0000", 2024-09-16T14:49:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.9877999999999998e-05 2024-09-16T14:49:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "dispatched", 2024-09-16T14:49:37.942 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916014+0000", 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0988999999999999e-05 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "queued_for_pg", 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916022+0000", 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 7.8930000000000005e-06 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "reached_pg", 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916084+0000", 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 6.1385999999999998e-05 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_started", 2024-09-16T14:49:37.943 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.916095+0000", 2024-09-16T14:49:37.944 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 1.0852999999999999e-05 2024-09-16T14:49:37.944 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.944 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.944 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "sub_op_committed", 2024-09-16T14:49:37.944 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.926132+0000", 2024-09-16T14:49:37.944 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 0.010037842999999999 2024-09-16T14:49:37.944 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-16T14:49:37.944 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-16T14:49:37.944 INFO:teuthology.orchestra.run.smithi040.stdout: "event": "done", 2024-09-16T14:49:37.944 INFO:teuthology.orchestra.run.smithi040.stdout: "time": "2024-09-16T14:49:23.926169+0000", 2024-09-16T14:49:37.944 INFO:teuthology.orchestra.run.smithi040.stdout: "duration": 3.6777999999999999e-05 2024-09-16T14:49:37.944 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.945 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.945 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.945 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-16T14:49:37.945 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-16T14:49:37.945 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:37.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:49:37.947 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is unset 2024-09-16T14:49:37.947 INFO:tasks.rados.rados.0.smithi040.stdout:341: done (1 left) 2024-09-16T14:49:37.947 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 26 current snap is 32 2024-09-16T14:49:37.947 INFO:tasks.rados.rados.0.smithi040.stdout:342: seq_num 124 ranges {2670592=540672,3211264=720896,3932160=786432,4718592=49152} 2024-09-16T14:49:37.950 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:37.945+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:37.968 INFO:tasks.rados.rados.0.smithi040.stdout:342: writing smithi04040151-26 from 2670592 to 3211264 tid 1 2024-09-16T14:49:37.977 INFO:tasks.rados.rados.0.smithi040.stdout:342: writing smithi04040151-26 from 3211264 to 3932160 tid 2 2024-09-16T14:49:37.987 INFO:tasks.rados.rados.0.smithi040.stdout:342: writing smithi04040151-26 from 3932160 to 4718592 tid 3 2024-09-16T14:49:37.987 INFO:tasks.rados.rados.0.smithi040.stdout:342: writing smithi04040151-26 from 4718592 to 4767744 tid 4 2024-09-16T14:49:37.987 INFO:tasks.rados.rados.0.smithi040.stdout:343: read oid 28 snap 28 2024-09-16T14:49:37.988 INFO:tasks.rados.rados.0.smithi040.stdout:343: expect (ObjNum 68 snap 4 seq_num 68) 2024-09-16T14:49:37.988 INFO:tasks.rados.rados.0.smithi040.stdout:344: read oid 4 snap 31 2024-09-16T14:49:37.988 INFO:tasks.rados.rados.0.smithi040.stdout:344: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-16T14:49:37.988 INFO:tasks.rados.rados.0.smithi040.stdout:345: read oid 21 snap 28 2024-09-16T14:49:37.988 INFO:tasks.rados.rados.0.smithi040.stdout:345: expect deleted 2024-09-16T14:49:37.988 INFO:tasks.rados.rados.0.smithi040.stdout:346: snap_remove snap 27 2024-09-16T14:49:38.047 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:38.045+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:38.101 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:38.101 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:38.101 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:38.101 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:38.109 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:49:38.150 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:38.148+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:38.178 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:49:38.250 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:38.248+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:38.257 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:38.257 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:38.266 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:49:38.315 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:38.315 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:38.332 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:38.332 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:38.351 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:38.350+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:38.364 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:38.364 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:38.396 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:38.396 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:38.428 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:38.428 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:38.447 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:38.448 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:38.448 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:38.449 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:38.452 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:38.451+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:38.458 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:49:38.464 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:38.464 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:38.480 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:38.481 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:38.497 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:38.498 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:38.514 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:38.515 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:38.531 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:38.531 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:38.547 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:38.547 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:38.553 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:38.551+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:38.571 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:38.572 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:38.581 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:49:38.653 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:38.653+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:38.728 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:38.729 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:38.737 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:49:38.755 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:38.753+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:38.844 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:38.845 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:38.853 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:49:38.855 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:38.854+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:38.957 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:38.956+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:38.960 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:38.960 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:38.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:49:38.973 INFO:tasks.rados.rados.0.smithi040.stdout:346: done (5 left) 2024-09-16T14:49:38.973 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 39 current snap is 32 2024-09-16T14:49:38.974 INFO:tasks.rados.rados.0.smithi040.stdout:342: finishing write tid 1 to smithi04040151-26 2024-09-16T14:49:38.974 INFO:tasks.rados.rados.0.smithi040.stdout:342: finishing write tid 2 to smithi04040151-26 2024-09-16T14:49:38.974 INFO:tasks.rados.rados.0.smithi040.stdout:342: finishing write tid 3 to smithi04040151-26 2024-09-16T14:49:38.974 INFO:tasks.rados.rados.0.smithi040.stdout:342: finishing write tid 4 to smithi04040151-26 2024-09-16T14:49:38.974 INFO:tasks.rados.rados.0.smithi040.stdout:342: finishing write tid 5 to smithi04040151-26 2024-09-16T14:49:38.974 INFO:tasks.rados.rados.0.smithi040.stdout:344: expect (ObjNum 3 snap 0 seq_num 3) 2024-09-16T14:49:38.983 INFO:tasks.rados.rados.0.smithi040.stdout:343: expect (ObjNum 68 snap 4 seq_num 68) 2024-09-16T14:49:38.990 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:38.988+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.051 INFO:tasks.rados.rados.0.smithi040.stdout:342: finishing write tid 6 to smithi04040151-26 2024-09-16T14:49:39.051 INFO:tasks.rados.rados.0.smithi040.stdout:342: oid 26 updating version 0 to 146 2024-09-16T14:49:39.052 INFO:tasks.rados.rados.0.smithi040.stdout:342: oid 26 version 146 is already newer than 144 2024-09-16T14:49:39.052 INFO:tasks.rados.rados.0.smithi040.stdout:342: oid 26 updating version 146 to 148 2024-09-16T14:49:39.052 INFO:tasks.rados.rados.0.smithi040.stdout:342: oid 26 version 148 is already newer than 145 2024-09-16T14:49:39.052 INFO:tasks.rados.rados.0.smithi040.stdout:342: oid 26 version 148 is already newer than 147 2024-09-16T14:49:39.052 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 26 v 148 (ObjNum 124 snap 32 seq_num 124) dirty exists 2024-09-16T14:49:39.052 INFO:tasks.rados.rados.0.smithi040.stdout:342: left oid 26 (ObjNum 124 snap 32 seq_num 124) 2024-09-16T14:49:39.052 INFO:tasks.rados.rados.0.smithi040.stdout:340: finishing copy_from racing read to smithi04040151-27 2024-09-16T14:49:39.052 INFO:tasks.rados.rados.0.smithi040.stdout:340: finishing copy_from to smithi04040151-27 2024-09-16T14:49:39.052 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 27 v 52 (ObjNum 111 snap 25 seq_num 111) dirty exists 2024-09-16T14:49:39.052 INFO:tasks.rados.rados.0.smithi040.stdout:347: seq_num 125 ranges {2441216=557056,2998272=475136,3473408=507904,3981312=475136,4456448=98304} 2024-09-16T14:49:39.057 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:39.057+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:39.067 INFO:tasks.rados.rados.0.smithi040.stdout:347: writing smithi04040151-39 from 2441216 to 2998272 tid 1 2024-09-16T14:49:39.070 INFO:tasks.rados.rados.0.smithi040.stdout:347: writing smithi04040151-39 from 2998272 to 3473408 tid 2 2024-09-16T14:49:39.073 INFO:tasks.rados.rados.0.smithi040.stdout:347: writing smithi04040151-39 from 3473408 to 3981312 tid 3 2024-09-16T14:49:39.076 INFO:tasks.rados.rados.0.smithi040.stdout:347: writing smithi04040151-39 from 3981312 to 4456448 tid 4 2024-09-16T14:49:39.077 INFO:tasks.rados.rados.0.smithi040.stdout:347: writing smithi04040151-39 from 4456448 to 4554752 tid 5 2024-09-16T14:49:39.077 INFO:tasks.rados.rados.0.smithi040.stdout:340: done (5 left) 2024-09-16T14:49:39.077 INFO:tasks.rados.rados.0.smithi040.stdout:342: done (4 left) 2024-09-16T14:49:39.077 INFO:tasks.rados.rados.0.smithi040.stdout:343: done (3 left) 2024-09-16T14:49:39.077 INFO:tasks.rados.rados.0.smithi040.stdout:344: done (2 left) 2024-09-16T14:49:39.077 INFO:tasks.rados.rados.0.smithi040.stdout:345: done (1 left) 2024-09-16T14:49:39.078 INFO:tasks.rados.rados.0.smithi040.stdout:348: read oid 44 snap -1 2024-09-16T14:49:39.078 INFO:tasks.rados.rados.0.smithi040.stdout:348: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-16T14:49:39.078 INFO:tasks.rados.rados.0.smithi040.stdout:347: finishing write tid 1 to smithi04040151-39 2024-09-16T14:49:39.078 INFO:tasks.rados.rados.0.smithi040.stdout:347: finishing write tid 2 to smithi04040151-39 2024-09-16T14:49:39.078 INFO:tasks.rados.rados.0.smithi040.stdout:349: snap_create 2024-09-16T14:49:39.080 INFO:tasks.rados.rados.0.smithi040.stdout:347: finishing write tid 3 to smithi04040151-39 2024-09-16T14:49:39.082 INFO:tasks.rados.rados.0.smithi040.stdout:347: finishing write tid 4 to smithi04040151-39 2024-09-16T14:49:39.082 INFO:tasks.rados.rados.0.smithi040.stdout:347: finishing write tid 5 to smithi04040151-39 2024-09-16T14:49:39.082 INFO:tasks.rados.rados.0.smithi040.stdout:347: finishing write tid 6 to smithi04040151-39 2024-09-16T14:49:39.083 INFO:tasks.rados.rados.0.smithi040.stdout:348: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-16T14:49:39.098 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:39.098 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:39.109 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:49:39.113 INFO:tasks.rados.rados.0.smithi040.stdout:347: finishing write tid 7 to smithi04040151-39 2024-09-16T14:49:39.114 INFO:tasks.rados.rados.0.smithi040.stdout:347: oid 39 updating version 0 to 69 2024-09-16T14:49:39.114 INFO:tasks.rados.rados.0.smithi040.stdout:347: oid 39 version 69 is already newer than 66 2024-09-16T14:49:39.114 INFO:tasks.rados.rados.0.smithi040.stdout:347: oid 39 updating version 69 to 70 2024-09-16T14:49:39.114 INFO:tasks.rados.rados.0.smithi040.stdout:347: oid 39 version 70 is already newer than 67 2024-09-16T14:49:39.114 INFO:tasks.rados.rados.0.smithi040.stdout:347: oid 39 version 70 is already newer than 65 2024-09-16T14:49:39.114 INFO:tasks.rados.rados.0.smithi040.stdout:347: oid 39 version 70 is already newer than 68 2024-09-16T14:49:39.114 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 39 v 70 (ObjNum 125 snap 32 seq_num 125) dirty exists 2024-09-16T14:49:39.114 INFO:tasks.rados.rados.0.smithi040.stdout:347: left oid 39 (ObjNum 125 snap 32 seq_num 125) 2024-09-16T14:49:39.158 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:39.156+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:39.220 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:39.220 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:39.228 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:49:39.269 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.267+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:39.333 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:39.333 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:39.342 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:49:39.385 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:39.384+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:39.443 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:39.444 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:39.452 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:49:39.461 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:39.460+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:39.562 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.561+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:39.594 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:39.594 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:39.603 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:49:39.663 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:39.662+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:39.745 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:39.745 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:39.753 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:49:39.765 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:39.764+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:39.854 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:39.855 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:39.861 INFO:tasks.rados.rados.0.smithi040.stdout:347: done (2 left) 2024-09-16T14:49:39.861 INFO:tasks.rados.rados.0.smithi040.stdout:348: done (1 left) 2024-09-16T14:49:39.861 INFO:tasks.rados.rados.0.smithi040.stdout:349: done (0 left) 2024-09-16T14:49:39.861 INFO:tasks.rados.rados.0.smithi040.stdout:350: read oid 36 snap -1 2024-09-16T14:49:39.861 INFO:tasks.rados.rados.0.smithi040.stdout:350: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-16T14:49:39.862 INFO:tasks.rados.rados.0.smithi040.stdout:351: snap_remove snap 32 2024-09-16T14:49:39.866 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.861+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.866 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.861+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.866 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.861+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.866 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.861+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.866 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.861+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.866 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.861+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.866 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.861+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.867 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.861+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.867 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.861+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.867 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.861+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.867 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.861+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.867 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.861+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.867 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.861+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.867 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.861+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.867 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.867 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.867 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.867 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.867 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.867 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.868 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.868 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.868 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.868 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.868 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.868 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.868 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.868 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.868 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.868 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.868 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.868 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.868 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.868 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.869 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.869 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.869 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.869 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.869 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.869 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.869 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.869 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.869 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.869 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.869 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.869 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.869 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.870 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.871 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.871 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.871 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.871 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.871 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.871 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.871 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.871 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.871 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.871 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.871 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.871 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.871 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.872 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.872 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.872 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.872 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.872 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.872 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.872 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.872 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.872 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.872 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.872 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.872 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.872 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.872 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.873 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.873 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.873 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.873 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.873 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.862+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.873 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.863+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.873 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:39.863+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:39.875 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.876 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.876 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.876 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.876 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.876 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.876 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.876 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.876 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.876 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.876 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.876 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.876 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.876 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.877 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.877 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.877 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.877 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.877 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.877 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.877 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.877 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.877 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.877 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.877 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.877 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.877 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.862+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.878 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.879 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.879 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.879 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.879 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.879 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.879 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.879 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.879 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.879 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.879 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.879 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.879 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.879 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.879 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.880 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.880 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.880 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.880 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.880 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.880 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.880 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.880 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.880 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.880 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.880 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.880 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.880 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.881 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.881 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.881 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.881 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.881 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.881 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.881 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.881 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.881 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.881 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.881 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.881 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.881 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.881 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.882 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.883 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.883 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.883 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.883 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.883 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.883 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.883 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.883 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.883 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.883 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.883 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.883 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.883 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.884 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.884 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.884 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.884 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.884 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.884 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.884 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.884 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.884 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.884 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.884 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.884 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.884 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.884 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.885 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.885 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.885 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.863+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.885 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.885 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.885 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.885 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.885 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.885 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.885 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.885 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.885 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.885 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.886 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.886 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.886 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.886 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.886 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.886 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.886 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.886 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.886 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.886 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.886 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.886 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.886 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.886 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.887 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.887 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.887 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.887 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.887 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.887 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.887 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.887 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.887 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.887 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.887 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.887 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.888 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.862+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.888 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.862+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.888 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.862+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.888 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.862+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.888 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.862+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.888 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.862+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.888 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.862+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.888 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.862+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.888 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.862+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.889 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.862+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.889 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.862+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.889 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.862+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.889 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.862+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.889 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.889 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.889 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.889 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.889 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.889 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.889 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.889 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.889 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.890 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.890 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.890 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.890 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.890 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.890 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.890 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.890 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.890 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.890 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.890 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.890 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.891 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.891 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.891 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.891 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.891 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.891 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.891 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.891 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.891 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.891 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.891 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.891 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.891 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.892 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.892 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.892 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.892 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.892 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.892 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.892 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.892 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.892 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.892 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.892 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.863+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.892 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.864+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.892 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.864+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.893 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.864+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.893 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.864+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.893 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.864+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.893 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.864+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.893 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.864+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.893 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.864+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.893 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.864+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.893 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.864+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.893 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.864+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.893 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.864+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.893 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.865+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.893 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.865+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.893 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.865+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.894 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.865+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.894 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.865+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.894 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.865+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.894 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.865+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.894 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.865+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.894 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.865+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.894 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.865+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.894 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.865+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.894 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.894 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.894 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.894 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.894 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.894 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.895 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.895 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.895 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.895 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.895 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.895 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.895 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.895 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.895 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.895 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.895 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.895 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.895 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.895 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.866+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.896 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.897 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.897 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.897 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.897 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.897 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.897 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.897 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.897 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.897 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.897 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.898 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.898 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.898 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.898 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.898 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.898 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.899 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.899 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.899 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.899 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.899 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.899 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.899 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.899 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.899 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.899 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.899 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.899 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.899 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.899 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.900 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.900 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.900 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.900 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.900 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.900 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.900 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.900 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.900 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.900 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.864+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.900 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.900 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.900 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.901 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.901 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.901 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.901 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.901 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.901 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.901 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.901 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.901 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.901 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.901 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.901 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.901 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.901 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.902 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.902 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.902 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.902 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.902 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.902 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.902 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.902 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.902 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.902 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.902 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.902 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.902 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.902 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.903 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.903 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.903 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.903 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.903 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.903 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.903 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.903 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.903 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.903 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.903 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.903 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.903 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.904 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.904 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.865+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.904 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.904 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.904 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.904 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.904 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.904 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.904 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.904 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.904 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.904 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.904 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.905 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.905 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.905 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.905 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.905 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.905 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.905 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.905 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.905 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.905 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.905 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.905 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.905 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.905 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.906 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.906 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.906 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.906 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.906 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.906 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.906 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.906 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.906 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.906 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.906 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.906 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.906 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.907 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.907 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.907 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.907 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.907 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.907 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.907 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.907 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.907 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.907 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.907 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.907 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.866+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.907 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.908 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.908 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.908 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.908 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.908 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.908 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.908 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.908 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.908 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.908 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.909 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.909 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.909 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.909 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.909 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.909 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.909 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.909 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.909 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.909 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.909 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.867+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.909 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.868+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.909 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.868+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.909 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.868+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.910 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.910 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.910 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.910 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.910 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.910 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.911 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.911 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.911 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.911 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.911 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.911 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.911 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.911 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.911 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.911 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.867+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.911 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.912 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.868+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.912 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.868+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.912 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.868+0000 7fbcc3722640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 2,3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.913 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.913 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.913 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.913 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.913 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.913 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.913 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.913 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.913 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.913 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.913 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.914 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.914 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.914 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.914 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.914 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.914 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.914 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.914 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.914 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.914 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.914 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.914 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.914 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.915 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.915 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.915 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.915 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.915 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.915 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.915 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.915 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.915 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.915 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.915 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.915 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.915 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.916 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.916 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.916 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.916 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.916 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.916 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.916 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.916 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.916 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.916 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.916 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.916 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.868+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.916 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.917 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.917 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.917 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.917 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.917 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.917 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.917 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.917 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.917 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.917 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.917 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.917 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.917 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.917 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.918 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.918 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.918 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.918 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.918 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.918 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.918 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.918 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.918 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.918 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.918 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.918 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.918 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.918 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.919 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.920 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.920 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.920 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.920 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.920 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.920 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.920 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.920 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.920 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.920 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.920 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.920 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.920 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.921 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.921 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.921 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.921 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.921 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.921 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.921 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.921 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.921 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.921 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.921 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.921 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.921 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.921 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.922 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.922 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.922 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.922 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.922 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.922 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.922 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.922 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.922 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.922 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.922 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.922 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.922 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.923 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.923 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.923 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.923 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.923 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.923 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.923 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.923 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.923 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.923 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.923 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.923 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.923 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.923 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.924 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.869+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.924 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.924 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.924 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.924 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.924 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.924 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.924 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.924 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.924 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.924 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.924 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.925 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.925 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.925 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.925 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.925 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.925 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.925 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.925 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.925 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.925 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.925 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.925 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.925 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.925 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.926 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.926 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.926 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.926 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.926 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.926 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.926 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.926 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.926 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.926 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.926 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.926 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.926 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.926 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.927 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.927 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.927 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.927 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.927 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.927 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.870+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,3,4,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.928 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:49:39.929 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.921+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.929 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.929 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.929 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.930 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.930 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.930 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.930 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.930 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.930 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.930 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.930 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.930 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.930 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.930 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.930 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.931 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.931 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.931 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.931 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.931 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.931 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.931 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.931 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.931 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.931 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.931 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.931 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.931 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.864+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.931 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.933 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.933 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.933 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.933 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.933 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.933 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.933 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.933 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.933 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.933 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.933 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.933 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.933 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.933 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.934 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.934 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.934 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.934 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.934 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.934 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.934 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.934 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.934 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.934 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.934 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.934 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.934 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.865+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.936 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.936 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.936 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.936 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.936 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.936 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.936 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.936 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.936 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.936 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.937 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.937 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.937 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.937 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.937 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.937 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.937 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.937 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.937 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.937 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.937 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.937 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.937 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.938 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.938 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.938 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.938 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.938 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.938 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.938 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.938 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.938 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.938 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.938 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.938 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.938 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.938 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.939 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.939 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.939 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.939 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.939 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.939 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.939 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.939 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.939 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.939 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.939 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.939 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.939 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.940 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.940 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.940 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.940 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.940 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.940 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.940 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.940 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.940 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.940 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.940 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.940 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.940 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.940 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.941 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.941 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.941 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.941 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.941 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.941 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.941 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.941 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.941 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.941 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.941 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.941 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.942 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.942 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.942 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.942 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.942 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.942 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.942 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.942 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.942 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.942 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.942 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.942 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.942 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.942 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.943 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.943 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.943 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.943 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.943 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.867+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.943 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.943 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.943 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.943 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.943 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.943 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.943 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.943 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.944 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.944 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.944 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.944 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.944 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.944 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.944 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.944 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.944 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.945 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.946 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.946 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.946 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.946 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.946 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.946 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.946 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.946 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.946 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.946 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.947 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.947 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.947 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.947 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.947 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.947 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.947 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.947 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.947 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.947 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.948 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.948 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.948 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.948 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.948 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.948 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.948 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.948 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.948 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.948 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.948 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.948 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.948 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.949 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.949 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.949 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.949 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.949 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.949 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.949 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.949 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.949 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.950 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.950 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.950 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.950 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.950 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.950 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.950 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.950 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.950 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.950 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.950 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.950 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.951 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.951 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.951 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.951 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.951 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.951 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.951 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.951 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.951 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.952 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.952 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.952 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.952 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.952 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.952 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.952 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.952 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.952 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.952 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.952 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.952 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.952 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.953 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.953 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.953 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.953 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.953 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.953 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.953 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.953 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.953 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.953 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.953 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.953 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.953 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.953 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.954 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.954 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.954 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.954 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.954 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:39.941+0000 7fbcbf71a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.956 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.956 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.956 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.956 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.956 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.956 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.956 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.956 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.956 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.956 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.956 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.956 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.956 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.957 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.957 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.957 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.957 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.957 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.957 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.957 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.957 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.957 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.957 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.957 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.957 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.957 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.957 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.958 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.958 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.958 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.958 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.958 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.958 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.958 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.958 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.958 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.958 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.866+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.959 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.960 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.960 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.960 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.960 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.960 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.960 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.960 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.960 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.961 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.961 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.961 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.962 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.962 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.962 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.962 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.962 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.962 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.962 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.962 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.962 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.962 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.962 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.962 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.969 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.969 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.969 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.969 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.969 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.970 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.970 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.970 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.970 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.970 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.970 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.970 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.970 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.970 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.970 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.970 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.970 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.970 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.971 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.971 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.971 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.971 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.971 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.971 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.971 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.971 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.971 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.971 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.971 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.971 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.971 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.971 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.973 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.974 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.974 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.974 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.974 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.974 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.974 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.974 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.974 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.974 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.974 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.974 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.974 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.975 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.975 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.975 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.975 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.975 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.975 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.975 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.975 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.975 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.975 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.975 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.975 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.975 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.976 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.976 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.976 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.976 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.976 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.976 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.976 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.976 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.976 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.976 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.976 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.976 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.976 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.977 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.977 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.977 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.977 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.977 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.977 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.977 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.977 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.977 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.978 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.978 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.978 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.868+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.978 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.978 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.978 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.980 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.980 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.980 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.980 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.966+0000 7f408158d640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.980 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:39.965+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:39.981 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.981 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.981 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.981 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.982 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.982 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.982 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.982 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.982 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.982 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.983 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.983 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.983 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.983 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.983 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.983 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.983 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.983 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.983 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.983 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.983 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.983 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.983 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.983 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.984 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.984 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.984 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.984 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.984 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.984 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.984 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.984 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.869+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.984 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.984 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.984 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.984 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.984 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.985 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.985 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.985 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.985 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.985 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.985 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.985 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.985 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.985 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.985 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.985 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:49:39.993 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:39.988+0000 7f4085595640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.998 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.998 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.998 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.998 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.999 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.999 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.999 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.999 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.999 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.999 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.999 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.999 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.999 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.999 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.999 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.999 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:39.999 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.000 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.000 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.000 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.000 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.000 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.000 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.000 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.000 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.000 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.000 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.000 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.000 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.000 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.000 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.001 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.001 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.001 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.001 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.001 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.001 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.001 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.001 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.001 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.001 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.001 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.001 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.001 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.002 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.002 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.002 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.002 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.002 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.002 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.002 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.002 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.002 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.002 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.002 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.002 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.002 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.002 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.003 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.003 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.003 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.003 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.003 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.003 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.003 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.003 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.003 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.003 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.003 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.003 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.003 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.004 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.004 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.004 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.004 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.004 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.004 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.004 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.004 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.004 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.004 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.004 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.004 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.004 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.004 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.005 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.005 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.005 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.005 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.005 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.005 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.005 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.005 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.005 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.005 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.005 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.005 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.005 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.005 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.006 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.006 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.006 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.006 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.006 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.006 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.006 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.006 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.006 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.006 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.006 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.006 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.006 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.007 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.007 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.007 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.007 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.007 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.007 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.007 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.007 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.007 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.007 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.007 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.007 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.007 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.007 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.008 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.008 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.008 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.008 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.008 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.008 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.008 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.008 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.008 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.008 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.008 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.008 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.009 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.009 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.009 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.009 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.009 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.009 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.009 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.009 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.009 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.009 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.009 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.009 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.009 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.009 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.010 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.010 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.010 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.010 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.010 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.010 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.010 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.010 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.010 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.010 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.010 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.010 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.010 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.011 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.011 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.011 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.011 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.011 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.011 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.011 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.011 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.011 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.011 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.011 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.011 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.011 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.012 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.012 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.012 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.012 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.012 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.012 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.012 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.012 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.012 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.012 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.012 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.012 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.012 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.012 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.013 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.013 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.013 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.013 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.013 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.013 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.013 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.013 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.013 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.013 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.013 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.013 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.013 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.014 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.014 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.014 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.014 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.014 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.014 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.014 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.014 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.014 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.014 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.014 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.014 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.014 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.014 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.015 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.015 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.015 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.015 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.015 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.015 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.015 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.015 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.015 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.015 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.015 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.015 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.015 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.015 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.016 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.016 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.016 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.016 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.016 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.016 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.016 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.016 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.016 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.016 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.016 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.016 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.016 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.017 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.017 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.017 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.017 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.017 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.017 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.017 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.017 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.017 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.017 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.017 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.017 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.017 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.017 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.018 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.018 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.018 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.018 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.018 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.018 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.018 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.018 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.018 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.018 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.018 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.018 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.018 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.018 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.019 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.019 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.019 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.019 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.019 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.019 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.019 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.019 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.019 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.019 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.019 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.019 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.019 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.020 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.020 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.020 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.020 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.020 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.020 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.020 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.020 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.020 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.020 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.020 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.020 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.020 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.020 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.021 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.021 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.021 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.021 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.021 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.021 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.021 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.021 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.021 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.021 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.021 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.021 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.021 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.022 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.022 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.022 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.022 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.022 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.022 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.022 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.022 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.022 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.022 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.022 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.023 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.023 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.023 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.023 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.023 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.024 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.024 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.024 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.024 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.024 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.024 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.024 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.024 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.025 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.025 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.025 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.025 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.025 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.025 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.025 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.025 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.025 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.026 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.026 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.026 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.026 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.026 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.026 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.026 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.026 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.027 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.027 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.027 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.027 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.027 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.027 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.027 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.027 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.028 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.028 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.028 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.028 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.028 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.029 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.029 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.029 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.029 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.029 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.029 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.029 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.029 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.029 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.029 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.029 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.029 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.029 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.030 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.030 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.030 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.030 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.030 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.030 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.030 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.031 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.031 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.031 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.877+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.031 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.031 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.031 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.031 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.031 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.032 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.032 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.032 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.032 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.032 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.032 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.032 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.870+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.033 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.033 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.033 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.033 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.033 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.033 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.033 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.034 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.034 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.034 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.034 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.034 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.034 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.034 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.035 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.035 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.035 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.035 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.035 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.036 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.036 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.036 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.036 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.036 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.036 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.036 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.036 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.037 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.037 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.037 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.037 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.037 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.037 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.037 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.037 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.038 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.038 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.038 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.038 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.038 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.038 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.038 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.039 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.039 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.039 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.039 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.039 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.039 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.039 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.039 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.040 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.040 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.040 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.040 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.040 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.040 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.040 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.041 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.041 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.041 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.041 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.041 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.041 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.042 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.042 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.042 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.042 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.042 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.042 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.042 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.042 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.043 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.043 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.043 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.043 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.043 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.043 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.043 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.878+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.051 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.051 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.051 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.051 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.052 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.052 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.052 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.052 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.052 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.052 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.052 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.052 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.052 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.052 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.052 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.052 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.052 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.871+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.053 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.054 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.054 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.054 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.054 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.054 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.054 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.054 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.054 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.054 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.054 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.054 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.054 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.054 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.054 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.055 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.056 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.056 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.056 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.056 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.056 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.056 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.056 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.056 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.056 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.056 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.056 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.056 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.057 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.057 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.057 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.057 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.057 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.057 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.057 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.057 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.057 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.057 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.057 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.057 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.872+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.057 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.057 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.058 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.058 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.058 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.058 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.058 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.058 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.058 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.058 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.058 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.058 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.058 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.058 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.058 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.058 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.059 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.059 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.059 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.059 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.059 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.059 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.059 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.059 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.059 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.059 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.059 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.059 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.059 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.059 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.060 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.060 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.060 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.060 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.060 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.060 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.060 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.060 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.060 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.060 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.060 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.060 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.060 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.061 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.061 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.061 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.061 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.061 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.061 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.061 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.061 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.061 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.061 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.061 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.061 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.061 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.062 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.062 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.062 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.062 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.062 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.062 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.062 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.062 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.062 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.062 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.873+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.062 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.062 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.062 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.062 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.063 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.063 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.063 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.063 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.063 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.063 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.063 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.063 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.063 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.063 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.063 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.063 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.063 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.063 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.064 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.064 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.064 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.064 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.064 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.064 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.064 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.064 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.064 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.064 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.064 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.064 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.064 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.065 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.065 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.065 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.065 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.065 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.065 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.065 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.065 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.065 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.065 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.065 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.065 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.065 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.066 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.066 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.066 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.066 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.874+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.066 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.066 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.875+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.066 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.066 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.066 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.066 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.066 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.066 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.066 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.066 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.067 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.067 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.067 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.067 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.067 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.067 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.067 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.067 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.876+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,1,2,3,4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.067 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:39.904+0000 7f7cafb91640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.068 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:39.990+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.068 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:40.004+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.068 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:40.023+0000 7f7de5c82640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.069 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:40.029+0000 7f7cb3b99640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,4 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.069 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:40.035+0000 7f7de1c7a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:40.070 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:40.068+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:40.073 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:40.073 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:40.082 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:49:40.169 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:40.168+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:40.226 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:40.227 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:40.235 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:49:40.279 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:40.278+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:40.337 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:40.337 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:40.346 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:49:40.373 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:40.371+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:40.473 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:40.472+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:40.489 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:40.489 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:40.497 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:49:40.559 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:49:40.600 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:40.600 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:40.604 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:40.602+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:40.608 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:49:40.675 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.673+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:40.685 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:40.685 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:40.702 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:40.702 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:40.711 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:40.711 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:40.718 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:40.719 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:49:40.734 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:40.734 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:40.750 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:40.750 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:40.767 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:40.767 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:40.776 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:40.775+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:40.784 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:40.784 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:40.801 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:40.801 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:40.818 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:40.819 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:40.824 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:40.824 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:40.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:40.824 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:40.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.824 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:295 3.9s0 3:96d2b183:::smithi04040151-29:head [copy-from ver 0] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T14:49:40.824 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:40.824 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 56.905750978, 2024-09-16T14:49:40.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010216596999999999, 2024-09-16T14:49:40.824 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.825 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:40.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:40.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:40.825 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:40.825 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 295 2024-09-16T14:49:40.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.825 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:40.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:40.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915430+0000", 2024-09-16T14:49:40.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:40.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915441+0000", 2024-09-16T14:49:40.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0789e-05 2024-09-16T14:49:40.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915462+0000", 2024-09-16T14:49:40.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0845000000000001e-05 2024-09-16T14:49:40.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915473+0000", 2024-09-16T14:49:40.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0805000000000001e-05 2024-09-16T14:49:40.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915528+0000", 2024-09-16T14:49:40.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4772999999999999e-05 2024-09-16T14:49:40.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:40.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915537+0000", 2024-09-16T14:49:40.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6700000000000006e-06 2024-09-16T14:49:40.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.919813+0000", 2024-09-16T14:49:40.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042757840000000004 2024-09-16T14:49:40.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:40.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.919907+0000", 2024-09-16T14:49:40.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504999999999993e-05 2024-09-16T14:49:40.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925649+0000", 2024-09-16T14:49:40.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057427870000000001 2024-09-16T14:49:40.829 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.829 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.829 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.830 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:296 3.9s0 3:96d2b183:::smithi04040151-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-16T14:49:40.830 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:40.830 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 56.905699061, 2024-09-16T14:49:40.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010423623, 2024-09-16T14:49:40.830 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.830 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:40.830 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:40.830 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:40.830 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:40.830 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 296 2024-09-16T14:49:40.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.830 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:40.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:40.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915484+0000", 2024-09-16T14:49:40.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:40.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915489+0000", 2024-09-16T14:49:40.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6079999999999998e-06 2024-09-16T14:49:40.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915496+0000", 2024-09-16T14:49:40.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7589999999999997e-06 2024-09-16T14:49:40.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915506+0000", 2024-09-16T14:49:40.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0340000000000002e-06 2024-09-16T14:49:40.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915588+0000", 2024-09-16T14:49:40.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2198000000000001e-05 2024-09-16T14:49:40.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:40.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915594+0000", 2024-09-16T14:49:40.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:49:40.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920099+0000", 2024-09-16T14:49:40.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045048700000000002 2024-09-16T14:49:40.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:40.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920147+0000", 2024-09-16T14:49:40.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8239999999999999e-05 2024-09-16T14:49:40.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925726+0000", 2024-09-16T14:49:40.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055787529999999997 2024-09-16T14:49:40.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:40.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925816+0000", 2024-09-16T14:49:40.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0185000000000005e-05 2024-09-16T14:49:40.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925908+0000", 2024-09-16T14:49:40.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1948999999999997e-05 2024-09-16T14:49:40.835 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.835 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.836 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.836 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as4 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:40.836 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:40.836 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 56.897144410000003, 2024-09-16T14:49:40.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088260140000000001, 2024-09-16T14:49:40.836 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.836 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:40.836 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:40.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:40.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:40.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:40.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924062+0000", 2024-09-16T14:49:40.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2892e-05 2024-09-16T14:49:40.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924070+0000", 2024-09-16T14:49:40.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8690000000000004e-06 2024-09-16T14:49:40.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924076+0000", 2024-09-16T14:49:40.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0739999999999998e-06 2024-09-16T14:49:40.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924119+0000", 2024-09-16T14:49:40.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2549999999999997e-05 2024-09-16T14:49:40.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:40.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924126+0000", 2024-09-16T14:49:40.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8960000000000003e-06 2024-09-16T14:49:40.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:40.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932835+0000", 2024-09-16T14:49:40.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087089200000000002 2024-09-16T14:49:40.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932865+0000", 2024-09-16T14:49:40.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0054000000000001e-05 2024-09-16T14:49:40.840 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.840 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.840 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.840 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es1 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:40.840 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:40.840 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 56.897029822, 2024-09-16T14:49:40.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087127249999999993, 2024-09-16T14:49:40.841 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.841 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:40.841 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:40.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:40.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:40.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:40.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924168+0000", 2024-09-16T14:49:40.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4800000000000001e-05 2024-09-16T14:49:40.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924176+0000", 2024-09-16T14:49:40.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.25e-06 2024-09-16T14:49:40.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924182+0000", 2024-09-16T14:49:40.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3640000000000004e-06 2024-09-16T14:49:40.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924229+0000", 2024-09-16T14:49:40.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6477999999999998e-05 2024-09-16T14:49:40.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:40.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924238+0000", 2024-09-16T14:49:40.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4429999999999992e-06 2024-09-16T14:49:40.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:40.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932848+0000", 2024-09-16T14:49:40.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086097910000000003 2024-09-16T14:49:40.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932867+0000", 2024-09-16T14:49:40.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8927999999999999e-05 2024-09-16T14:49:40.845 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.845 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.845 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.845 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:394 3.9s0 3:93498a0b:::smithi04040151-32:head [append 0~491520] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:40.845 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:40.845 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.951611341000003, 2024-09-16T14:49:40.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016461614999999999, 2024-09-16T14:49:40.845 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.846 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:40.846 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:40.846 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:40.846 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:40.846 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 394 2024-09-16T14:49:40.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.846 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:40.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:40.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869570+0000", 2024-09-16T14:49:40.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:40.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872427+0000", 2024-09-16T14:49:40.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028569070000000001 2024-09-16T14:49:40.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872450+0000", 2024-09-16T14:49:40.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2943e-05 2024-09-16T14:49:40.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872461+0000", 2024-09-16T14:49:40.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0882e-05 2024-09-16T14:49:40.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872521+0000", 2024-09-16T14:49:40.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9639000000000003e-05 2024-09-16T14:49:40.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:40.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872530+0000", 2024-09-16T14:49:40.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8810000000000002e-06 2024-09-16T14:49:40.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.878309+0000", 2024-09-16T14:49:40.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057788520000000001 2024-09-16T14:49:40.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:40.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.878434+0000", 2024-09-16T14:49:40.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012550899999999999 2024-09-16T14:49:40.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:40.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880126+0000", 2024-09-16T14:49:40.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016919019999999999 2024-09-16T14:49:40.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:40.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885887+0000", 2024-09-16T14:49:40.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057608959999999997 2024-09-16T14:49:40.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:40.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885957+0000", 2024-09-16T14:49:40.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0504000000000001e-05 2024-09-16T14:49:40.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886034+0000", 2024-09-16T14:49:40.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6661000000000003e-05 2024-09-16T14:49:40.852 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.852 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.853 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.853 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:395 3.9s0 3:93498a0b:::smithi04040151-32:head [append 0~770048] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:40.853 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:40.853 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.946883925999998, 2024-09-16T14:49:40.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.018911853999999999, 2024-09-16T14:49:40.853 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.853 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:40.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:40.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:40.853 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:40.853 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 395 2024-09-16T14:49:40.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.854 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:40.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:40.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874298+0000", 2024-09-16T14:49:40.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:40.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875671+0000", 2024-09-16T14:49:40.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0013727069999999999 2024-09-16T14:49:40.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875689+0000", 2024-09-16T14:49:40.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7832000000000001e-05 2024-09-16T14:49:40.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875700+0000", 2024-09-16T14:49:40.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1267000000000001e-05 2024-09-16T14:49:40.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875758+0000", 2024-09-16T14:49:40.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8060999999999999e-05 2024-09-16T14:49:40.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:40.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875766+0000", 2024-09-16T14:49:40.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9580000000000007e-06 2024-09-16T14:49:40.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881004+0000", 2024-09-16T14:49:40.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052375169999999997 2024-09-16T14:49:40.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:40.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881098+0000", 2024-09-16T14:49:40.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3875999999999994e-05 2024-09-16T14:49:40.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:40.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882500+0000", 2024-09-16T14:49:40.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014019600000000001 2024-09-16T14:49:40.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:40.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886041+0000", 2024-09-16T14:49:40.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035415210000000002 2024-09-16T14:49:40.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:40.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893136+0000", 2024-09-16T14:49:40.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070949330000000003 2024-09-16T14:49:40.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893212+0000", 2024-09-16T14:49:40.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5739000000000001e-05 2024-09-16T14:49:40.861 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.861 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.861 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.861 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:396 3.9s0 3:93498a0b:::smithi04040151-32:head [setxattr _header (57) in=7b] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:40.861 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:40.862 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.945466771, 2024-09-16T14:49:40.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017781597999999999, 2024-09-16T14:49:40.862 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.862 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:40.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:40.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:40.862 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:40.862 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 396 2024-09-16T14:49:40.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.862 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:40.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:40.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875715+0000", 2024-09-16T14:49:40.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:40.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875721+0000", 2024-09-16T14:49:40.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6989999999999997e-06 2024-09-16T14:49:40.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875728+0000", 2024-09-16T14:49:40.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9040000000000003e-06 2024-09-16T14:49:40.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875734+0000", 2024-09-16T14:49:40.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4519999999999996e-06 2024-09-16T14:49:40.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875800+0000", 2024-09-16T14:49:40.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6538000000000006e-05 2024-09-16T14:49:40.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:40.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875804+0000", 2024-09-16T14:49:40.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.771e-06 2024-09-16T14:49:40.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883232+0000", 2024-09-16T14:49:40.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074282009999999997 2024-09-16T14:49:40.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883323+0000", 2024-09-16T14:49:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0740000000000002e-05 2024-09-16T14:49:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884057+0000", 2024-09-16T14:49:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00073422299999999999 2024-09-16T14:49:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893020+0000", 2024-09-16T14:49:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089623210000000005 2024-09-16T14:49:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893403+0000", 2024-09-16T14:49:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00038375799999999999 2024-09-16T14:49:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893499+0000", 2024-09-16T14:49:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5284000000000005e-05 2024-09-16T14:49:40.868 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.869 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.869 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.869 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:397 3.9s0 3:93498a0b:::smithi04040151-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:40.869 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:40.869 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.945435646, 2024-09-16T14:49:40.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019838090999999999, 2024-09-16T14:49:40.869 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.869 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:40.869 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:40.869 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:40.869 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:40.870 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 397 2024-09-16T14:49:40.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.870 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:40.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:40.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:40.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:40.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875750+0000", 2024-09-16T14:49:40.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3939999999999999e-06 2024-09-16T14:49:40.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875755+0000", 2024-09-16T14:49:40.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.31e-06 2024-09-16T14:49:40.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875760+0000", 2024-09-16T14:49:40.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4769999999999997e-06 2024-09-16T14:49:40.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875824+0000", 2024-09-16T14:49:40.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3893000000000006e-05 2024-09-16T14:49:40.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:40.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875829+0000", 2024-09-16T14:49:40.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3489999999999999e-06 2024-09-16T14:49:40.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885652+0000", 2024-09-16T14:49:40.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0098232530000000005 2024-09-16T14:49:40.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:40.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885748+0000", 2024-09-16T14:49:40.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5366999999999997e-05 2024-09-16T14:49:40.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893876+0000", 2024-09-16T14:49:40.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081279479999999994 2024-09-16T14:49:40.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:40.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893967+0000", 2024-09-16T14:49:40.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0777000000000002e-05 2024-09-16T14:49:40.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895586+0000", 2024-09-16T14:49:40.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016197029999999999 2024-09-16T14:49:40.875 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.875 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.875 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.875 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(osd.3.17:4 3.9s0 3:9b721ec3:::smithi04040151-12:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:40.876 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:40.876 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.937213352000001, 2024-09-16T14:49:40.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011299052, 2024-09-16T14:49:40.876 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.876 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:40.876 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:40.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:40.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883969+0000", 2024-09-16T14:49:40.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:40.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883976+0000", 2024-09-16T14:49:40.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3730000000000004e-06 2024-09-16T14:49:40.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883999+0000", 2024-09-16T14:49:40.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3390000000000001e-05 2024-09-16T14:49:40.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884007+0000", 2024-09-16T14:49:40.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1489999999999997e-06 2024-09-16T14:49:40.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886141+0000", 2024-09-16T14:49:40.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0021347240000000002 2024-09-16T14:49:40.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:40.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886230+0000", 2024-09-16T14:49:40.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8234999999999998e-05 2024-09-16T14:49:40.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895269+0000", 2024-09-16T14:49:40.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090399230000000001 2024-09-16T14:49:40.879 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.879 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.880 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.880 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:440 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:40.880 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:40.880 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.945160031, 2024-09-16T14:49:40.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01253772, 2024-09-16T14:49:40.880 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.880 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:40.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:40.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:40.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:40.881 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 440 2024-09-16T14:49:40.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.881 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:40.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:40.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876022+0000", 2024-09-16T14:49:40.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:40.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876033+0000", 2024-09-16T14:49:40.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1474000000000001e-05 2024-09-16T14:49:40.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876051+0000", 2024-09-16T14:49:40.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7628000000000001e-05 2024-09-16T14:49:40.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876060+0000", 2024-09-16T14:49:40.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339999999999996e-06 2024-09-16T14:49:40.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876128+0000", 2024-09-16T14:49:40.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7460999999999997e-05 2024-09-16T14:49:40.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:40.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876138+0000", 2024-09-16T14:49:40.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8649999999999996e-06 2024-09-16T14:49:40.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882141+0000", 2024-09-16T14:49:40.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060036550000000001 2024-09-16T14:49:40.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:40.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882233+0000", 2024-09-16T14:49:40.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1626000000000007e-05 2024-09-16T14:49:40.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.888561+0000", 2024-09-16T14:49:40.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063284769999999999 2024-09-16T14:49:40.885 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.885 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.885 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.885 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:441 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:40.885 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:40.885 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.945112973999997, 2024-09-16T14:49:40.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.013066019999999999, 2024-09-16T14:49:40.886 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.886 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:40.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:40.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:40.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:40.886 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 441 2024-09-16T14:49:40.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.886 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:40.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:40.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876070+0000", 2024-09-16T14:49:40.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:40.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876074+0000", 2024-09-16T14:49:40.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1330000000000001e-06 2024-09-16T14:49:40.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876083+0000", 2024-09-16T14:49:40.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2519999999999999e-06 2024-09-16T14:49:40.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876088+0000", 2024-09-16T14:49:40.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5509999999999999e-06 2024-09-16T14:49:40.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876235+0000", 2024-09-16T14:49:40.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014688900000000001 2024-09-16T14:49:40.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:40.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876243+0000", 2024-09-16T14:49:40.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5490000000000001e-06 2024-09-16T14:49:40.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.884127+0000", 2024-09-16T14:49:40.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078848489999999993 2024-09-16T14:49:40.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:40.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.884238+0000", 2024-09-16T14:49:40.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111007 2024-09-16T14:49:40.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889137+0000", 2024-09-16T14:49:40.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048983739999999996 2024-09-16T14:49:40.890 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.891 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.891 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.891 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:442 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:40.891 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:40.891 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 42.945074316000003, 2024-09-16T14:49:40.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014710668, 2024-09-16T14:49:40.891 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.891 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:40.891 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:40.891 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:40.891 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:40.892 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 442 2024-09-16T14:49:40.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.892 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:40.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:40.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:40.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:40.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876113+0000", 2024-09-16T14:49:40.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0759999999999996e-06 2024-09-16T14:49:40.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876119+0000", 2024-09-16T14:49:40.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3339999999999998e-06 2024-09-16T14:49:40.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876126+0000", 2024-09-16T14:49:40.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8410000000000002e-06 2024-09-16T14:49:40.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876273+0000", 2024-09-16T14:49:40.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014714900000000001 2024-09-16T14:49:40.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:40.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876279+0000", 2024-09-16T14:49:40.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7819999999999999e-06 2024-09-16T14:49:40.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.885397+0000", 2024-09-16T14:49:40.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091178149999999996 2024-09-16T14:49:40.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:40.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.885492+0000", 2024-09-16T14:49:40.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5384999999999996e-05 2024-09-16T14:49:40.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.890820+0000", 2024-09-16T14:49:40.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053278559999999997 2024-09-16T14:49:40.896 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.896 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.896 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.896 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs3 61/27 ECSubWrite(tid=238, reqid=osd.3.0:238, at_version=61'42, trim_to=0'0, roll_forward_to=60'38))", 2024-09-16T14:49:40.897 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:40.897 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.894315532, 2024-09-16T14:49:40.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089143629999999998, 2024-09-16T14:49:40.897 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.897 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:40.897 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:40.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:40.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:40.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:40.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926896+0000", 2024-09-16T14:49:40.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7764999999999999e-05 2024-09-16T14:49:40.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926909+0000", 2024-09-16T14:49:40.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3454e-05 2024-09-16T14:49:40.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926918+0000", 2024-09-16T14:49:40.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7099999999999996e-06 2024-09-16T14:49:40.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926968+0000", 2024-09-16T14:49:40.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0778000000000001e-05 2024-09-16T14:49:40.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:40.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926989+0000", 2024-09-16T14:49:40.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0069999999999999e-05 2024-09-16T14:49:40.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:40.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935750+0000", 2024-09-16T14:49:40.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.008761876 2024-09-16T14:49:40.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935783+0000", 2024-09-16T14:49:40.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2202000000000001e-05 2024-09-16T14:49:40.901 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.901 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.901 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.901 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s6 61/27 ECSubWrite(tid=239, reqid=osd.3.0:239, at_version=61'79, trim_to=0'0, roll_forward_to=60'75))", 2024-09-16T14:49:40.901 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:40.901 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.894252735000002, 2024-09-16T14:49:40.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.009451849, 2024-09-16T14:49:40.901 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.902 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:40.902 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:40.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:40.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:40.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:40.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926951+0000", 2024-09-16T14:49:40.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9959999999999999e-05 2024-09-16T14:49:40.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926962+0000", 2024-09-16T14:49:40.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1498000000000001e-05 2024-09-16T14:49:40.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926970+0000", 2024-09-16T14:49:40.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5410000000000003e-06 2024-09-16T14:49:40.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.927857+0000", 2024-09-16T14:49:40.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00088758200000000004 2024-09-16T14:49:40.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:40.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.927868+0000", 2024-09-16T14:49:40.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1234e-05 2024-09-16T14:49:40.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:40.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936355+0000", 2024-09-16T14:49:40.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0084864520000000002 2024-09-16T14:49:40.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936383+0000", 2024-09-16T14:49:40.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8068e-05 2024-09-16T14:49:40.906 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.906 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.906 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.906 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:709 3.9s0 3:96d2b183:::smithi04040151-29:head [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:40.906 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.900189+0000", 2024-09-16T14:49:40.906 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 16.920995261000002, 2024-09-16T14:49:40.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011070020999999999, 2024-09-16T14:49:40.906 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.906 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:40.906 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:40.906 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:40.907 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:40.907 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 709 2024-09-16T14:49:40.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.907 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900189+0000", 2024-09-16T14:49:40.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900189+0000", 2024-09-16T14:49:40.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900188+0000", 2024-09-16T14:49:40.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:40.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900192+0000", 2024-09-16T14:49:40.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7670000000000001e-06 2024-09-16T14:49:40.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900196+0000", 2024-09-16T14:49:40.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5480000000000003e-06 2024-09-16T14:49:40.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900201+0000", 2024-09-16T14:49:40.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5310000000000003e-06 2024-09-16T14:49:40.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900321+0000", 2024-09-16T14:49:40.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120096 2024-09-16T14:49:40.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:40.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900327+0000", 2024-09-16T14:49:40.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3389999999999998e-06 2024-09-16T14:49:40.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.907572+0000", 2024-09-16T14:49:40.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0072446180000000004 2024-09-16T14:49:40.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:40.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.907672+0000", 2024-09-16T14:49:40.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9617000000000005e-05 2024-09-16T14:49:40.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.911259+0000", 2024-09-16T14:49:40.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035870350000000001 2024-09-16T14:49:40.911 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.911 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.911 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.911 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs3 81/65 ECSubWrite(tid=337, reqid=osd.3.0:337, at_version=81'65, trim_to=0'0, roll_forward_to=80'61))", 2024-09-16T14:49:40.911 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:27.927767+0000", 2024-09-16T14:49:40.911 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.893417246, 2024-09-16T14:49:40.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088756419999999996, 2024-09-16T14:49:40.912 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.912 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:40.912 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927767+0000", 2024-09-16T14:49:40.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927767+0000", 2024-09-16T14:49:40.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927766+0000", 2024-09-16T14:49:40.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:40.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927792+0000", 2024-09-16T14:49:40.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5914999999999998e-05 2024-09-16T14:49:40.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927809+0000", 2024-09-16T14:49:40.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7926000000000001e-05 2024-09-16T14:49:40.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927820+0000", 2024-09-16T14:49:40.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0277e-05 2024-09-16T14:49:40.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930629+0000", 2024-09-16T14:49:40.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00280919 2024-09-16T14:49:40.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:40.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930637+0000", 2024-09-16T14:49:40.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3000000000000002e-06 2024-09-16T14:49:40.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:40.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.936614+0000", 2024-09-16T14:49:40.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059770400000000003 2024-09-16T14:49:40.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.936642+0000", 2024-09-16T14:49:40.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7911000000000001e-05 2024-09-16T14:49:40.916 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.916 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.916 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.916 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:822 3.19s0 3:9b72282b:::smithi04040151-27:head [copy-from ver 52] snapc 3b=[3b,3a,38,37] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T14:49:40.916 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.944334+0000", 2024-09-16T14:49:40.916 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.8768493189999997, 2024-09-16T14:49:40.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.073023423, 2024-09-16T14:49:40.916 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.916 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:40.916 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:40.916 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:40.917 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:40.917 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 822 2024-09-16T14:49:40.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.917 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944334+0000", 2024-09-16T14:49:40.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944334+0000", 2024-09-16T14:49:40.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944332+0000", 2024-09-16T14:49:40.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:40.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944343+0000", 2024-09-16T14:49:40.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0324e-05 2024-09-16T14:49:40.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944361+0000", 2024-09-16T14:49:40.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8354000000000001e-05 2024-09-16T14:49:40.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944372+0000", 2024-09-16T14:49:40.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1123e-05 2024-09-16T14:49:40.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958524+0000", 2024-09-16T14:49:40.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014151403999999999 2024-09-16T14:49:40.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:40.920 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958527+0000", 2024-09-16T14:49:40.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.101e-06 2024-09-16T14:49:40.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.970154+0000", 2024-09-16T14:49:40.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0116273200000001 2024-09-16T14:49:40.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:40.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.970333+0000", 2024-09-16T14:49:40.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017934799999999999 2024-09-16T14:49:40.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:40.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.005943+0000", 2024-09-16T14:49:40.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035610044 2024-09-16T14:49:40.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:40.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.014246+0000", 2024-09-16T14:49:40.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083026430000000002 2024-09-16T14:49:40.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:40.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017305+0000", 2024-09-16T14:49:40.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003059291 2024-09-16T14:49:40.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017358+0000", 2024-09-16T14:49:40.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2552000000000001e-05 2024-09-16T14:49:40.923 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.923 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.923 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.924 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:823 3.19s0 3:9b72282b:::smithi04040151-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90)", 2024-09-16T14:49:40.924 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:40.924 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.8767973229999999, 2024-09-16T14:49:40.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0666982119999999, 2024-09-16T14:49:40.924 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.924 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:40.924 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:40.924 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:40.924 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:40.924 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 823 2024-09-16T14:49:40.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.924 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:40.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:40.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:40.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:40.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944389+0000", 2024-09-16T14:49:40.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9960000000000002e-06 2024-09-16T14:49:40.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944399+0000", 2024-09-16T14:49:40.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8179999999999999e-06 2024-09-16T14:49:40.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944407+0000", 2024-09-16T14:49:40.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.456e-06 2024-09-16T14:49:40.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958549+0000", 2024-09-16T14:49:40.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014142050999999999 2024-09-16T14:49:40.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:40.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958550+0000", 2024-09-16T14:49:40.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1200000000000002e-07 2024-09-16T14:49:40.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971413+0000", 2024-09-16T14:49:40.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.012862798 2024-09-16T14:49:40.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:40.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971466+0000", 2024-09-16T14:49:40.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3282999999999998e-05 2024-09-16T14:49:40.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010860+0000", 2024-09-16T14:49:40.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.039393862000000002 2024-09-16T14:49:40.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:40.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010968+0000", 2024-09-16T14:49:40.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108378 2024-09-16T14:49:40.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.011085+0000", 2024-09-16T14:49:40.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116402 2024-09-16T14:49:40.929 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.929 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.929 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.930 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs2 90/27 ECSubWrite(tid=310, reqid=osd.7.0:310, at_version=90'48, trim_to=0'0, roll_forward_to=89'46))", 2024-09-16T14:49:40.930 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.953355+0000", 2024-09-16T14:49:40.930 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.867828662, 2024-09-16T14:49:40.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081091860000000009, 2024-09-16T14:49:40.930 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.930 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:40.930 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.930 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.930 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.930 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.953355+0000", 2024-09-16T14:49:40.930 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.953355+0000", 2024-09-16T14:49:40.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.953355+0000", 2024-09-16T14:49:40.931 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:40.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.931 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.931 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.931 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.953378+0000", 2024-09-16T14:49:40.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3226000000000001e-05 2024-09-16T14:49:40.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.953386+0000", 2024-09-16T14:49:40.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0250000000000002e-06 2024-09-16T14:49:40.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.932 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.932 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.932 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.953394+0000", 2024-09-16T14:49:40.932 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1929999999999993e-06 2024-09-16T14:49:40.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.955101+0000", 2024-09-16T14:49:40.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0017067060000000001 2024-09-16T14:49:40.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:40.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.955109+0000", 2024-09-16T14:49:40.933 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1759999999999993e-06 2024-09-16T14:49:40.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.933 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.933 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:40.933 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.961439+0000", 2024-09-16T14:49:40.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006330263 2024-09-16T14:49:40.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.934 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.934 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.961464+0000", 2024-09-16T14:49:40.934 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4868999999999999e-05 2024-09-16T14:49:40.934 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.934 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.934 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.934 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.934 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es1 90/27 ECSubWrite(tid=301, reqid=osd.11.0:301, at_version=90'62, trim_to=0'0, roll_forward_to=83'60))", 2024-09-16T14:49:40.934 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.956242+0000", 2024-09-16T14:49:40.934 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.864942272, 2024-09-16T14:49:40.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083679859999999991, 2024-09-16T14:49:40.935 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:40.935 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:40.935 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:40.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:40.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956242+0000", 2024-09-16T14:49:40.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.935 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.935 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:40.935 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956242+0000", 2024-09-16T14:49:40.935 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:40.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:40.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956240+0000", 2024-09-16T14:49:40.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:40.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.936 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:40.936 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956267+0000", 2024-09-16T14:49:40.936 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6101999999999998e-05 2024-09-16T14:49:40.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.936 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:40.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956279+0000", 2024-09-16T14:49:40.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2118e-05 2024-09-16T14:49:40.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:40.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956290+0000", 2024-09-16T14:49:40.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1311000000000001e-05 2024-09-16T14:49:40.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.937 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.937 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:40.937 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956492+0000", 2024-09-16T14:49:40.937 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000201795 2024-09-16T14:49:40.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:40.938 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956501+0000", 2024-09-16T14:49:40.938 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1060000000000006e-06 2024-09-16T14:49:40.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.938 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.938 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:40.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.964580+0000", 2024-09-16T14:49:40.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0080795469999999994 2024-09-16T14:49:40.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:40.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:40.939 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:40.939 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.964609+0000", 2024-09-16T14:49:40.939 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9043e-05 2024-09-16T14:49:40.939 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.939 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.939 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.939 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:40.939 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:40.939 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:40.940 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:40.941 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:40.942 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:40.942 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:40.942 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:49:40.943 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:40.940+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:40.943 INFO:tasks.rados.rados.0.smithi040.stdout:351: done (1 left) 2024-09-16T14:49:40.943 INFO:tasks.rados.rados.0.smithi040.stdout:352: snap_remove snap 28 2024-09-16T14:49:40.943 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is unset 2024-09-16T14:49:40.960 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 6] 2024-09-16T14:49:40.960 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T14:49:41.011 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.011 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.011 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.011 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.012 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.012 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.012 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.012 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.012 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.012 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.012 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.012 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.012 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.012 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.012 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.012 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.012 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.013 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.013 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.013 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.013 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.013 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.013 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.013 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.013 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.013 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.013 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.013 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.013 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.013 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.014 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.015 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.860+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.015 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.015 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.015 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.015 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.015 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.015 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.015 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.015 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.015 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.015 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.015 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.015 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.016 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.016 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.016 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.016 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.016 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.016 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.016 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.016 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.016 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.016 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.016 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.016 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.016 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.017 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.017 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.017 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.017 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.017 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.017 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.017 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.017 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.017 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.017 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.017 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.018 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.018 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.018 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.018 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.018 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.018 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.018 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.018 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.018 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.018 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.018 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.019 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.019 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.019 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.019 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.019 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.019 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.019 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.019 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.019 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.019 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.019 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.019 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.019 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.020 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.020 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.020 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.020 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.020 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.020 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.020 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.020 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.020 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.020 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.020 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.020 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.021 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.021 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.021 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.021 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.021 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.021 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.021 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.021 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.021 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.021 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.021 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.021 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.021 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.022 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.022 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.022 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.022 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.022 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.022 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.022 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.022 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.022 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.022 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.022 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.022 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.022 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.023 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.023 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.023 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.023 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.023 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.023 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.023 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.023 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.023 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.023 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.023 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.023 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.023 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.024 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.024 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.024 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.024 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.024 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.024 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.024 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.024 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.024 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.024 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.024 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.024 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.024 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.025 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.025 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.025 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.025 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.025 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.025 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.025 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.025 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.025 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.025 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.025 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.025 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.025 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.026 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.026 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.026 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.026 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.026 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.026 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.026 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.026 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.026 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.026 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.026 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.026 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.026 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.027 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.027 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.027 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.027 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.027 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.027 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.027 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.027 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.027 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.027 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.027 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.027 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.027 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.027 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.028 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.028 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.028 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.028 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.028 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.028 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.028 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.028 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.028 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.028 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.028 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.028 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.028 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.029 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.029 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.029 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.029 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.029 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.029 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.029 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.029 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.029 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.861+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.029 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.029 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.029 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.029 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.029 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.030 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.030 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.030 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.030 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.030 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.030 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.030 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.030 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.030 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.030 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.030 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.030 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.030 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.031 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.031 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.031 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.031 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.031 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.031 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.031 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.031 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.031 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.031 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.031 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.031 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.031 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.032 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.032 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.032 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.032 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.032 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.032 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.032 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.032 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.032 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.032 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.032 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.032 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.032 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.033 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.033 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.033 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.033 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.033 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.033 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.033 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.033 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.033 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.033 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.033 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.033 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.033 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.034 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.034 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.034 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.034 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.034 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.034 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.034 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.034 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.034 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.034 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.034 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.034 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.034 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.035 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.035 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.035 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.035 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.035 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.035 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.035 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.035 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.035 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.035 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.035 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.035 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.035 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.036 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.036 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.036 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.036 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.036 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.036 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.036 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.036 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.036 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.036 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.036 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.036 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.036 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.037 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.037 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.037 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.037 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.037 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.037 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.037 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.037 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.037 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.037 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.037 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.037 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.037 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.038 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.038 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.038 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.038 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.038 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.038 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.038 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.038 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.038 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.038 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.038 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.039 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.039 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.039 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.039 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.039 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.039 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.039 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.039 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.039 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.039 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.039 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.039 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.039 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.039 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.040 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.040 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.040 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.040 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.040 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.040 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.040 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.040 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.040 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.040 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.040 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.040 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.041 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.041 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.041 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.041 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.041 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.041 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.041 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.041 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.041 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.041 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.041 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.041 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.042 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.042 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.042 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.042 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.042 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.042 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.042 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.862+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.042 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.042 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.042 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.042 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.042 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.042 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.042 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.043 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.043 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.043 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.043 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.043 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.043 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.043 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.043 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.043 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.043 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.043 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.043 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.043 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.043 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.044 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.044 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.044 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.044 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.044 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.044 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.044 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.044 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.044 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.044 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.044 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.044 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.044 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.044 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.045 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.046 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.046 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.046 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.046 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.046 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.046 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.046 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.046 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.046 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.046 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.046 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.046 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.046 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.046 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.047 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.047 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.047 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.047 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.047 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.047 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.047 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.047 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.047 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.047 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.047 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.047 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.047 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.048 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.048 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.048 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.048 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.048 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.048 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.048 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.048 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.048 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.048 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.048 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.048 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.048 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.049 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.049 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.049 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.049 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.049 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.049 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.049 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.049 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.049 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.049 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.049 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.049 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.049 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.050 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.050 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.050 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.050 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.050 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.050 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.050 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.050 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.050 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.050 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.050 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.050 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.050 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.050 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.051 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.051 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.051 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.051 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.051 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.051 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.051 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.051 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.051 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.051 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.051 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.051 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.052 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.052 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.052 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.052 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.052 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.052 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.052 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.052 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.052 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.052 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.052 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.052 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.052 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.053 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.053 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.053 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.053 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.053 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.053 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.053 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.053 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.053 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.053 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.053 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.053 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.053 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.055 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.055 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.055 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.055 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.055 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.863+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.055 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.055 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.055 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.055 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.055 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.055 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.055 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.055 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.055 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.056 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.056 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.056 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.056 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.056 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.056 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.056 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.056 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.056 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.056 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.056 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.056 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.057 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.057 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.057 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.057 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.057 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.057 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.057 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.057 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.057 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.057 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.057 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.057 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.057 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.058 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.058 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.058 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.058 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.058 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.058 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.058 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.058 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.058 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.058 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.058 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.058 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.058 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.059 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.059 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.059 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.059 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.059 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.059 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.059 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.059 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.059 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.059 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.059 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.059 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.059 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.060 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.060 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.060 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.060 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.060 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.060 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.060 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.060 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.060 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.060 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.060 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.060 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.060 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.061 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.061 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.061 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.061 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.061 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.061 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.061 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.061 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.061 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.061 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.061 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.061 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.062 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.062 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.062 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.062 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.062 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.062 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.062 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.062 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.062 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.062 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.062 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.062 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.062 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.062 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.063 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.063 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.063 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.063 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.063 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.063 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.063 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.063 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.063 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.063 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.063 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.063 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.063 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.063 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.064 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.064 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.064 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.064 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.064 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.064 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.064 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.064 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.064 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.064 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.064 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.064 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.065 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.065 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.065 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.065 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.065 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.065 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.065 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.065 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.065 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.065 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.065 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.065 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.065 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.067 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.067 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.067 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.864+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.067 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.067 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.067 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.067 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.067 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.067 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.067 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.067 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.067 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.067 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.068 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.068 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.068 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.068 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.068 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.068 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.068 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.068 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.068 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.068 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.068 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.068 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.068 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.068 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.069 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.069 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.069 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.069 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.069 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.069 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.069 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.069 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.069 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.069 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.069 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.069 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.069 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.069 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.070 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.070 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.070 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.070 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.070 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.070 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.070 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.070 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.070 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.070 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.070 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.070 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.070 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.070 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.071 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.071 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.071 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.071 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.071 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.071 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.071 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.071 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.071 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.071 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.071 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.071 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.071 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.071 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.072 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.072 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.072 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.072 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.072 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.072 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.072 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.072 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.072 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.072 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.073 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.073 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.073 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.073 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.073 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.073 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.073 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.073 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.073 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.073 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.073 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.073 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.073 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.074 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.074 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.074 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.074 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.074 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.074 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.074 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.074 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.074 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.074 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.074 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.074 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.075 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.075 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.075 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.075 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.075 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.075 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.075 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.075 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.075 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.075 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.075 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.076 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.076 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.076 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.076 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.076 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.076 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.076 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.076 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.076 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.076 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.076 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.076 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.077 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.077 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.077 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.077 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.077 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.077 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.077 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.077 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.077 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.077 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.077 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.077 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.077 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.077 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.078 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.078 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.078 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.078 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.078 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.078 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.078 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.078 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.078 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.078 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.078 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.078 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.078 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.079 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.079 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.079 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.079 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.079 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.079 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.079 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.079 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.079 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.079 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.079 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.079 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.080 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.080 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.080 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.080 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.080 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.080 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.080 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.080 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.080 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.080 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.080 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.080 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.080 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.865+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.080 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.081 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.081 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.081 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.081 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.081 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.081 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.081 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.081 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.081 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.081 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.081 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.081 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.081 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.082 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.082 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.082 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.082 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.082 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.082 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.082 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.082 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.082 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.082 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.082 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.082 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.082 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.082 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.083 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.083 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.083 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.083 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.083 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.083 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.083 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.083 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.083 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.083 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.083 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.083 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.083 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.083 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.084 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.084 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.084 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.084 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.084 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.084 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.084 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.084 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.084 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.084 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.084 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.084 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.084 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.085 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.085 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.085 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.085 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.085 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.085 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.085 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.085 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.085 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.085 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.085 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.085 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.085 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.085 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.087 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.087 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.087 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.087 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.087 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.087 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.087 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.087 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.087 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.087 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.087 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.087 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.088 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.088 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.088 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.088 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.088 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.088 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.088 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.088 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.088 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.088 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.088 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.088 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.088 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.089 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.089 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.089 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.089 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.089 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.089 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.089 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.089 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.089 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.089 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.089 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.089 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.089 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.089 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.090 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.090 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.867+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.090 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.090 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.090 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.090 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.090 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.090 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.866+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.090 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.090 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.090 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.090 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.090 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.091 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.091 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.091 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.091 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.091 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.091 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.091 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.091 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.091 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.091 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.091 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.091 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.091 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.092 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.092 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.092 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.092 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.092 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.092 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.092 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.092 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.092 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.092 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.092 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.092 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.092 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.093 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.093 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.093 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.093 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.093 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.093 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.093 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.093 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.093 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.093 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.093 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.093 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.093 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.094 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.094 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.094 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.094 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.094 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.094 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.094 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.094 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.094 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.094 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.094 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.095 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.095 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.095 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.095 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.095 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.095 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.095 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.095 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.095 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.095 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.095 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.096 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.096 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.096 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.096 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.096 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.096 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.096 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.096 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.096 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.096 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.096 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.097 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.097 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.097 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.097 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.097 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.097 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.097 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.097 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.097 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.097 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.097 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.097 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.097 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.098 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.098 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.098 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.098 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.098 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.098 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.098 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.098 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.098 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.869+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.098 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.098 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.098 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.098 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.098 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.099 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.099 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.099 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.099 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.099 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.099 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.099 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.099 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.099 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.099 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.099 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.099 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.099 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.099 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.100 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.100 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.100 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.100 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.100 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.100 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.100 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.100 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.100 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.100 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.100 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.100 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.100 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.101 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.101 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.101 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.101 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.101 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.101 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.101 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.101 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.101 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.101 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.101 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.101 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.101 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.101 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.102 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.102 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.102 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.102 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.102 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.102 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.102 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.102 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.102 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.102 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.102 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.102 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.102 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.103 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.103 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.103 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.103 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.103 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.103 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.103 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.103 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.103 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.103 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.103 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.103 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.103 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.103 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.104 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.104 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.104 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.104 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.104 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.104 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.104 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.104 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.104 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.104 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.104 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.104 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.104 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.105 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.105 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.105 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.105 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.105 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.105 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.105 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.105 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.105 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.105 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.105 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.105 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.106 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.106 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.106 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.106 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.106 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.106 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.106 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.106 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.106 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.106 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.106 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.106 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.106 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.107 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.107 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.107 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.107 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.107 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.107 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.107 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.107 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.107 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 1,2,4,5,6 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.107 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.107 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.107 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.107 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.108 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.108 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.108 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.108 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.108 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.108 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.108 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.108 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.108 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.108 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.108 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.108 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.109 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.109 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.109 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.109 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.109 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.109 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.109 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.110 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.110 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.110 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.110 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.110 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.110 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.110 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.110 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.110 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.110 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.110 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.111 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.111 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.111 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.111 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.111 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.111 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.111 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.111 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.111 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.111 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.111 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.111 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.111 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.112 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.112 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.112 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.112 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.112 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.112 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.112 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.112 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.112 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.112 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.112 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.112 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.112 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.113 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.113 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.113 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.113 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.113 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.113 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.113 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.113 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.113 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.113 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.113 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.113 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.113 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.114 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.114 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.114 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.114 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.114 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.114 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.114 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.870+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.114 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.871+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.114 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.114 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.114 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.114 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.114 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.115 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.115 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.115 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.115 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.115 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.115 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.115 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.115 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.115 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.115 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.115 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.115 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.116 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.116 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.116 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.116 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.116 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.116 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.116 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.116 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.116 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.116 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.116 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.116 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.116 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.117 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.117 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.117 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.117 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.117 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.117 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.117 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.117 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.117 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.117 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.117 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.117 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.117 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.118 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.118 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.118 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.118 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.118 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.118 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.118 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.118 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.118 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.119 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.119 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.119 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.119 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.119 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.119 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.119 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.119 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.119 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.119 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.119 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.119 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.119 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.120 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.120 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.120 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.120 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.120 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.120 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.120 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.120 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.120 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.120 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.120 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.120 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.120 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.121 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.121 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.121 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.121 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.872+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.121 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.121 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.121 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.121 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.121 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.121 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.121 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.121 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.121 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.122 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.122 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.122 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.122 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.122 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.122 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.122 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.122 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.122 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.122 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.122 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.122 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.122 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.122 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.123 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.123 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.123 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.123 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.123 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.123 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.123 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.123 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.123 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.123 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.123 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.123 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.123 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.124 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.124 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.124 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.124 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.124 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.124 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.124 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.124 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.124 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.124 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.124 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.124 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.124 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.125 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.125 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.125 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.125 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.125 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.125 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.125 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.125 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.125 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.125 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.125 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.125 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.125 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.126 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.126 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.126 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.126 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.126 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.126 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.126 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.126 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.126 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.126 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.126 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.126 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.126 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.127 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.127 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.127 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.127 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.127 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.127 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.127 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.127 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.127 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.127 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.127 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.127 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.127 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.128 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.128 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.128 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.128 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.128 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.128 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.128 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.128 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.128 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.128 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.128 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.128 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.128 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.129 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.129 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.129 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.129 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.129 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.129 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.129 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.129 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.129 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.129 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.129 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.129 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.129 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.130 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.130 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.130 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.130 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.130 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.130 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.130 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.130 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.130 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.130 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.873+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.131 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.131 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.131 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.131 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.131 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.131 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.131 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.131 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.131 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.131 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.131 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.131 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.131 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.132 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.132 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.132 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.132 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.132 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.132 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.132 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.132 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.132 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.132 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.132 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.132 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.132 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.132 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.133 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.133 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.133 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.133 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.133 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.133 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.133 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.133 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.133 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.133 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.133 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.133 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.133 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.134 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.134 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.134 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.134 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.134 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.134 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.134 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.134 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.134 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.134 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.134 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.134 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.134 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.134 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.135 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.135 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.135 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.135 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.135 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.135 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.135 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.135 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.135 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.135 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.135 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.135 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.135 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.135 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.136 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.136 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.136 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.136 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.136 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.136 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.136 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.136 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.136 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.136 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.136 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.136 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.136 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.137 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.137 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.137 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.137 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.137 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.137 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.137 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.137 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.137 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.137 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.137 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.137 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.137 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.138 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.138 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.138 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.138 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.138 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.138 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.138 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.138 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.138 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.138 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.138 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.138 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.138 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.139 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.139 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.139 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.139 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.139 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.139 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.139 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.139 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.139 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.139 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.139 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.139 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.139 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.139 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.140 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.140 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.140 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.140 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.140 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.140 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.140 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.140 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.140 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.140 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.140 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.140 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.140 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.141 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.141 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.141 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.141 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.874+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.141 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.141 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.141 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.141 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.141 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.141 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.141 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.141 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.141 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.142 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.142 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.142 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.142 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.142 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.142 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.142 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.142 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.142 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.142 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.142 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.142 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.142 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.143 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.144 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.144 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.144 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.144 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.144 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.144 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.144 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.144 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.144 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.144 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.144 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.144 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.145 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.145 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.145 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.145 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.145 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.145 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.145 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.145 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.145 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.145 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.145 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.145 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.145 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.146 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.875+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.146 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.146 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.146 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.146 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.146 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.146 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.146 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.146 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.146 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.146 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.146 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.147 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.147 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.147 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.147 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.147 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.147 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.147 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.147 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.147 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.147 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.147 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.147 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.147 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.147 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.148 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.148 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.148 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.148 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.148 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.148 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.148 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.148 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.148 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.148 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.148 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.148 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.148 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.149 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.150 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.150 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.150 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.150 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.150 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.150 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.150 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.150 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.150 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.150 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.150 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.150 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.150 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.151 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.151 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.151 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.151 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.151 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.151 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.151 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.151 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.151 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.151 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.151 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.151 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.151 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.152 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.152 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.152 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.152 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.152 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.152 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.152 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.152 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.152 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.152 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.152 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.152 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.152 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.876+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.153 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.154 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.154 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.154 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.154 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.154 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.154 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.154 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.154 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.154 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.154 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.154 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.154 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.154 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.155 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.155 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.155 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.155 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.155 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.155 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.155 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.155 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.155 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.155 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.155 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.155 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.155 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.156 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.156 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.156 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.156 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.156 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.156 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.156 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.156 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.156 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.156 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.156 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.156 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.156 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.157 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.157 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.157 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.157 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.157 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.157 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.157 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.157 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.157 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.157 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.157 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.157 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.157 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.158 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.158 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.158 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.158 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.158 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.158 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.158 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.158 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.158 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.158 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.158 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.158 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.158 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.159 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.159 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.159 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.159 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.159 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.159 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.159 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.159 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.159 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.159 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.159 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.159 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.159 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.160 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.160 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.160 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.160 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.160 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.160 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.160 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.160 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.160 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.160 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.160 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.160 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.160 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.160 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.161 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.161 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.161 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.161 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.161 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.161 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.161 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.161 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.161 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.161 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.161 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.161 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.161 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.161 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.162 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.162 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.162 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.162 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.162 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.162 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.162 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.162 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.162 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:41.163 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:41.165 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:41.164+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:41.166 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:41.167 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:41.167 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:41.167 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:41.167 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:49:41.170 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.170 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.170 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.171 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.171 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.171 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.171 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.171 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.171 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.171 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.171 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.171 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.171 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.171 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.171 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.171 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.877+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.172 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.173 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.173 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.173 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.173 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.173 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.173 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.173 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.173 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.173 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.173 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.173 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.173 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.173 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.174 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.174 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.174 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.174 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.174 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.174 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.174 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.174 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.174 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.174 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.174 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.174 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.174 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.175 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.175 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.175 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.175 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.175 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.175 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.175 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.175 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.175 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.175 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.175 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.175 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.175 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.176 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.176 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.176 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.176 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.176 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.176 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.176 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.176 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.176 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.176 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.176 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.176 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.176 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.177 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.177 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.878+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.177 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.177 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.177 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.177 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.177 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.177 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.177 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.177 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.177 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.177 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.177 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.178 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.178 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.178 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.178 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.178 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.178 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.178 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.178 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.178 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.178 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.178 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.178 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.178 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.178 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.179 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.180 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.180 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.180 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.180 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.180 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.180 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.180 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.180 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.180 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.180 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.180 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.180 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.181 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.181 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.181 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.181 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.181 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.181 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.181 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.181 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.181 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.181 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.181 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.181 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.182 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.183 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.183 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.183 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.183 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.183 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.183 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.183 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.183 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.183 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.183 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.183 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.183 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.183 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.183 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.184 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.184 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.184 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.184 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.184 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.184 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.184 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.184 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.184 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.184 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.184 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.184 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.184 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.185 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.185 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.185 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.185 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.185 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.185 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.185 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.879+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.185 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.185 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.185 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.185 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.185 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.185 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.186 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.186 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.186 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.186 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.186 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.186 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.186 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.186 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.186 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.186 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.186 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.186 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.186 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.187 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.187 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.187 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.187 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.187 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.187 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.187 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.187 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.187 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.187 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.187 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.187 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.187 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.187 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.188 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.188 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.188 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.188 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.188 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.188 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.188 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.188 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.188 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.188 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.188 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.188 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.188 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.189 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.189 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.189 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.189 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.189 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.189 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.189 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.189 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.189 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.189 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.189 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.189 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.189 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.190 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.190 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.190 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.190 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.190 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.190 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.190 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.190 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.190 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.190 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.190 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.190 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.190 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.191 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.191 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.191 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.191 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.191 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.191 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.191 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.191 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.191 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.191 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.191 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.191 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.192 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.192 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.192 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.192 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.192 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.192 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.192 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.192 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.192 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.192 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.192 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.192 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.192 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.193 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.193 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.193 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.193 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.193 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.193 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.193 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.193 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.193 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.193 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.193 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.193 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.193 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.194 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.194 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.194 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.194 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.194 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.194 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.194 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.194 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.194 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.194 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.194 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.194 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.194 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.880+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.195 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.195 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.195 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.195 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.195 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.195 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.195 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.195 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.195 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.195 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.195 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.195 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.195 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.195 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.196 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.196 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.196 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.196 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.196 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.196 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.196 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.196 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.196 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.196 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.196 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.196 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.196 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.197 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.197 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.197 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.197 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.197 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.197 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.197 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.197 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.197 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.197 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.197 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.197 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.197 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.198 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.198 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.198 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.198 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.198 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.198 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.198 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.198 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.198 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.198 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.198 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.198 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.199 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.199 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.199 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.199 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.199 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.199 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.199 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.199 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.199 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.199 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.199 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.199 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.199 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.200 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.200 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.200 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.200 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.200 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.200 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.200 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.200 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.200 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.200 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.200 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.200 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.200 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.201 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.201 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.201 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.201 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.201 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.201 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.201 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.201 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.201 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.201 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.201 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.201 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.201 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.201 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.202 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.202 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.202 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.202 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.202 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.202 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.202 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.202 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.202 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.202 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.202 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.202 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.202 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.203 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.203 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.203 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.203 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.203 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.203 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.203 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.203 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.203 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.203 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.203 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.203 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.203 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.204 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.204 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.204 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.204 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.881+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.204 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.204 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.204 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.204 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.204 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.204 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.204 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.204 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.204 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.205 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.205 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.205 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.205 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.205 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.205 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.205 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.205 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.205 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.205 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.205 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.205 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.205 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.207 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.208 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.208 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.208 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.208 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.208 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.208 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.208 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.208 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.208 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.208 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.208 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.208 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.208 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.209 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.209 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.209 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.209 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.209 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.209 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.209 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.209 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.209 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.210 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.210 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.210 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.210 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.210 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.210 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.210 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.210 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.210 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.210 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.210 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.210 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.211 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.211 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.211 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.211 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.211 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.211 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.211 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.211 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.211 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.211 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.211 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.211 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.211 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.212 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.212 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.212 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.212 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.212 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.212 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.212 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.212 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.212 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.212 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.212 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.212 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.213 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.213 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.213 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.213 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.213 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.213 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.213 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.213 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.213 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.213 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.213 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.213 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.882+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.213 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.214 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.214 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.214 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.214 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.214 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.214 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.214 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.214 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.214 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.214 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.214 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.214 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.214 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.215 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.215 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.215 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.215 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.215 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.215 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.215 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.218 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.218 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.218 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.219 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.219 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.219 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.219 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.219 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.219 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.219 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.219 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.219 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.219 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.219 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.219 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.219 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.220 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.220 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.220 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.220 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.220 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.220 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.220 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.220 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.220 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.220 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.220 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.220 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.220 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.221 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.221 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.221 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.221 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.221 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.221 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.221 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.221 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.221 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.221 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.221 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.221 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.221 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.222 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.222 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.222 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.222 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.222 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.222 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.222 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.222 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.222 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.222 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.222 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.222 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.222 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.222 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.223 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.223 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.223 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.223 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.223 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.223 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.223 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.223 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.223 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.223 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.223 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.223 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.224 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.224 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.224 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.224 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.224 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.224 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.224 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.224 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.224 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.224 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.224 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.224 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.224 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.225 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.226 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.226 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.226 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.226 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.226 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.226 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.226 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.226 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.226 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.226 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.226 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.226 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.226 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.227 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.227 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.227 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.227 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.227 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.227 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.227 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.227 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.227 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.227 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.227 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.227 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.227 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.228 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.228 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.228 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.228 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.228 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.883+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.228 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.228 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.228 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.228 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.228 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.228 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.228 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.228 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.229 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.229 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.229 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.229 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.229 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.229 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.229 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.229 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.229 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.229 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.229 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.229 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.229 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.230 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.230 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.230 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.230 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.230 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.230 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.230 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.230 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.230 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.230 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.230 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.230 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.230 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.231 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.231 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.231 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.231 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.231 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.231 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.231 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.231 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.231 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.231 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.231 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.231 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.231 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.231 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.232 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.232 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.232 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.232 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.232 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.232 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.232 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.232 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.232 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.232 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.232 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.232 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.233 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.233 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.233 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.233 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.233 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.233 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.233 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.233 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.233 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.233 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.233 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.233 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.233 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.234 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.234 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.234 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.234 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.234 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.234 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.234 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.234 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.234 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.234 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.234 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.234 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.234 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.235 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.235 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.235 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.235 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.235 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.884+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.235 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.235 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.235 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.235 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.235 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.235 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.235 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.235 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.236 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.236 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.236 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.236 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.236 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.236 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.236 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.236 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.236 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.236 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.236 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.236 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.236 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.237 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.237 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.237 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.237 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.237 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.237 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.237 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.237 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.237 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.237 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.237 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.237 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.237 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.237 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.238 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.238 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.238 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.238 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.238 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.238 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.238 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.238 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.238 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.238 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.238 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.238 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.238 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.239 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.239 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.239 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.239 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.239 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.239 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.239 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.239 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.239 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.239 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.239 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.239 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.239 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.240 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.240 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.240 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.240 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.240 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.240 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.240 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.240 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.240 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.240 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.240 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.240 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.240 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.241 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.241 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.241 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.241 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.241 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.241 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.241 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.241 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.241 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.241 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.241 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.241 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.241 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.885+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.242 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.243 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.243 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.243 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.243 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.243 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.243 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.243 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.243 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.243 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.243 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.243 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.243 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.243 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.244 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.244 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.244 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.244 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.244 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.244 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.244 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.244 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.244 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.244 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.244 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.244 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.244 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.245 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.245 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.245 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.245 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.245 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.245 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.245 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.245 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.245 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.245 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.245 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.245 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.245 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.245 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.246 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.246 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.246 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.246 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.246 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.246 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.246 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.246 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.246 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.246 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.246 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.246 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.246 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.247 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.247 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.247 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.247 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.247 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.247 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.247 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.247 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.247 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.247 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.247 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.247 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.247 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.248 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.248 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.248 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.248 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.248 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.248 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.248 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.248 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.886+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.248 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.248 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.248 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.248 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.248 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.249 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.249 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.249 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.249 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.249 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.249 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.249 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.249 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.249 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.249 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.249 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.249 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.249 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.250 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.250 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.250 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.250 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.250 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.250 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.250 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.250 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.250 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.250 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.250 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.250 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.250 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.251 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.251 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.251 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.251 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.251 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.251 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.251 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.251 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.251 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.251 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.251 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.251 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.251 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.251 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.252 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.252 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.252 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.252 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.252 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.252 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.252 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.252 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.252 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.252 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.252 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.252 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.252 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.253 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.253 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.253 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.253 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.253 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.253 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.253 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.253 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.253 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.253 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.253 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.253 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.253 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.254 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.254 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.254 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.254 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.254 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.254 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.254 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.254 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.254 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.254 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.254 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.887+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.254 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.255 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.255 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.255 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.255 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.255 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.255 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.255 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.255 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.255 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.255 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.255 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.255 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.255 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.255 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.256 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.256 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.256 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.256 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.256 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.256 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.256 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.256 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.256 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.256 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.256 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.256 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.256 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.257 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.257 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.257 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.257 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.257 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.257 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.257 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.257 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.257 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.257 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.257 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.257 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.257 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.257 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.258 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.258 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.258 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.258 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.258 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.258 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.258 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.258 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.258 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.258 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.258 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.258 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.258 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.259 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.259 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.259 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.259 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.259 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.259 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.259 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.259 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.259 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.259 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.259 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.259 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.259 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.259 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.260 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.260 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.260 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.260 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.260 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.260 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.260 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.260 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.260 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.260 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.260 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.260 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.260 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.261 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.261 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.261 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.261 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.261 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.261 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.261 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.261 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.261 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.261 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.261 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.261 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.261 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.262 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.262 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.262 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.262 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.262 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.262 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.262 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.262 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.262 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.888+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.262 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.262 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.262 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.262 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.262 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.263 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.263 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.263 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.263 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.263 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.263 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.263 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.263 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.263 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.263 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.263 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.263 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.263 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.264 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.264 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.264 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.264 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.264 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.264 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.264 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.264 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.264 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.264 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.264 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.264 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.265 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.265 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.265 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.265 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.265 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.265 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.265 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.265 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.265 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.265 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.265 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.265 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.265 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.265 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.266 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.266 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.266 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.266 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.266 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.266 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.266 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.266 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.266 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.266 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.266 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.266 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.266 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.267 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.267 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.267 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.267 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.267 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.267 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.267 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.267 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.267 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.267 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.267 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.267 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.267 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.267 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.268 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.268 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.268 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.268 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.268 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.268 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.268 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.268 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.268 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.268 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.268 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.268 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.268 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.269 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.270 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.271 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.271 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.271 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.271 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.271 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.271 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.271 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.271 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.271 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.271 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.272 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.272 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.272 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.272 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.272 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.272 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.272 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.272 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.272 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.272 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.272 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.272 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.272 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.889+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.274 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.274 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.274 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.274 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.274 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.274 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.274 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.274 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.274 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.274 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.274 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.274 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.274 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.275 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.275 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.275 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.275 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.275 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.275 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.275 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.275 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.275 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.275 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.275 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.275 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.275 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.276 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.277 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.277 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:40.890+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 3,4,5,6,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.277 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:41.145+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.277 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:41.203+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.278 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:41.243+0000 7f550cf0f640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.279 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:41.276+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:41.279 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:41.260+0000 7f5508f07640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.279 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:41.266+0000 7f5506702640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.326 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:41.327 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:41.327 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:41.327 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:41.327 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:41.336 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:49:41.337 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:41.335+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.379 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:41.378+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:41.383 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:41.381+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.406 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:41.404+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.427 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:41.425+0000 7f550a70a640 -1 ErasureCodeLrc: _minimum_to_decode not enough chunks in 0,7 to read 0,1,2,3,4,5,6,7 2024-09-16T14:49:41.449 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:41.450 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:41.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:41.450 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:41.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.450 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:49:41.450 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:41.450 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 60.537271148999999, 2024-09-16T14:49:41.450 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007542137, 2024-09-16T14:49:41.450 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.450 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.450 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.450 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:41.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:41.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.451 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.451 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.451 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:41.451 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:41.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909917+0000", 2024-09-16T14:49:41.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6513000000000001e-05 2024-09-16T14:49:41.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.452 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:49:41.452 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:49:41.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.452 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.452 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909941+0000", 2024-09-16T14:49:41.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1859999999999992e-06 2024-09-16T14:49:41.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910004+0000", 2024-09-16T14:49:41.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2706999999999994e-05 2024-09-16T14:49:41.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.453 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.453 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.453 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910012+0000", 2024-09-16T14:49:41.453 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.157e-06 2024-09-16T14:49:41.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917395+0000", 2024-09-16T14:49:41.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073827659999999998 2024-09-16T14:49:41.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.454 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.454 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.454 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917423+0000", 2024-09-16T14:49:41.454 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8568000000000002e-05 2024-09-16T14:49:41.454 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.454 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.455 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.455 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=37'32, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:41.455 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:41.455 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.526160805000004, 2024-09-16T14:49:41.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070991120000000003, 2024-09-16T14:49:41.455 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.455 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.455 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.455 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.455 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.455 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:41.455 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:41.456 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.456 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.456 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.456 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920991+0000", 2024-09-16T14:49:41.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:41.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921010+0000", 2024-09-16T14:49:41.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8529000000000002e-05 2024-09-16T14:49:41.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.457 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.457 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921018+0000", 2024-09-16T14:49:41.457 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4670000000000004e-06 2024-09-16T14:49:41.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.457 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921025+0000", 2024-09-16T14:49:41.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9330000000000004e-06 2024-09-16T14:49:41.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921076+0000", 2024-09-16T14:49:41.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0662999999999999e-05 2024-09-16T14:49:41.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.458 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.458 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.458 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921084+0000", 2024-09-16T14:49:41.458 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4780000000000008e-06 2024-09-16T14:49:41.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928073+0000", 2024-09-16T14:49:41.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069883940000000002 2024-09-16T14:49:41.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.459 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.459 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.459 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928091+0000", 2024-09-16T14:49:41.459 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7992999999999999e-05 2024-09-16T14:49:41.459 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.459 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.459 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.460 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:41.460 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:49:41.460 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.525161500000003, 2024-09-16T14:49:41.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070100509999999998, 2024-09-16T14:49:41.460 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.460 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.460 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.460 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.460 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.460 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:49:41.460 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921991+0000", 2024-09-16T14:49:41.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921990+0000", 2024-09-16T14:49:41.461 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:41.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.461 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.461 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.461 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922012+0000", 2024-09-16T14:49:41.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1681000000000001e-05 2024-09-16T14:49:41.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922025+0000", 2024-09-16T14:49:41.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3101999999999999e-05 2024-09-16T14:49:41.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.462 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.462 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922033+0000", 2024-09-16T14:49:41.462 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1899999999999995e-06 2024-09-16T14:49:41.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922727+0000", 2024-09-16T14:49:41.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00069429699999999997 2024-09-16T14:49:41.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.463 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.463 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922737+0000", 2024-09-16T14:49:41.463 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.285e-06 2024-09-16T14:49:41.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.463 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928986+0000", 2024-09-16T14:49:41.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062492679999999997 2024-09-16T14:49:41.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.464 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.464 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.464 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.929001+0000", 2024-09-16T14:49:41.464 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5058999999999999e-05 2024-09-16T14:49:41.464 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.464 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.464 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.465 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:41.465 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:41.465 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.521952435000003, 2024-09-16T14:49:41.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079457759999999999, 2024-09-16T14:49:41.465 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.465 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.465 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.465 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.465 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.465 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:41.465 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:41.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:41.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:41.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.466 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.466 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.466 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925222+0000", 2024-09-16T14:49:41.466 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1871000000000001e-05 2024-09-16T14:49:41.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925238+0000", 2024-09-16T14:49:41.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6719999999999999e-05 2024-09-16T14:49:41.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.467 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925244+0000", 2024-09-16T14:49:41.467 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3630000000000002e-06 2024-09-16T14:49:41.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.467 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.467 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925291+0000", 2024-09-16T14:49:41.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6944000000000003e-05 2024-09-16T14:49:41.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925299+0000", 2024-09-16T14:49:41.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:49:41.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.468 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.468 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.468 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933129+0000", 2024-09-16T14:49:41.468 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078297330000000002 2024-09-16T14:49:41.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.469 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.469 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933146+0000", 2024-09-16T14:49:41.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.664e-05 2024-09-16T14:49:41.469 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.469 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.469 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.469 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.469 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es5 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:41.469 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:41.469 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.521900799999997, 2024-09-16T14:49:41.469 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078358790000000005, 2024-09-16T14:49:41.469 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925251+0000", 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925265+0000", 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4078e-05 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925273+0000", 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1069999999999999e-06 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925279+0000", 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4409999999999998e-06 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925330+0000", 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1208999999999997e-05 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925339+0000", 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8109999999999992e-06 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933064+0000", 2024-09-16T14:49:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077252149999999997 2024-09-16T14:49:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933088+0000", 2024-09-16T14:49:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3317999999999999e-05 2024-09-16T14:49:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933425+0000", 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.513727621999998, 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090001820000000007, 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933425+0000", 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933425+0000", 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933424+0000", 2024-09-16T14:49:41.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:41.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.475 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.475 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933478+0000", 2024-09-16T14:49:41.475 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3735e-05 2024-09-16T14:49:41.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.475 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933493+0000", 2024-09-16T14:49:41.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4978e-05 2024-09-16T14:49:41.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933502+0000", 2024-09-16T14:49:41.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6509999999999996e-06 2024-09-16T14:49:41.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.476 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.476 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.476 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933761+0000", 2024-09-16T14:49:41.476 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025924500000000002 2024-09-16T14:49:41.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933772+0000", 2024-09-16T14:49:41.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0694e-05 2024-09-16T14:49:41.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.477 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942409+0000", 2024-09-16T14:49:41.477 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086371539999999993 2024-09-16T14:49:41.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.477 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.477 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.478 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942425+0000", 2024-09-16T14:49:41.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5781e-05 2024-09-16T14:49:41.478 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.478 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.478 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.478 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.478 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.cs4 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:41.478 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:41.478 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.513366015999999, 2024-09-16T14:49:41.478 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086187320000000005, 2024-09-16T14:49:41.478 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.478 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.479 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:41.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:41.479 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.479 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.479 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.479 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:41.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:41.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933807+0000", 2024-09-16T14:49:41.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1107e-05 2024-09-16T14:49:41.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.480 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.480 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933817+0000", 2024-09-16T14:49:41.480 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8870000000000004e-06 2024-09-16T14:49:41.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.480 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933826+0000", 2024-09-16T14:49:41.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5319999999999995e-06 2024-09-16T14:49:41.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933899+0000", 2024-09-16T14:49:41.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2294999999999998e-05 2024-09-16T14:49:41.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.481 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.481 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.481 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933908+0000", 2024-09-16T14:49:41.481 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4110000000000002e-06 2024-09-16T14:49:41.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942383+0000", 2024-09-16T14:49:41.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0084747099999999999 2024-09-16T14:49:41.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.482 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.482 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.482 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942405+0000", 2024-09-16T14:49:41.482 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2399e-05 2024-09-16T14:49:41.482 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.934497+0000", 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.512655862999999, 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078697109999999997, 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934497+0000", 2024-09-16T14:49:41.483 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934497+0000", 2024-09-16T14:49:41.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.484 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934496+0000", 2024-09-16T14:49:41.484 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:41.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.484 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.484 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935463+0000", 2024-09-16T14:49:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00096685900000000003 2024-09-16T14:49:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935498+0000", 2024-09-16T14:49:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5179000000000003e-05 2024-09-16T14:49:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935509+0000", 2024-09-16T14:49:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1223000000000001e-05 2024-09-16T14:49:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.485 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935573+0000", 2024-09-16T14:49:41.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4114000000000003e-05 2024-09-16T14:49:41.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935583+0000", 2024-09-16T14:49:41.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.008e-05 2024-09-16T14:49:41.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.486 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.486 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942344+0000", 2024-09-16T14:49:41.486 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067610760000000004 2024-09-16T14:49:41.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.487 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.487 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.487 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942366+0000", 2024-09-16T14:49:41.487 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2059e-05 2024-09-16T14:49:41.487 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.488 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.488 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.488 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.488 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 61/27 ECSubWrite(tid=242, reqid=osd.3.0:242, at_version=61'79, trim_to=0'0, roll_forward_to=60'77))", 2024-09-16T14:49:41.488 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.944331+0000", 2024-09-16T14:49:41.489 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.502821732999998, 2024-09-16T14:49:41.489 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083214770000000007, 2024-09-16T14:49:41.489 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.489 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.489 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.490 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.490 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.490 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944331+0000", 2024-09-16T14:49:41.490 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.491 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944331+0000", 2024-09-16T14:49:41.491 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.491 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.491 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944330+0000", 2024-09-16T14:49:41.492 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:41.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.492 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.492 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.492 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944354+0000", 2024-09-16T14:49:41.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3702000000000001e-05 2024-09-16T14:49:41.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.493 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.493 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.493 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944367+0000", 2024-09-16T14:49:41.493 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3485999999999999e-05 2024-09-16T14:49:41.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944375+0000", 2024-09-16T14:49:41.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3219999999999993e-06 2024-09-16T14:49:41.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944425+0000", 2024-09-16T14:49:41.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9289000000000002e-05 2024-09-16T14:49:41.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.494 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.494 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.494 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944436+0000", 2024-09-16T14:49:41.494 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1381e-05 2024-09-16T14:49:41.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.952629+0000", 2024-09-16T14:49:41.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081932840000000003 2024-09-16T14:49:41.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.495 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.495 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.495 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.952652+0000", 2024-09-16T14:49:41.495 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.296e-05 2024-09-16T14:49:41.495 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.495 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.495 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.496 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 77/27 ECSubWrite(tid=311, reqid=osd.3.0:311, at_version=77'100, trim_to=0'0, roll_forward_to=76'98))", 2024-09-16T14:49:41.496 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.915451+0000", 2024-09-16T14:49:41.496 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.531701769000001, 2024-09-16T14:49:41.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079661820000000005, 2024-09-16T14:49:41.496 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.496 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.496 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.496 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.496 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915451+0000", 2024-09-16T14:49:41.496 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.496 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915451+0000", 2024-09-16T14:49:41.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915450+0000", 2024-09-16T14:49:41.497 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:41.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.497 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.497 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.497 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915534+0000", 2024-09-16T14:49:41.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4160999999999996e-05 2024-09-16T14:49:41.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915547+0000", 2024-09-16T14:49:41.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2764e-05 2024-09-16T14:49:41.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.498 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.498 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915555+0000", 2024-09-16T14:49:41.498 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2090000000000005e-06 2024-09-16T14:49:41.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.498 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915659+0000", 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103879 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915685+0000", 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5959999999999999e-05 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923392+0000", 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077072379999999999 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.499 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.500 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.500 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923417+0000", 2024-09-16T14:49:41.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5000000000000001e-05 2024-09-16T14:49:41.500 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.500 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.500 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.500 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.0s1 77/65 ECSubWrite(tid=279, reqid=osd.11.0:279, at_version=77'35, trim_to=0'0, roll_forward_to=74'31))", 2024-09-16T14:49:41.500 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.916327+0000", 2024-09-16T14:49:41.500 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.530825706999998, 2024-09-16T14:49:41.500 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070483610000000004, 2024-09-16T14:49:41.500 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.501 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.501 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916327+0000", 2024-09-16T14:49:41.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.501 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916327+0000", 2024-09-16T14:49:41.501 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.501 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.501 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916326+0000", 2024-09-16T14:49:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916348+0000", 2024-09-16T14:49:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2327000000000002e-05 2024-09-16T14:49:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916363+0000", 2024-09-16T14:49:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4295e-05 2024-09-16T14:49:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.502 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916371+0000", 2024-09-16T14:49:41.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9519999999999994e-06 2024-09-16T14:49:41.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916417+0000", 2024-09-16T14:49:41.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6279000000000002e-05 2024-09-16T14:49:41.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.503 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.503 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.503 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916427+0000", 2024-09-16T14:49:41.503 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0164e-05 2024-09-16T14:49:41.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923355+0000", 2024-09-16T14:49:41.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069278569999999999 2024-09-16T14:49:41.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.504 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.504 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923375+0000", 2024-09-16T14:49:41.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0341999999999999e-05 2024-09-16T14:49:41.504 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.504 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.504 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.505 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 77/72 ECSubWrite(tid=14, reqid=client.4500.0:702, at_version=77'33, trim_to=0'0, roll_forward_to=77'32))", 2024-09-16T14:49:41.505 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:41.505 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 17.529462006999999, 2024-09-16T14:49:41.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070951110000000003, 2024-09-16T14:49:41.505 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.505 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.505 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:41.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:41.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:41.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:41.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918213+0000", 2024-09-16T14:49:41.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000523721 2024-09-16T14:49:41.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918232+0000", 2024-09-16T14:49:41.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8743999999999999e-05 2024-09-16T14:49:41.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918241+0000", 2024-09-16T14:49:41.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9539999999999993e-06 2024-09-16T14:49:41.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918356+0000", 2024-09-16T14:49:41.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011514999999999999 2024-09-16T14:49:41.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918367+0000", 2024-09-16T14:49:41.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0933e-05 2024-09-16T14:49:41.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.924758+0000", 2024-09-16T14:49:41.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063913499999999996 2024-09-16T14:49:41.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.924786+0000", 2024-09-16T14:49:41.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7129000000000001e-05 2024-09-16T14:49:41.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.510 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.510 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 81/27 ECSubWrite(tid=339, reqid=osd.3.0:339, at_version=81'108, trim_to=0'0, roll_forward_to=80'104))", 2024-09-16T14:49:41.510 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:27.930359+0000", 2024-09-16T14:49:41.510 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.51679369, 2024-09-16T14:49:41.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0085286319999999995, 2024-09-16T14:49:41.510 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.510 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.510 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930359+0000", 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930359+0000", 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930358+0000", 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930382+0000", 2024-09-16T14:49:41.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4034999999999999e-05 2024-09-16T14:49:41.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930393+0000", 2024-09-16T14:49:41.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1062e-05 2024-09-16T14:49:41.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930403+0000", 2024-09-16T14:49:41.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8779999999999993e-06 2024-09-16T14:49:41.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930675+0000", 2024-09-16T14:49:41.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027152900000000001 2024-09-16T14:49:41.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930681+0000", 2024-09-16T14:49:41.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.229e-06 2024-09-16T14:49:41.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.938865+0000", 2024-09-16T14:49:41.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081845249999999998 2024-09-16T14:49:41.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.938887+0000", 2024-09-16T14:49:41.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1946e-05 2024-09-16T14:49:41.514 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.514 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.514 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.514 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 81/65 ECSubWrite(tid=83, reqid=osd.1.0:83, at_version=81'66, trim_to=0'0, roll_forward_to=80'64))", 2024-09-16T14:49:41.514 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:27.931996+0000", 2024-09-16T14:49:41.514 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 13.515156934, 2024-09-16T14:49:41.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074671399999999997, 2024-09-16T14:49:41.515 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.515 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.515 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.931996+0000", 2024-09-16T14:49:41.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.931996+0000", 2024-09-16T14:49:41.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.931995+0000", 2024-09-16T14:49:41.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:41.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.932025+0000", 2024-09-16T14:49:41.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9884999999999999e-05 2024-09-16T14:49:41.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.932035+0000", 2024-09-16T14:49:41.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0182e-05 2024-09-16T14:49:41.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.932043+0000", 2024-09-16T14:49:41.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0900000000000005e-06 2024-09-16T14:49:41.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.933159+0000", 2024-09-16T14:49:41.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011161980000000001 2024-09-16T14:49:41.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.517 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.517 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.933170+0000", 2024-09-16T14:49:41.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0925e-05 2024-09-16T14:49:41.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.939441+0000", 2024-09-16T14:49:41.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062703339999999998 2024-09-16T14:49:41.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.939463+0000", 2024-09-16T14:49:41.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2005e-05 2024-09-16T14:49:41.518 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.519 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.519 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.519 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 90/27 ECSubWrite(tid=310, reqid=osd.7.0:310, at_version=90'48, trim_to=0'0, roll_forward_to=89'46))", 2024-09-16T14:49:41.519 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:41.519 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.492919487, 2024-09-16T14:49:41.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011963438999999999, 2024-09-16T14:49:41.519 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.519 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.519 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:41.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:41.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:41.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:41.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954255+0000", 2024-09-16T14:49:41.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2704000000000001e-05 2024-09-16T14:49:41.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954273+0000", 2024-09-16T14:49:41.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7609999999999999e-05 2024-09-16T14:49:41.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954283+0000", 2024-09-16T14:49:41.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9539999999999999e-06 2024-09-16T14:49:41.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954340+0000", 2024-09-16T14:49:41.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6968000000000001e-05 2024-09-16T14:49:41.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954352+0000", 2024-09-16T14:49:41.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1548e-05 2024-09-16T14:49:41.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.966177+0000", 2024-09-16T14:49:41.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011825293000000001 2024-09-16T14:49:41.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.966196+0000", 2024-09-16T14:49:41.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9636999999999999e-05 2024-09-16T14:49:41.523 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.523 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.523 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.524 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 91/90 ECSubWrite(tid=383, reqid=client.4500.0:825, at_version=91'144, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:41.524 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:41.524 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.4711981380000001, 2024-09-16T14:49:41.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079391419999999997, 2024-09-16T14:49:41.524 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.524 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.524 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:41.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:41.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:41.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:41.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976430+0000", 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000476222 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976441+0000", 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1457000000000001e-05 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976449+0000", 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9719999999999992e-06 2024-09-16T14:49:41.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976957+0000", 2024-09-16T14:49:41.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050779500000000001 2024-09-16T14:49:41.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976967+0000", 2024-09-16T14:49:41.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0424e-05 2024-09-16T14:49:41.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.983864+0000", 2024-09-16T14:49:41.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068963369999999998 2024-09-16T14:49:41.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.528 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.528 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.983894+0000", 2024-09-16T14:49:41.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9784000000000001e-05 2024-09-16T14:49:41.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.528 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.529 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.17s5 91/90 ECSubWrite(tid=394, reqid=osd.3.0:391, at_version=0'0, trim_to=0'0, roll_forward_to=91'144))", 2024-09-16T14:49:41.529 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.006074+0000", 2024-09-16T14:49:41.529 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.441078445, 2024-09-16T14:49:41.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010644525, 2024-09-16T14:49:41.529 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.529 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.529 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006074+0000", 2024-09-16T14:49:41.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006074+0000", 2024-09-16T14:49:41.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006073+0000", 2024-09-16T14:49:41.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:41.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006086+0000", 2024-09-16T14:49:41.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2862e-05 2024-09-16T14:49:41.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006102+0000", 2024-09-16T14:49:41.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6331e-05 2024-09-16T14:49:41.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006113+0000", 2024-09-16T14:49:41.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0518e-05 2024-09-16T14:49:41.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006171+0000", 2024-09-16T14:49:41.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8326999999999999e-05 2024-09-16T14:49:41.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006182+0000", 2024-09-16T14:49:41.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1134999999999999e-05 2024-09-16T14:49:41.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016684+0000", 2024-09-16T14:49:41.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010501664000000001 2024-09-16T14:49:41.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016719+0000", 2024-09-16T14:49:41.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4480000000000002e-05 2024-09-16T14:49:41.533 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.533 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.533 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.533 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.19s3 91/90 ECSubWrite(tid=102, reqid=client.4500.0:822, at_version=91'77, trim_to=0'0, roll_forward_to=91'75))", 2024-09-16T14:49:41.533 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.006461+0000", 2024-09-16T14:49:41.534 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.440691266, 2024-09-16T14:49:41.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010302113, 2024-09-16T14:49:41.534 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.534 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.534 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.534 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.534 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006461+0000", 2024-09-16T14:49:41.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006461+0000", 2024-09-16T14:49:41.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006460+0000", 2024-09-16T14:49:41.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:41.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007931+0000", 2024-09-16T14:49:41.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014713079999999999 2024-09-16T14:49:41.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007954+0000", 2024-09-16T14:49:41.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2308999999999999e-05 2024-09-16T14:49:41.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007970+0000", 2024-09-16T14:49:41.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5880999999999999e-05 2024-09-16T14:49:41.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008058+0000", 2024-09-16T14:49:41.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8506999999999998e-05 2024-09-16T14:49:41.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008071+0000", 2024-09-16T14:49:41.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2434e-05 2024-09-16T14:49:41.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016742+0000", 2024-09-16T14:49:41.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086718799999999999 2024-09-16T14:49:41.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016763+0000", 2024-09-16T14:49:41.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0890000000000002e-05 2024-09-16T14:49:41.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.539 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.539 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.539 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.17s5 91/90 ECSubWrite(tid=395, reqid=osd.3.0:395, at_version=91'146, trim_to=0'0, roll_forward_to=91'144))", 2024-09-16T14:49:41.539 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.008031+0000", 2024-09-16T14:49:41.539 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.4391210770000002, 2024-09-16T14:49:41.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087059660000000007, 2024-09-16T14:49:41.539 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.539 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.539 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008031+0000", 2024-09-16T14:49:41.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008031+0000", 2024-09-16T14:49:41.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008030+0000", 2024-09-16T14:49:41.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:41.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008049+0000", 2024-09-16T14:49:41.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9230999999999999e-05 2024-09-16T14:49:41.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008059+0000", 2024-09-16T14:49:41.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7729999999999994e-06 2024-09-16T14:49:41.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008066+0000", 2024-09-16T14:49:41.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4849999999999997e-06 2024-09-16T14:49:41.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008113+0000", 2024-09-16T14:49:41.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7154999999999998e-05 2024-09-16T14:49:41.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008124+0000", 2024-09-16T14:49:41.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0933e-05 2024-09-16T14:49:41.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016723+0000", 2024-09-16T14:49:41.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0085989640000000006 2024-09-16T14:49:41.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016737+0000", 2024-09-16T14:49:41.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4635e-05 2024-09-16T14:49:41.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.543 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.544 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.14s2 91/90 ECSubWrite(tid=103, reqid=osd.1.0:102, at_version=0'0, trim_to=0'0, roll_forward_to=91'76))", 2024-09-16T14:49:41.544 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.009355+0000", 2024-09-16T14:49:41.544 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.4377969999999998, 2024-09-16T14:49:41.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074028089999999998, 2024-09-16T14:49:41.544 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.544 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.544 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009355+0000", 2024-09-16T14:49:41.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009355+0000", 2024-09-16T14:49:41.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009354+0000", 2024-09-16T14:49:41.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:41.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009370+0000", 2024-09-16T14:49:41.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6163000000000001e-05 2024-09-16T14:49:41.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009384+0000", 2024-09-16T14:49:41.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3655000000000001e-05 2024-09-16T14:49:41.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009393+0000", 2024-09-16T14:49:41.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1919999999999994e-06 2024-09-16T14:49:41.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009453+0000", 2024-09-16T14:49:41.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9626000000000001e-05 2024-09-16T14:49:41.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009467+0000", 2024-09-16T14:49:41.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4123e-05 2024-09-16T14:49:41.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016732+0000", 2024-09-16T14:49:41.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007265395 2024-09-16T14:49:41.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016758+0000", 2024-09-16T14:49:41.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6157e-05 2024-09-16T14:49:41.548 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.548 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.548 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.548 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.548 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.548 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:41.550 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:49:41.550 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:41.549+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:41.650 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:41.649+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:41.692 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:41.692 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:41.692 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:41.701 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:49:41.751 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:41.749+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:41.803 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:41.803 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:41.803 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:41.803 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:41.803 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:41.811 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:49:41.852 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:41.851+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:41.863 INFO:tasks.rados.rados.0.smithi040.stdout:352: done (1 left) 2024-09-16T14:49:41.863 INFO:tasks.rados.rados.0.smithi040.stdout:353: read oid 21 snap 31 2024-09-16T14:49:41.863 INFO:tasks.rados.rados.0.smithi040.stdout:353: expect deleted 2024-09-16T14:49:41.863 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T14:49:41.864 INFO:tasks.rados.rados.0.smithi040.stdout:354: read oid 44 snap 31 2024-09-16T14:49:41.864 INFO:tasks.rados.rados.0.smithi040.stdout:354: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-16T14:49:41.864 INFO:tasks.rados.rados.0.smithi040.stdout:355: read oid 33 snap 31 2024-09-16T14:49:41.864 INFO:tasks.rados.rados.0.smithi040.stdout:355: expect (ObjNum 98 snap 18 seq_num 98) 2024-09-16T14:49:41.864 INFO:tasks.rados.rados.0.smithi040.stdout:356: copy_from oid 31 from oid 35 current snap is 33 2024-09-16T14:49:41.864 INFO:tasks.rados.rados.0.smithi040.stdout:350: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-16T14:49:41.892 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T14:49:41.895 INFO:tasks.rados.rados.0.smithi040.stdout:354: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-16T14:49:41.914 INFO:tasks.rados.rados.0.smithi040.stdout:355: expect (ObjNum 98 snap 18 seq_num 98) 2024-09-16T14:49:41.953 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:41.952+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:41.958 INFO:tasks.rados.rados.0.smithi040.stdout:350: done (4 left) 2024-09-16T14:49:41.958 INFO:tasks.rados.rados.0.smithi040.stdout:354: done (3 left) 2024-09-16T14:49:41.959 INFO:tasks.rados.rados.0.smithi040.stdout:355: done (2 left) 2024-09-16T14:49:41.959 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 22 current snap is 33 2024-09-16T14:49:41.959 INFO:tasks.rados.rados.0.smithi040.stdout:357: seq_num 126 ranges {3309568=573440,3883008=622592,4505600=622592,5128192=311296} 2024-09-16T14:49:41.962 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:41.962 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:41.962 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:41.962 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:41.962 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.962 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:49:41.962 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:41.963 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.051024102, 2024-09-16T14:49:41.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079855010000000008, 2024-09-16T14:49:41.963 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.963 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.963 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:41.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.963 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.963 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.963 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:41.963 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909483+0000", 2024-09-16T14:49:41.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:41.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.964 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909521+0000", 2024-09-16T14:49:41.964 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7185999999999999e-05 2024-09-16T14:49:41.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.964 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.964 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.965 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909543+0000", 2024-09-16T14:49:41.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2802999999999998e-05 2024-09-16T14:49:41.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.965 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.965 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909551+0000", 2024-09-16T14:49:41.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5039999999999996e-06 2024-09-16T14:49:41.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.965 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.965 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.965 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909927+0000", 2024-09-16T14:49:41.965 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037637400000000001 2024-09-16T14:49:41.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:49:41.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2040000000000002e-06 2024-09-16T14:49:41.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.966 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.966 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.966 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917442+0000", 2024-09-16T14:49:41.966 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007508659 2024-09-16T14:49:41.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.967 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.967 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917469+0000", 2024-09-16T14:49:41.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7197999999999999e-05 2024-09-16T14:49:41.967 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.967 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.967 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.967 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.967 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:49:41.967 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:41.967 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.046951755999999, 2024-09-16T14:49:41.967 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067701159999999996, 2024-09-16T14:49:41.967 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.968 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.968 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:41.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.968 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.968 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.968 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:41.968 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.969 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.969 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:41.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:41.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.969 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.969 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913579+0000", 2024-09-16T14:49:41.969 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2829000000000001e-05 2024-09-16T14:49:41.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.969 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913591+0000", 2024-09-16T14:49:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2264e-05 2024-09-16T14:49:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913600+0000", 2024-09-16T14:49:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9959999999999999e-06 2024-09-16T14:49:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913648+0000", 2024-09-16T14:49:41.970 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7759000000000001e-05 2024-09-16T14:49:41.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913657+0000", 2024-09-16T14:49:41.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6709999999999994e-06 2024-09-16T14:49:41.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920296+0000", 2024-09-16T14:49:41.971 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066385120000000001 2024-09-16T14:49:41.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.971 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.971 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.971 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920326+0000", 2024-09-16T14:49:41.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0405000000000001e-05 2024-09-16T14:49:41.972 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.972 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.972 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.972 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.972 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 34/27 ECSubWrite(tid=74, reqid=client.4500.0:229, at_version=34'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:49:41.972 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:41.972 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.046135798000002, 2024-09-16T14:49:41.972 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060056090000000003, 2024-09-16T14:49:41.972 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.972 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.972 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:41.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:41.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.973 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.973 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.973 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:41.973 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:41.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.974 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914468+0000", 2024-09-16T14:49:41.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6398999999999996e-05 2024-09-16T14:49:41.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.974 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914483+0000", 2024-09-16T14:49:41.974 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5238e-05 2024-09-16T14:49:41.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.974 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.974 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914494+0000", 2024-09-16T14:49:41.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0192e-05 2024-09-16T14:49:41.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914622+0000", 2024-09-16T14:49:41.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012855599999999999 2024-09-16T14:49:41.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.975 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.975 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.975 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914635+0000", 2024-09-16T14:49:41.975 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2510000000000001e-05 2024-09-16T14:49:41.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920339+0000", 2024-09-16T14:49:41.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057041549999999998 2024-09-16T14:49:41.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.976 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.976 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.976 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920378+0000", 2024-09-16T14:49:41.976 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8918999999999998e-05 2024-09-16T14:49:41.976 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.976 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.976 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.977 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:49:41.977 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:41.977 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.044123022000001, 2024-09-16T14:49:41.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069011319999999999, 2024-09-16T14:49:41.977 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.977 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.977 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.977 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.977 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:41.977 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.977 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:41.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:41.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:49:41.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.978 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.978 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.978 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916410+0000", 2024-09-16T14:49:41.978 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5343000000000001e-05 2024-09-16T14:49:41.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916418+0000", 2024-09-16T14:49:41.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2360000000000004e-06 2024-09-16T14:49:41.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916428+0000", 2024-09-16T14:49:41.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6779999999999995e-06 2024-09-16T14:49:41.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.979 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.979 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.979 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916492+0000", 2024-09-16T14:49:41.979 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4219000000000003e-05 2024-09-16T14:49:41.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916502+0000", 2024-09-16T14:49:41.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6690000000000001e-06 2024-09-16T14:49:41.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923254+0000", 2024-09-16T14:49:41.980 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067518070000000003 2024-09-16T14:49:41.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.980 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.980 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.980 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923286+0000", 2024-09-16T14:49:41.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2367999999999999e-05 2024-09-16T14:49:41.981 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.981 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.981 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.981 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s7 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:41.981 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:41.981 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.038818638000002, 2024-09-16T14:49:41.981 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006972692, 2024-09-16T14:49:41.981 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.981 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.981 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.981 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:41.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:41.982 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.982 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.982 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.982 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:41.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:41.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921709+0000", 2024-09-16T14:49:41.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0411000000000002e-05 2024-09-16T14:49:41.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.983 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.983 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921719+0000", 2024-09-16T14:49:41.983 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2650000000000002e-06 2024-09-16T14:49:41.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.983 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921726+0000", 2024-09-16T14:49:41.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8770000000000004e-06 2024-09-16T14:49:41.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.984 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.984 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921773+0000", 2024-09-16T14:49:41.984 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7435999999999998e-05 2024-09-16T14:49:41.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.984 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921782+0000", 2024-09-16T14:49:41.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6039999999999993e-06 2024-09-16T14:49:41.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928635+0000", 2024-09-16T14:49:41.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068533129999999998 2024-09-16T14:49:41.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.985 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.985 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.985 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928662+0000", 2024-09-16T14:49:41.985 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7089000000000001e-05 2024-09-16T14:49:41.985 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.986 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.986 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.986 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:41.986 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:41.986 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.038322149999999, 2024-09-16T14:49:41.986 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064783530000000001, 2024-09-16T14:49:41.986 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.986 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.986 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.986 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.986 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.986 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:41.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:41.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.987 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922185+0000", 2024-09-16T14:49:41.987 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:41.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.987 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.987 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922205+0000", 2024-09-16T14:49:41.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9871e-05 2024-09-16T14:49:41.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922219+0000", 2024-09-16T14:49:41.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4266e-05 2024-09-16T14:49:41.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.988 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.988 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.988 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922229+0000", 2024-09-16T14:49:41.988 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5619999999999993e-06 2024-09-16T14:49:41.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922270+0000", 2024-09-16T14:49:41.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0917000000000001e-05 2024-09-16T14:49:41.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922278+0000", 2024-09-16T14:49:41.989 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3890000000000005e-06 2024-09-16T14:49:41.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.989 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.989 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.989 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928630+0000", 2024-09-16T14:49:41.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063524439999999996 2024-09-16T14:49:41.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.990 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.990 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928664+0000", 2024-09-16T14:49:41.990 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3651e-05 2024-09-16T14:49:41.990 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.990 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.990 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.990 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.990 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s5 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:49:41.990 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:41.990 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.036873745999998, 2024-09-16T14:49:41.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069615850000000002, 2024-09-16T14:49:41.991 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.991 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.991 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:41.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.991 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.991 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.991 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:41.991 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923633+0000", 2024-09-16T14:49:41.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:41.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923644+0000", 2024-09-16T14:49:41.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.08e-05 2024-09-16T14:49:41.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.992 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.992 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.992 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923657+0000", 2024-09-16T14:49:41.992 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2836999999999999e-05 2024-09-16T14:49:41.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923664+0000", 2024-09-16T14:49:41.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1470000000000004e-06 2024-09-16T14:49:41.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923885+0000", 2024-09-16T14:49:41.993 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022155800000000001 2024-09-16T14:49:41.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.993 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.993 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.993 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923894+0000", 2024-09-16T14:49:41.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9260000000000006e-06 2024-09-16T14:49:41.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930571+0000", 2024-09-16T14:49:41.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066770420000000002 2024-09-16T14:49:41.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.994 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.994 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.994 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930596+0000", 2024-09-16T14:49:41.994 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4383e-05 2024-09-16T14:49:41.994 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.994 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.994 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.995 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as3 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:41.995 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:41.995 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.03612751, 2024-09-16T14:49:41.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062471519999999997, 2024-09-16T14:49:41.995 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:41.995 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:41.995 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:41.995 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.995 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:41.995 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:41.995 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:41.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:41.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:41.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:41.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:41.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:41.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.996 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.996 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:41.996 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924405+0000", 2024-09-16T14:49:41.996 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5147000000000002e-05 2024-09-16T14:49:41.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:41.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924414+0000", 2024-09-16T14:49:41.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4810000000000006e-06 2024-09-16T14:49:41.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:41.997 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924421+0000", 2024-09-16T14:49:41.997 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0899999999999999e-06 2024-09-16T14:49:41.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.997 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.997 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:41.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924468+0000", 2024-09-16T14:49:41.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.727e-05 2024-09-16T14:49:41.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:41.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924477+0000", 2024-09-16T14:49:41.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7360000000000007e-06 2024-09-16T14:49:41.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.998 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:41.998 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930601+0000", 2024-09-16T14:49:41.998 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061242529999999996 2024-09-16T14:49:41.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.998 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.999 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:41.999 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930628+0000", 2024-09-16T14:49:41.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6517e-05 2024-09-16T14:49:41.999 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.999 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:41.999 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:41.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:41.999 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:41.999 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:41.999 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:41.999 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.035877306000003, 2024-09-16T14:49:41.999 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006019328, 2024-09-16T14:49:41.999 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.000 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.000 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:42.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.000 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.000 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.000 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:42.000 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924630+0000", 2024-09-16T14:49:42.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924650+0000", 2024-09-16T14:49:42.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0024e-05 2024-09-16T14:49:42.001 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.001 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.001 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.001 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924660+0000", 2024-09-16T14:49:42.001 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9219999999999992e-06 2024-09-16T14:49:42.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924668+0000", 2024-09-16T14:49:42.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6820000000000003e-06 2024-09-16T14:49:42.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924711+0000", 2024-09-16T14:49:42.002 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3009000000000001e-05 2024-09-16T14:49:42.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.002 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.002 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.002 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924720+0000", 2024-09-16T14:49:42.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9290000000000004e-06 2024-09-16T14:49:42.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930634+0000", 2024-09-16T14:49:42.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059144810000000001 2024-09-16T14:49:42.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.003 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.003 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.003 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930650+0000", 2024-09-16T14:49:42.003 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5688999999999998e-05 2024-09-16T14:49:42.003 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.003 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.003 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.004 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.8s6 51/27 ECSubWrite(tid=173, reqid=osd.11.0:173, at_version=51'54, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:42.004 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.881283+0000", 2024-09-16T14:49:42.004 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.079225217999998, 2024-09-16T14:49:42.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057129700000000004, 2024-09-16T14:49:42.004 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.004 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.004 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.004 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.004 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.004 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881283+0000", 2024-09-16T14:49:42.004 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881283+0000", 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881282+0000", 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881333+0000", 2024-09-16T14:49:42.005 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0413e-05 2024-09-16T14:49:42.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881345+0000", 2024-09-16T14:49:42.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1933000000000001e-05 2024-09-16T14:49:42.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.006 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881353+0000", 2024-09-16T14:49:42.006 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0469999999999994e-06 2024-09-16T14:49:42.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.006 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.006 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882512+0000", 2024-09-16T14:49:42.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001159806 2024-09-16T14:49:42.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882521+0000", 2024-09-16T14:49:42.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9639999999999992e-06 2024-09-16T14:49:42.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.007 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.007 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.886958+0000", 2024-09-16T14:49:42.007 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044365170000000001 2024-09-16T14:49:42.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.007 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.008 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.008 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.886996+0000", 2024-09-16T14:49:42.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7808999999999997e-05 2024-09-16T14:49:42.008 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.008 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.008 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.008 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.008 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:42.008 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:42.008 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.027508187000002, 2024-09-16T14:49:42.008 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062137989999999999, 2024-09-16T14:49:42.008 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.008 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.009 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:42.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.009 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:42.009 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.009 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.009 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.932999+0000", 2024-09-16T14:49:42.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:42.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933022+0000", 2024-09-16T14:49:42.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3518000000000001e-05 2024-09-16T14:49:42.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.010 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.010 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933035+0000", 2024-09-16T14:49:42.010 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2565000000000001e-05 2024-09-16T14:49:42.010 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.010 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933044+0000", 2024-09-16T14:49:42.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3009999999999995e-06 2024-09-16T14:49:42.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934016+0000", 2024-09-16T14:49:42.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00097159700000000004 2024-09-16T14:49:42.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.011 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.011 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.011 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934022+0000", 2024-09-16T14:49:42.011 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4869999999999999e-06 2024-09-16T14:49:42.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939202+0000", 2024-09-16T14:49:42.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051795820000000003 2024-09-16T14:49:42.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.012 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.012 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.012 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939214+0000", 2024-09-16T14:49:42.012 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1663e-05 2024-09-16T14:49:42.012 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.012 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.012 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.013 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.cs3 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:42.013 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:42.013 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.027041652000001, 2024-09-16T14:49:42.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057306709999999997, 2024-09-16T14:49:42.013 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.013 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.013 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.013 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.013 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.013 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:42.013 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933465+0000", 2024-09-16T14:49:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933489+0000", 2024-09-16T14:49:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3940000000000001e-05 2024-09-16T14:49:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933501+0000", 2024-09-16T14:49:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1932e-05 2024-09-16T14:49:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933510+0000", 2024-09-16T14:49:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5820000000000002e-06 2024-09-16T14:49:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936680+0000", 2024-09-16T14:49:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003169893 2024-09-16T14:49:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936689+0000", 2024-09-16T14:49:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5810000000000003e-06 2024-09-16T14:49:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939176+0000", 2024-09-16T14:49:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002486238 2024-09-16T14:49:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939197+0000", 2024-09-16T14:49:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1325e-05 2024-09-16T14:49:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 62/27 ECSubWrite(tid=244, reqid=osd.7.0:244, at_version=62'27, trim_to=0'0, roll_forward_to=61'25))", 2024-09-16T14:49:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.017678644999997, 2024-09-16T14:49:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007111081, 2024-09-16T14:49:42.017 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.018 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.018 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:42.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:42.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:42.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942855+0000", 2024-09-16T14:49:42.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5999999999999998e-05 2024-09-16T14:49:42.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942867+0000", 2024-09-16T14:49:42.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2481e-05 2024-09-16T14:49:42.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942879+0000", 2024-09-16T14:49:42.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1838000000000001e-05 2024-09-16T14:49:42.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942938+0000", 2024-09-16T14:49:42.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8282000000000002e-05 2024-09-16T14:49:42.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942948+0000", 2024-09-16T14:49:42.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0497e-05 2024-09-16T14:49:42.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949905+0000", 2024-09-16T14:49:42.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069572940000000002 2024-09-16T14:49:42.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949940+0000", 2024-09-16T14:49:42.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4965999999999997e-05 2024-09-16T14:49:42.021 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.021 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.021 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.022 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 62/27 ECSubWrite(tid=234, reqid=osd.11.0:234, at_version=62'39, trim_to=0'0, roll_forward_to=61'37))", 2024-09-16T14:49:42.022 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:42.022 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.017245160999998, 2024-09-16T14:49:42.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066952440000000004, 2024-09-16T14:49:42.022 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.022 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.022 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:42.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:42.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943262+0000", 2024-09-16T14:49:42.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943285+0000", 2024-09-16T14:49:42.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2650000000000002e-05 2024-09-16T14:49:42.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943301+0000", 2024-09-16T14:49:42.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5398000000000001e-05 2024-09-16T14:49:42.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943309+0000", 2024-09-16T14:49:42.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9269999999999994e-06 2024-09-16T14:49:42.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943373+0000", 2024-09-16T14:49:42.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3194999999999994e-05 2024-09-16T14:49:42.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943383+0000", 2024-09-16T14:49:42.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0778000000000001e-05 2024-09-16T14:49:42.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949934+0000", 2024-09-16T14:49:42.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065509790000000002 2024-09-16T14:49:42.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949958+0000", 2024-09-16T14:49:42.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3598999999999999e-05 2024-09-16T14:49:42.026 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.026 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.026 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.026 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 76/27 ECSubWrite(tid=303, reqid=client.4500.0:699, at_version=76'98, trim_to=0'0, roll_forward_to=76'95))", 2024-09-16T14:49:42.026 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:42.026 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.050474615999999, 2024-09-16T14:49:42.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006028032, 2024-09-16T14:49:42.026 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.027 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.027 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:42.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:42.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910032+0000", 2024-09-16T14:49:42.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:42.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910053+0000", 2024-09-16T14:49:42.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0665e-05 2024-09-16T14:49:42.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910067+0000", 2024-09-16T14:49:42.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4416000000000001e-05 2024-09-16T14:49:42.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910077+0000", 2024-09-16T14:49:42.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4509999999999998e-06 2024-09-16T14:49:42.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.029 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910367+0000", 2024-09-16T14:49:42.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00029006799999999999 2024-09-16T14:49:42.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.029 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910378+0000", 2024-09-16T14:49:42.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0628000000000001e-05 2024-09-16T14:49:42.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.916036+0000", 2024-09-16T14:49:42.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0056580270000000004 2024-09-16T14:49:42.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.916061+0000", 2024-09-16T14:49:42.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5658000000000001e-05 2024-09-16T14:49:42.030 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.030 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.031 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.031 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 77/27 ECSubWrite(tid=277, reqid=osd.11.0:277, at_version=77'54, trim_to=0'0, roll_forward_to=75'52))", 2024-09-16T14:49:42.031 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:42.031 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.049987248000001, 2024-09-16T14:49:42.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064553950000000001, 2024-09-16T14:49:42.031 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.031 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.031 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:42.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:42.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910520+0000", 2024-09-16T14:49:42.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:49:42.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910554+0000", 2024-09-16T14:49:42.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3111999999999997e-05 2024-09-16T14:49:42.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910578+0000", 2024-09-16T14:49:42.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4757999999999999e-05 2024-09-16T14:49:42.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910590+0000", 2024-09-16T14:49:42.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2156e-05 2024-09-16T14:49:42.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910646+0000", 2024-09-16T14:49:42.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5979000000000001e-05 2024-09-16T14:49:42.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910656+0000", 2024-09-16T14:49:42.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9410000000000002e-06 2024-09-16T14:49:42.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916943+0000", 2024-09-16T14:49:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062870260000000002 2024-09-16T14:49:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916976+0000", 2024-09-16T14:49:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2641999999999997e-05 2024-09-16T14:49:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.035 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 91/27 ECSubWrite(tid=316, reqid=osd.7.0:315, at_version=0'0, trim_to=0'0, roll_forward_to=91'50))", 2024-09-16T14:49:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.963325+0000", 2024-09-16T14:49:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.9971831570000003, 2024-09-16T14:49:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060303650000000002, 2024-09-16T14:49:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963325+0000", 2024-09-16T14:49:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963325+0000", 2024-09-16T14:49:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963324+0000", 2024-09-16T14:49:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963338+0000", 2024-09-16T14:49:42.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3813e-05 2024-09-16T14:49:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963350+0000", 2024-09-16T14:49:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2512e-05 2024-09-16T14:49:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963360+0000", 2024-09-16T14:49:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0141e-05 2024-09-16T14:49:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963425+0000", 2024-09-16T14:49:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4579999999999995e-05 2024-09-16T14:49:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963437+0000", 2024-09-16T14:49:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1780000000000001e-05 2024-09-16T14:49:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.969335+0000", 2024-09-16T14:49:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0058980500000000002 2024-09-16T14:49:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.039 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.969355+0000", 2024-09-16T14:49:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0489e-05 2024-09-16T14:49:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 91/90 ECSubWrite(tid=383, reqid=client.4500.0:825, at_version=91'144, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.976070+0000", 2024-09-16T14:49:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.9844379490000001, 2024-09-16T14:49:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066451510000000002, 2024-09-16T14:49:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.040 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976070+0000", 2024-09-16T14:49:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976070+0000", 2024-09-16T14:49:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976069+0000", 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976953+0000", 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00088349499999999998 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977007+0000", 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4676000000000003e-05 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977083+0000", 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5831e-05 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977120+0000", 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6887000000000001e-05 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977129+0000", 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9120000000000004e-06 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.982689+0000", 2024-09-16T14:49:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055597590000000001 2024-09-16T14:49:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.982715+0000", 2024-09-16T14:49:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6478999999999998e-05 2024-09-16T14:49:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.19s7 91/90 ECSubWrite(tid=102, reqid=client.4500.0:822, at_version=91'77, trim_to=0'0, roll_forward_to=91'75))", 2024-09-16T14:49:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.007185+0000", 2024-09-16T14:49:42.044 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.9533232150000002, 2024-09-16T14:49:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068803600000000003, 2024-09-16T14:49:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007185+0000", 2024-09-16T14:49:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007185+0000", 2024-09-16T14:49:42.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007183+0000", 2024-09-16T14:49:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008457+0000", 2024-09-16T14:49:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001274025 2024-09-16T14:49:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008473+0000", 2024-09-16T14:49:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5605999999999999e-05 2024-09-16T14:49:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008483+0000", 2024-09-16T14:49:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8430000000000005e-06 2024-09-16T14:49:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008541+0000", 2024-09-16T14:49:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7911999999999997e-05 2024-09-16T14:49:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008551+0000", 2024-09-16T14:49:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0353999999999999e-05 2024-09-16T14:49:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.014044+0000", 2024-09-16T14:49:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054927439999999999 2024-09-16T14:49:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.014065+0000", 2024-09-16T14:49:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1124999999999998e-05 2024-09-16T14:49:42.048 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.17s4 91/90 ECSubWrite(tid=395, reqid=osd.3.0:395, at_version=91'146, trim_to=0'0, roll_forward_to=91'144))", 2024-09-16T14:49:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 3.951053597, 2024-09-16T14:49:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0076634049999999999, 2024-09-16T14:49:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009478+0000", 2024-09-16T14:49:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4235000000000001e-05 2024-09-16T14:49:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009486+0000", 2024-09-16T14:49:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7940000000000003e-06 2024-09-16T14:49:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009492+0000", 2024-09-16T14:49:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7980000000000002e-06 2024-09-16T14:49:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010420+0000", 2024-09-16T14:49:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00092807100000000004 2024-09-16T14:49:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010429+0000", 2024-09-16T14:49:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8389999999999996e-06 2024-09-16T14:49:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017094+0000", 2024-09-16T14:49:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066650879999999996 2024-09-16T14:49:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017118+0000", 2024-09-16T14:49:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3941e-05 2024-09-16T14:49:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.053 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.053 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:42.056 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:49:42.056 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:42.053+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:42.058 INFO:tasks.rados.rados.0.smithi040.stdout:357: writing smithi04040151-22 from 3309568 to 3883008 tid 1 2024-09-16T14:49:42.058 INFO:tasks.rados.rados.0.smithi040.stdout:357: writing smithi04040151-22 from 3883008 to 4505600 tid 2 2024-09-16T14:49:42.058 INFO:tasks.rados.rados.0.smithi040.stdout:357: writing smithi04040151-22 from 4505600 to 5128192 tid 3 2024-09-16T14:49:42.058 INFO:tasks.rados.rados.0.smithi040.stdout:357: writing smithi04040151-22 from 5128192 to 5439488 tid 4 2024-09-16T14:49:42.058 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 2 current snap is 33 2024-09-16T14:49:42.058 INFO:tasks.rados.rados.0.smithi040.stdout:357: finishing write tid 1 to smithi04040151-22 2024-09-16T14:49:42.058 INFO:tasks.rados.rados.0.smithi040.stdout:357: finishing write tid 2 to smithi04040151-22 2024-09-16T14:49:42.058 INFO:tasks.rados.rados.0.smithi040.stdout:358: seq_num 127 ranges {1654784=589824,2244608=753664,2998272=475136,3473408=327680} 2024-09-16T14:49:42.059 INFO:tasks.rados.rados.0.smithi040.stdout:358: writing smithi04040151-2 from 1654784 to 2244608 tid 1 2024-09-16T14:49:42.059 INFO:tasks.rados.rados.0.smithi040.stdout:358: writing smithi04040151-2 from 2244608 to 2998272 tid 2 2024-09-16T14:49:42.059 INFO:tasks.rados.rados.0.smithi040.stdout:358: writing smithi04040151-2 from 2998272 to 3473408 tid 3 2024-09-16T14:49:42.059 INFO:tasks.rados.rados.0.smithi040.stdout:358: writing smithi04040151-2 from 3473408 to 3801088 tid 4 2024-09-16T14:49:42.059 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 37 current snap is 33 2024-09-16T14:49:42.059 INFO:tasks.rados.rados.0.smithi040.stdout:359: seq_num 128 ranges {0=606208,606208=507904,1114112=573440,1687552=475136} 2024-09-16T14:49:42.059 INFO:tasks.rados.rados.0.smithi040.stdout:359: writing smithi04040151-37 from 0 to 606208 tid 1 2024-09-16T14:49:42.059 INFO:tasks.rados.rados.0.smithi040.stdout:359: writing smithi04040151-37 from 606208 to 1114112 tid 2 2024-09-16T14:49:42.059 INFO:tasks.rados.rados.0.smithi040.stdout:359: writing smithi04040151-37 from 1114112 to 1687552 tid 3 2024-09-16T14:49:42.059 INFO:tasks.rados.rados.0.smithi040.stdout:359: writing smithi04040151-37 from 1687552 to 2162688 tid 4 2024-09-16T14:49:42.059 INFO:tasks.rados.rados.0.smithi040.stdout:360: snap_create 2024-09-16T14:49:42.059 INFO:tasks.rados.rados.0.smithi040.stdout:357: finishing write tid 3 to smithi04040151-22 2024-09-16T14:49:42.059 INFO:tasks.rados.rados.0.smithi040.stdout:357: finishing write tid 4 to smithi04040151-22 2024-09-16T14:49:42.060 INFO:tasks.rados.rados.0.smithi040.stdout:357: finishing write tid 5 to smithi04040151-22 2024-09-16T14:49:42.060 INFO:tasks.rados.rados.0.smithi040.stdout:357: finishing write tid 6 to smithi04040151-22 2024-09-16T14:49:42.060 INFO:tasks.rados.rados.0.smithi040.stdout:357: oid 22 updating version 0 to 65 2024-09-16T14:49:42.060 INFO:tasks.rados.rados.0.smithi040.stdout:357: oid 22 updating version 65 to 67 2024-09-16T14:49:42.060 INFO:tasks.rados.rados.0.smithi040.stdout:357: oid 22 updating version 67 to 68 2024-09-16T14:49:42.060 INFO:tasks.rados.rados.0.smithi040.stdout:357: oid 22 version 68 is already newer than 64 2024-09-16T14:49:42.060 INFO:tasks.rados.rados.0.smithi040.stdout:357: oid 22 version 68 is already newer than 66 2024-09-16T14:49:42.060 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 22 v 68 (ObjNum 126 snap 33 seq_num 126) dirty exists 2024-09-16T14:49:42.060 INFO:tasks.rados.rados.0.smithi040.stdout:357: left oid 22 (ObjNum 126 snap 33 seq_num 126) 2024-09-16T14:49:42.060 INFO:tasks.rados.rados.0.smithi040.stdout:358: finishing write tid 1 to smithi04040151-2 2024-09-16T14:49:42.060 INFO:tasks.rados.rados.0.smithi040.stdout:358: finishing write tid 2 to smithi04040151-2 2024-09-16T14:49:42.060 INFO:tasks.rados.rados.0.smithi040.stdout:358: finishing write tid 3 to smithi04040151-2 2024-09-16T14:49:42.060 INFO:tasks.rados.rados.0.smithi040.stdout:358: finishing write tid 4 to smithi04040151-2 2024-09-16T14:49:42.060 INFO:tasks.rados.rados.0.smithi040.stdout:358: finishing write tid 5 to smithi04040151-2 2024-09-16T14:49:42.061 INFO:tasks.rados.rados.0.smithi040.stdout:358: finishing write tid 6 to smithi04040151-2 2024-09-16T14:49:42.061 INFO:tasks.rados.rados.0.smithi040.stdout:358: oid 2 updating version 0 to 64 2024-09-16T14:49:42.061 INFO:tasks.rados.rados.0.smithi040.stdout:358: oid 2 updating version 64 to 65 2024-09-16T14:49:42.061 INFO:tasks.rados.rados.0.smithi040.stdout:358: oid 2 updating version 65 to 68 2024-09-16T14:49:42.061 INFO:tasks.rados.rados.0.smithi040.stdout:358: oid 2 version 68 is already newer than 66 2024-09-16T14:49:42.061 INFO:tasks.rados.rados.0.smithi040.stdout:358: oid 2 version 68 is already newer than 67 2024-09-16T14:49:42.061 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 2 v 68 (ObjNum 127 snap 33 seq_num 127) dirty exists 2024-09-16T14:49:42.061 INFO:tasks.rados.rados.0.smithi040.stdout:358: left oid 2 (ObjNum 127 snap 33 seq_num 127) 2024-09-16T14:49:42.061 INFO:tasks.rados.rados.0.smithi040.stdout:359: finishing write tid 1 to smithi04040151-37 2024-09-16T14:49:42.061 INFO:tasks.rados.rados.0.smithi040.stdout:359: finishing write tid 2 to smithi04040151-37 2024-09-16T14:49:42.061 INFO:tasks.rados.rados.0.smithi040.stdout:359: finishing write tid 3 to smithi04040151-37 2024-09-16T14:49:42.061 INFO:tasks.rados.rados.0.smithi040.stdout:359: finishing write tid 4 to smithi04040151-37 2024-09-16T14:49:42.061 INFO:tasks.rados.rados.0.smithi040.stdout:359: finishing write tid 5 to smithi04040151-37 2024-09-16T14:49:42.062 INFO:tasks.rados.rados.0.smithi040.stdout:359: finishing write tid 6 to smithi04040151-37 2024-09-16T14:49:42.062 INFO:tasks.rados.rados.0.smithi040.stdout:359: oid 37 updating version 0 to 107 2024-09-16T14:49:42.062 INFO:tasks.rados.rados.0.smithi040.stdout:359: oid 37 updating version 107 to 108 2024-09-16T14:49:42.062 INFO:tasks.rados.rados.0.smithi040.stdout:359: oid 37 version 108 is already newer than 105 2024-09-16T14:49:42.062 INFO:tasks.rados.rados.0.smithi040.stdout:359: oid 37 updating version 108 to 109 2024-09-16T14:49:42.062 INFO:tasks.rados.rados.0.smithi040.stdout:359: oid 37 version 109 is already newer than 106 2024-09-16T14:49:42.062 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 37 v 109 (ObjNum 128 snap 33 seq_num 128) dirty exists 2024-09-16T14:49:42.062 INFO:tasks.rados.rados.0.smithi040.stdout:359: left oid 37 (ObjNum 128 snap 33 seq_num 128) 2024-09-16T14:49:42.156 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:42.155+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:42.199 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:42.199 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:42.199 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:42.199 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:42.208 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:49:42.257 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:42.256+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:42.350 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:42.350 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:42.350 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:42.350 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:42.350 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:42.357 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:42.356+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:42.359 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:49:42.459 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:42.457+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:42.504 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:42.504 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:42.504 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:42.504 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:42.504 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.504 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:229 3.es0 3:7ecfd044:::smithi04040151-35:head [copy-from ver 4] snapc 4=[4,2] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:49:42.504 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:42.504 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 61.597648257000003, 2024-09-16T14:49:42.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016506461, 2024-09-16T14:49:42.505 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.505 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:42.505 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.505 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.505 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.505 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 229 2024-09-16T14:49:42.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.505 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.505 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.505 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.505 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:42.505 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904168+0000", 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904187+0000", 2024-09-16T14:49:42.506 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9449999999999998e-05 2024-09-16T14:49:42.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904206+0000", 2024-09-16T14:49:42.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8698e-05 2024-09-16T14:49:42.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904215+0000", 2024-09-16T14:49:42.507 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:49:42.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.507 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.507 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.507 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904276+0000", 2024-09-16T14:49:42.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1130999999999995e-05 2024-09-16T14:49:42.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:42.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904284+0000", 2024-09-16T14:49:42.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0609999999999996e-06 2024-09-16T14:49:42.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.508 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906272+0000", 2024-09-16T14:49:42.508 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019884210000000002 2024-09-16T14:49:42.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.508 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.508 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906391+0000", 2024-09-16T14:49:42.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000118843 2024-09-16T14:49:42.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.912595+0000", 2024-09-16T14:49:42.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062038190000000002 2024-09-16T14:49:42.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.509 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.509 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917584+0000", 2024-09-16T14:49:42.509 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049889069999999999 2024-09-16T14:49:42.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:42.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920592+0000", 2024-09-16T14:49:42.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030081639999999998 2024-09-16T14:49:42.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.510 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.510 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920676+0000", 2024-09-16T14:49:42.510 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3974999999999995e-05 2024-09-16T14:49:42.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.510 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.510 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.510 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:298 3.es0 3:7ecfd044:::smithi04040151-35:head [append 0~720896] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.572984441999999, 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014187808, 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 298 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.511 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:42.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:42.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.512 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.512 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928832+0000", 2024-09-16T14:49:42.512 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:42.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.512 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929746+0000", 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00091434199999999995 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929760+0000", 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3519e-05 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929767+0000", 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3180000000000001e-06 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.513 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929849+0000", 2024-09-16T14:49:42.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2020000000000004e-05 2024-09-16T14:49:42.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:42.514 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929854+0000", 2024-09-16T14:49:42.514 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1209999999999998e-06 2024-09-16T14:49:42.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.514 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.514 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931714+0000", 2024-09-16T14:49:42.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001859989 2024-09-16T14:49:42.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931845+0000", 2024-09-16T14:49:42.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130901 2024-09-16T14:49:42.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.515 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.515 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933089+0000", 2024-09-16T14:49:42.515 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001243535 2024-09-16T14:49:42.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.515 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936370+0000", 2024-09-16T14:49:42.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032817660000000002 2024-09-16T14:49:42.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:42.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942945+0000", 2024-09-16T14:49:42.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065741560000000003 2024-09-16T14:49:42.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.516 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.516 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.516 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943021+0000", 2024-09-16T14:49:42.516 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6734000000000001e-05 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:299 3.es0 3:7ecfd044:::smithi04040151-35:head [append 0~376832] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.570694641999999, 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012149627, 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.517 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 299 2024-09-16T14:49:42.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.518 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:42.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.518 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.518 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.518 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:42.518 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931122+0000", 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931722+0000", 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059944100000000004 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931732+0000", 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9369999999999999e-06 2024-09-16T14:49:42.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931739+0000", 2024-09-16T14:49:42.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2819999999999999e-06 2024-09-16T14:49:42.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933835+0000", 2024-09-16T14:49:42.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020961140000000001 2024-09-16T14:49:42.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.520 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.520 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.520 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933929+0000", 2024-09-16T14:49:42.520 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3683000000000004e-05 2024-09-16T14:49:42.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.934917+0000", 2024-09-16T14:49:42.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00098803200000000002 2024-09-16T14:49:42.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.939558+0000", 2024-09-16T14:49:42.521 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00464175 2024-09-16T14:49:42.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.521 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.521 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:42.521 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943205+0000", 2024-09-16T14:49:42.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036465820000000002 2024-09-16T14:49:42.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.522 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.522 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943273+0000", 2024-09-16T14:49:42.522 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7917999999999996e-05 2024-09-16T14:49:42.522 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.522 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.522 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.522 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.522 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:300 3.es0 3:7ecfd044:::smithi04040151-35:head [setxattr _header (56) in=7b] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:42.523 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:42.523 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.570068548000002, 2024-09-16T14:49:42.523 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011740911, 2024-09-16T14:49:42.523 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.523 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:42.523 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.523 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.523 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.523 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 300 2024-09-16T14:49:42.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.523 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.523 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.523 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.523 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:42.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:42.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.524 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.524 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:42.524 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:42.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.524 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931753+0000", 2024-09-16T14:49:42.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0380000000000001e-06 2024-09-16T14:49:42.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931759+0000", 2024-09-16T14:49:42.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3430000000000001e-06 2024-09-16T14:49:42.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.525 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.525 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931765+0000", 2024-09-16T14:49:42.525 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-16T14:49:42.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.935623+0000", 2024-09-16T14:49:42.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038583049999999998 2024-09-16T14:49:42.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.935717+0000", 2024-09-16T14:49:42.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4587999999999996e-05 2024-09-16T14:49:42.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.526 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.526 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.526 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936150+0000", 2024-09-16T14:49:42.526 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043270599999999999 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.939576+0000", 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034257620000000002 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943399+0000", 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003823145 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943490+0000", 2024-09-16T14:49:42.527 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1241000000000003e-05 2024-09-16T14:49:42.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.528 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.528 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.528 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.528 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:301 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:42.528 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:42.528 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.570047248000002, 2024-09-16T14:49:42.528 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014215955000000001, 2024-09-16T14:49:42.528 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.528 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:42.528 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.529 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.529 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.529 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 301 2024-09-16T14:49:42.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.529 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.529 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:42.529 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.529 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.529 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:42.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931770+0000", 2024-09-16T14:49:42.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.530 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.530 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.530 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931775+0000", 2024-09-16T14:49:42.530 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4530000000000004e-06 2024-09-16T14:49:42.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931782+0000", 2024-09-16T14:49:42.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0550000000000002e-06 2024-09-16T14:49:42.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.531 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931787+0000", 2024-09-16T14:49:42.531 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2310000000000004e-06 2024-09-16T14:49:42.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.531 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936401+0000", 2024-09-16T14:49:42.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046143419999999996 2024-09-16T14:49:42.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:42.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936482+0000", 2024-09-16T14:49:42.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0407000000000005e-05 2024-09-16T14:49:42.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.532 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.532 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.532 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943851+0000", 2024-09-16T14:49:42.532 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073695000000000002 2024-09-16T14:49:42.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943951+0000", 2024-09-16T14:49:42.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9327000000000006e-05 2024-09-16T14:49:42.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.533 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.533 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.533 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.945987+0000", 2024-09-16T14:49:42.533 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002036057 2024-09-16T14:49:42.533 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.533 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.533 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.534 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.534 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:369 3.8s0 3:1f96bdfc:::smithi04040151-37:head [append 0~475136] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:42.534 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:42.534 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 53.544105254999998, 2024-09-16T14:49:42.534 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010979078999999999, 2024-09-16T14:49:42.534 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.534 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:42.534 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.534 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.534 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.534 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 369 2024-09-16T14:49:42.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.534 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:42.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:42.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.535 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.535 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957711+0000", 2024-09-16T14:49:42.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:42.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958323+0000", 2024-09-16T14:49:42.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000612439 2024-09-16T14:49:42.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958339+0000", 2024-09-16T14:49:42.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5985999999999999e-05 2024-09-16T14:49:42.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958348+0000", 2024-09-16T14:49:42.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8699999999999998e-06 2024-09-16T14:49:42.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958393+0000", 2024-09-16T14:49:42.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4981000000000002e-05 2024-09-16T14:49:42.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958478+0000", 2024-09-16T14:49:42.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4845999999999994e-05 2024-09-16T14:49:42.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.959659+0000", 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011808999999999999 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962687+0000", 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030278330000000002 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968651+0000", 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005964764 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968692+0000", 2024-09-16T14:49:42.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0374999999999999e-05 2024-09-16T14:49:42.539 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.539 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.539 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.539 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:387 3.8s0 3:1f96bdfc:::smithi04040151-37:head [read 0~2686976,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:42.539 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:49:42.539 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.637354895000001, 2024-09-16T14:49:42.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010154198999999999, 2024-09-16T14:49:42.540 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.540 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:42.540 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.540 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.540 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.540 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 387 2024-09-16T14:49:42.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.540 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:49:42.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:49:42.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864463+0000", 2024-09-16T14:49:42.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864465+0000", 2024-09-16T14:49:42.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.774e-06 2024-09-16T14:49:42.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864472+0000", 2024-09-16T14:49:42.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0040000000000002e-06 2024-09-16T14:49:42.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.864477+0000", 2024-09-16T14:49:42.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0779999999999997e-06 2024-09-16T14:49:42.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.866679+0000", 2024-09-16T14:49:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00220228 2024-09-16T14:49:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.866770+0000", 2024-09-16T14:49:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0952999999999995e-05 2024-09-16T14:49:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874617+0000", 2024-09-16T14:49:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007847527 2024-09-16T14:49:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.544 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.544 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:398 3.as0 3:572aff57:::smithi04040151-20:head [append 0~507904] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:42.544 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:42.544 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.622740489000002, 2024-09-16T14:49:42.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010447988, 2024-09-16T14:49:42.544 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.544 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:42.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.544 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.544 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 398 2024-09-16T14:49:42.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.545 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:42.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.545 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.545 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:42.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879076+0000", 2024-09-16T14:49:42.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:42.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881851+0000", 2024-09-16T14:49:42.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0027753420000000001 2024-09-16T14:49:42.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881863+0000", 2024-09-16T14:49:42.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2203e-05 2024-09-16T14:49:42.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881871+0000", 2024-09-16T14:49:42.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4059999999999999e-06 2024-09-16T14:49:42.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881924+0000", 2024-09-16T14:49:42.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2932000000000001e-05 2024-09-16T14:49:42.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882118+0000", 2024-09-16T14:49:42.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019413500000000001 2024-09-16T14:49:42.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884114+0000", 2024-09-16T14:49:42.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001996154 2024-09-16T14:49:42.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889295+0000", 2024-09-16T14:49:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051809899999999999 2024-09-16T14:49:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889480+0000", 2024-09-16T14:49:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018519700000000001 2024-09-16T14:49:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889525+0000", 2024-09-16T14:49:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5399999999999999e-05 2024-09-16T14:49:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.549 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.550 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.550 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:400 3.as0 3:572aff57:::smithi04040151-20:head [append 0~262144] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:42.550 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:42.550 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.618231459, 2024-09-16T14:49:42.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010619327, 2024-09-16T14:49:42.550 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.550 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:42.550 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.550 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.550 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.550 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 400 2024-09-16T14:49:42.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.551 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:42.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:42.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883585+0000", 2024-09-16T14:49:42.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:42.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883868+0000", 2024-09-16T14:49:42.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028385599999999997 2024-09-16T14:49:42.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883880+0000", 2024-09-16T14:49:42.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1181e-05 2024-09-16T14:49:42.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883888+0000", 2024-09-16T14:49:42.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.926e-06 2024-09-16T14:49:42.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887739+0000", 2024-09-16T14:49:42.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003851658 2024-09-16T14:49:42.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887849+0000", 2024-09-16T14:49:42.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109616 2024-09-16T14:49:42.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.888684+0000", 2024-09-16T14:49:42.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083518000000000004 2024-09-16T14:49:42.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894010+0000", 2024-09-16T14:49:42.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053260670000000003 2024-09-16T14:49:42.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:42.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894143+0000", 2024-09-16T14:49:42.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000133077 2024-09-16T14:49:42.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894206+0000", 2024-09-16T14:49:42.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2658999999999998e-05 2024-09-16T14:49:42.555 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.555 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.555 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.556 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:401 3.as0 3:572aff57:::smithi04040151-20:head [setxattr _header (57) in=7b] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:42.556 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:42.556 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.617921535000001, 2024-09-16T14:49:42.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010524774000000001, 2024-09-16T14:49:42.556 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.556 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:42.556 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.556 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.556 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.556 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 401 2024-09-16T14:49:42.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.556 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:42.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:42.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.557 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.557 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:42.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883900+0000", 2024-09-16T14:49:42.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1400000000000002e-06 2024-09-16T14:49:42.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883908+0000", 2024-09-16T14:49:42.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2689999999999999e-06 2024-09-16T14:49:42.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883913+0000", 2024-09-16T14:49:42.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-16T14:49:42.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889560+0000", 2024-09-16T14:49:42.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005647165 2024-09-16T14:49:42.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889627+0000", 2024-09-16T14:49:42.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6951000000000003e-05 2024-09-16T14:49:42.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889898+0000", 2024-09-16T14:49:42.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270709 2024-09-16T14:49:42.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894012+0000", 2024-09-16T14:49:42.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041137810000000004 2024-09-16T14:49:42.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:42.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894344+0000", 2024-09-16T14:49:42.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000332121 2024-09-16T14:49:42.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894421+0000", 2024-09-16T14:49:42.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7406999999999999e-05 2024-09-16T14:49:42.561 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.561 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.561 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.561 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:402 3.as0 3:572aff57:::smithi04040151-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:42.561 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:42.561 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.617899659000003, 2024-09-16T14:49:42.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011864743000000001, 2024-09-16T14:49:42.562 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.562 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:42.562 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.562 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.562 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.562 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 402 2024-09-16T14:49:42.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.562 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:42.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:42.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:42.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883921+0000", 2024-09-16T14:49:42.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7520000000000001e-06 2024-09-16T14:49:42.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883927+0000", 2024-09-16T14:49:42.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4679999999999997e-06 2024-09-16T14:49:42.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.564 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883932+0000", 2024-09-16T14:49:42.564 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1440000000000002e-06 2024-09-16T14:49:42.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.564 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890477+0000", 2024-09-16T14:49:42.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065446430000000002 2024-09-16T14:49:42.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:42.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890586+0000", 2024-09-16T14:49:42.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010909 2024-09-16T14:49:42.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894750+0000", 2024-09-16T14:49:42.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00416359 2024-09-16T14:49:42.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894834+0000", 2024-09-16T14:49:42.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4596000000000001e-05 2024-09-16T14:49:42.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895783+0000", 2024-09-16T14:49:42.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00094892000000000002 2024-09-16T14:49:42.566 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.566 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.566 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.567 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:443 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:42.567 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:42.567 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.625689086000001, 2024-09-16T14:49:42.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.013766494000000001, 2024-09-16T14:49:42.567 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.567 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:42.567 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.567 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.567 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.567 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 443 2024-09-16T14:49:42.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.567 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:42.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:42.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876127+0000", 2024-09-16T14:49:42.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:42.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876138+0000", 2024-09-16T14:49:42.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1344e-05 2024-09-16T14:49:42.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876159+0000", 2024-09-16T14:49:42.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1157000000000001e-05 2024-09-16T14:49:42.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876170+0000", 2024-09-16T14:49:42.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0651e-05 2024-09-16T14:49:42.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876234+0000", 2024-09-16T14:49:42.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4114000000000003e-05 2024-09-16T14:49:42.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:42.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876243+0000", 2024-09-16T14:49:42.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0049999999999993e-06 2024-09-16T14:49:42.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879445+0000", 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032019269999999998 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879550+0000", 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010536500000000001 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889895+0000", 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010345043 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.571 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.572 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:444 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:42.572 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:42.572 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.625599379000001, 2024-09-16T14:49:42.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014716241, 2024-09-16T14:49:42.572 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.572 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:42.572 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.572 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.572 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.572 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 444 2024-09-16T14:49:42.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.572 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:42.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:42.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876218+0000", 2024-09-16T14:49:42.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:42.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876224+0000", 2024-09-16T14:49:42.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:49:42.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876235+0000", 2024-09-16T14:49:42.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.025e-05 2024-09-16T14:49:42.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876242+0000", 2024-09-16T14:49:42.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8790000000000003e-06 2024-09-16T14:49:42.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876315+0000", 2024-09-16T14:49:42.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2260000000000003e-05 2024-09-16T14:49:42.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:42.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876320+0000", 2024-09-16T14:49:42.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5690000000000004e-06 2024-09-16T14:49:42.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880078+0000", 2024-09-16T14:49:42.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0037576430000000002 2024-09-16T14:49:42.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880168+0000", 2024-09-16T14:49:42.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0353999999999996e-05 2024-09-16T14:49:42.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.890935+0000", 2024-09-16T14:49:42.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010766556 2024-09-16T14:49:42.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.577 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.577 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:445 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:42.577 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:42.577 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.625555396999999, 2024-09-16T14:49:42.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016529716999999999, 2024-09-16T14:49:42.577 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.577 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:42.577 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.577 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.577 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.578 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 445 2024-09-16T14:49:42.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.578 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:42.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:42.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876262+0000", 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876269+0000", 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9469999999999997e-06 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876276+0000", 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5950000000000004e-06 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876281+0000", 2024-09-16T14:49:42.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:49:42.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876343+0000", 2024-09-16T14:49:42.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1685000000000003e-05 2024-09-16T14:49:42.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:42.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876348+0000", 2024-09-16T14:49:42.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7269999999999998e-06 2024-09-16T14:49:42.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881091+0000", 2024-09-16T14:49:42.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047434599999999997 2024-09-16T14:49:42.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881181+0000", 2024-09-16T14:49:42.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9838000000000003e-05 2024-09-16T14:49:42.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.892792+0000", 2024-09-16T14:49:42.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011611316 2024-09-16T14:49:42.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.582 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.582 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s1 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:42.582 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:42.582 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.568745147000001, 2024-09-16T14:49:42.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011783765999999999, 2024-09-16T14:49:42.582 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.583 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.583 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:42.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:42.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933072+0000", 2024-09-16T14:49:42.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:42.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933092+0000", 2024-09-16T14:49:42.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0000000000000002e-05 2024-09-16T14:49:42.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933119+0000", 2024-09-16T14:49:42.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6877e-05 2024-09-16T14:49:42.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933127+0000", 2024-09-16T14:49:42.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4600000000000003e-06 2024-09-16T14:49:42.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.937087+0000", 2024-09-16T14:49:42.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0039599129999999998 2024-09-16T14:49:42.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.937096+0000", 2024-09-16T14:49:42.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8910000000000001e-06 2024-09-16T14:49:42.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944834+0000", 2024-09-16T14:49:42.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077376809999999997 2024-09-16T14:49:42.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944857+0000", 2024-09-16T14:49:42.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2785999999999998e-05 2024-09-16T14:49:42.587 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.587 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.587 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.587 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:591 3.0s0 3:060eef9f:::smithi04040151-16:head [copy-from ver 27] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:42.587 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:42.587 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.525946138999998, 2024-09-16T14:49:42.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0279670919999999, 2024-09-16T14:49:42.588 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.588 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:42.588 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.588 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.588 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.588 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 591 2024-09-16T14:49:42.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.588 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:42.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:42.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975870+0000", 2024-09-16T14:49:42.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:42.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975878+0000", 2024-09-16T14:49:42.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2619999999999998e-06 2024-09-16T14:49:42.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975902+0000", 2024-09-16T14:49:42.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4036000000000002e-05 2024-09-16T14:49:42.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975912+0000", 2024-09-16T14:49:42.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-16T14:49:42.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975986+0000", 2024-09-16T14:49:42.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3548999999999997e-05 2024-09-16T14:49:42.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:42.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975990+0000", 2024-09-16T14:49:42.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7670000000000001e-06 2024-09-16T14:49:42.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.975658+0000", 2024-09-16T14:49:42.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99966882800000001 2024-09-16T14:49:42.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for missing object", 2024-09-16T14:49:42.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.975885+0000", 2024-09-16T14:49:42.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000226682 2024-09-16T14:49:42.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.982217+0000", 2024-09-16T14:49:42.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063321530000000001 2024-09-16T14:49:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.982350+0000", 2024-09-16T14:49:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013316700000000001 2024-09-16T14:49:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.999798+0000", 2024-09-16T14:49:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017447608 2024-09-16T14:49:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001884+0000", 2024-09-16T14:49:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020859889999999999 2024-09-16T14:49:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.003629+0000", 2024-09-16T14:49:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001745106 2024-09-16T14:49:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.003839+0000", 2024-09-16T14:49:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020980099999999999 2024-09-16T14:49:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.594 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:592 3.0s0 3:060eef9f:::smithi04040151-16:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 30.525889214999999, 2024-09-16T14:49:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0256284010000001, 2024-09-16T14:49:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 592 2024-09-16T14:49:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.595 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975928+0000", 2024-09-16T14:49:42.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:42.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975936+0000", 2024-09-16T14:49:42.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6090000000000003e-06 2024-09-16T14:49:42.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975943+0000", 2024-09-16T14:49:42.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:42.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975949+0000", 2024-09-16T14:49:42.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8300000000000001e-06 2024-09-16T14:49:42.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.976030+0000", 2024-09-16T14:49:42.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1267000000000006e-05 2024-09-16T14:49:42.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:42.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.976033+0000", 2024-09-16T14:49:42.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7590000000000002e-06 2024-09-16T14:49:42.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.977041+0000", 2024-09-16T14:49:42.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0010081639999999 2024-09-16T14:49:42.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for missing object", 2024-09-16T14:49:42.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.977086+0000", 2024-09-16T14:49:42.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4879999999999997e-05 2024-09-16T14:49:42.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.983015+0000", 2024-09-16T14:49:42.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059287480000000002 2024-09-16T14:49:42.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:42.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.983090+0000", 2024-09-16T14:49:42.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5081999999999997e-05 2024-09-16T14:49:42.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001193+0000", 2024-09-16T14:49:42.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.018102713999999999 2024-09-16T14:49:42.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001306+0000", 2024-09-16T14:49:42.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011330799999999999 2024-09-16T14:49:42.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001557+0000", 2024-09-16T14:49:42.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025100900000000001 2024-09-16T14:49:42.601 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.601 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.601 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.601 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs4 90/65 ECSubWrite(tid=380, reqid=osd.3.0:380, at_version=90'77, trim_to=0'0, roll_forward_to=83'73))", 2024-09-16T14:49:42.602 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.950581+0000", 2024-09-16T14:49:42.602 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.5512370210000004, 2024-09-16T14:49:42.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010219577000000001, 2024-09-16T14:49:42.602 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.602 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.602 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.950581+0000", 2024-09-16T14:49:42.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.950581+0000", 2024-09-16T14:49:42.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.950580+0000", 2024-09-16T14:49:42.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:42.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.950608+0000", 2024-09-16T14:49:42.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7637e-05 2024-09-16T14:49:42.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.951627+0000", 2024-09-16T14:49:42.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001019439 2024-09-16T14:49:42.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.951638+0000", 2024-09-16T14:49:42.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0930999999999999e-05 2024-09-16T14:49:42.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.951676+0000", 2024-09-16T14:49:42.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7883999999999998e-05 2024-09-16T14:49:42.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.951686+0000", 2024-09-16T14:49:42.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0675e-05 2024-09-16T14:49:42.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.960787+0000", 2024-09-16T14:49:42.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091004250000000005 2024-09-16T14:49:42.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.960801+0000", 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3613e-05 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(osd.8.17:4 3.es0 3:7ecfd044:::smithi04040151-35:head [assert-version v52,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.971941+0000", 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.52987684, 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016070661, 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971941+0000", 2024-09-16T14:49:42.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971941+0000", 2024-09-16T14:49:42.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971939+0000", 2024-09-16T14:49:42.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:42.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971949+0000", 2024-09-16T14:49:42.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2420000000000006e-06 2024-09-16T14:49:42.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971959+0000", 2024-09-16T14:49:42.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0827e-05 2024-09-16T14:49:42.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971969+0000", 2024-09-16T14:49:42.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6579999999999997e-06 2024-09-16T14:49:42.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.972028+0000", 2024-09-16T14:49:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9457999999999999e-05 2024-09-16T14:49:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.972120+0000", 2024-09-16T14:49:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1174999999999994e-05 2024-09-16T14:49:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.988012+0000", 2024-09-16T14:49:42.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015892125 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.19s1 91/90 ECSubWrite(tid=102, reqid=client.4500.0:822, at_version=91'77, trim_to=0'0, roll_forward_to=91'75))", 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.006520+0000", 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.4952981310000002, 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01031002, 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:42.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006520+0000", 2024-09-16T14:49:42.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:42.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006520+0000", 2024-09-16T14:49:42.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:42.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:42.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006519+0000", 2024-09-16T14:49:42.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:42.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006954+0000", 2024-09-16T14:49:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043463699999999998 2024-09-16T14:49:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006973+0000", 2024-09-16T14:49:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9389e-05 2024-09-16T14:49:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006994+0000", 2024-09-16T14:49:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1229999999999998e-05 2024-09-16T14:49:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:42.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007063+0000", 2024-09-16T14:49:42.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8979999999999993e-05 2024-09-16T14:49:42.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007077+0000", 2024-09-16T14:49:42.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4119e-05 2024-09-16T14:49:42.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016803+0000", 2024-09-16T14:49:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0097257460000000004 2024-09-16T14:49:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016830+0000", 2024-09-16T14:49:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6835999999999999e-05 2024-09-16T14:49:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:42.614 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:42.614 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:42.616 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:49:42.616 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:42.614+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:42.717 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:42.716+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:42.719 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:42.719 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:42.719 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:42.719 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:42.728 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:49:42.818 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:42.817+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:42.830 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:42.830 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:42.830 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:42.830 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:42.830 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:42.838 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:49:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T14:49:42.862 INFO:tasks.rados.rados.0.smithi040.stdout:357: done (5 left) 2024-09-16T14:49:42.863 INFO:tasks.rados.rados.0.smithi040.stdout:358: done (4 left) 2024-09-16T14:49:42.863 INFO:tasks.rados.rados.0.smithi040.stdout:359: done (3 left) 2024-09-16T14:49:42.863 INFO:tasks.rados.rados.0.smithi040.stdout:360: done (2 left) 2024-09-16T14:49:42.863 INFO:tasks.rados.rados.0.smithi040.stdout:361: snap_remove snap 33 2024-09-16T14:49:42.889 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T14:49:42.919 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:42.918+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:42.951 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:42.952 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:42.952 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:42.952 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:42.952 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.952 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 66/65 ECSubWrite(tid=39, reqid=client.4500.0:597, at_version=66'26, trim_to=0'0, roll_forward_to=66'23))", 2024-09-16T14:49:42.952 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:42.952 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.937086469, 2024-09-16T14:49:42.952 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0060634030000000002, 2024-09-16T14:49:42.952 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:42.952 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.952 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:42.952 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.952 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:42.953 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:42.953 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.953 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.953 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.953 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:42.953 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:42.953 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.953 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.953 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.953 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:42.953 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012388+0000", 2024-09-16T14:49:42.953 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:42.953 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.954 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.954 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:42.954 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012433+0000", 2024-09-16T14:49:42.954 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4588000000000001e-05 2024-09-16T14:49:42.954 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.954 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.954 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:42.954 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012604+0000", 2024-09-16T14:49:42.954 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017141300000000001 2024-09-16T14:49:42.954 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.954 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.954 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.955 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012617+0000", 2024-09-16T14:49:42.955 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2792999999999999e-05 2024-09-16T14:49:42.955 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.955 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.955 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:42.955 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012666+0000", 2024-09-16T14:49:42.955 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9079000000000002e-05 2024-09-16T14:49:42.955 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.955 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.955 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.955 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012678+0000", 2024-09-16T14:49:42.955 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1405e-05 2024-09-16T14:49:42.955 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.956 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.956 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.956 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018433+0000", 2024-09-16T14:49:42.956 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005755146 2024-09-16T14:49:42.956 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.956 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.956 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:42.956 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018453+0000", 2024-09-16T14:49:42.956 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9780999999999999e-05 2024-09-16T14:49:42.956 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:42.956 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:42.956 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:42.956 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.957 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.957 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:702 3.as0 3:572aff57:::smithi04040151-20:head [copy-from ver 4] snapc 2f=[2f,2e,29,28,25] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:42.957 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:42.957 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 19.049398187000001, 2024-09-16T14:49:42.957 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.032342711000000003, 2024-09-16T14:49:42.957 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:42.957 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:42.957 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:42.957 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:42.957 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.957 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 702 2024-09-16T14:49:42.957 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.957 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:42.958 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.958 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:42.958 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:42.958 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.958 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.958 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.958 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:42.958 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:42.958 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.958 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.958 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.958 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:42.958 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900076+0000", 2024-09-16T14:49:42.958 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:42.959 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.959 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.959 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:42.959 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900082+0000", 2024-09-16T14:49:42.959 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3640000000000004e-06 2024-09-16T14:49:42.959 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.959 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.959 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:42.959 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900094+0000", 2024-09-16T14:49:42.959 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1208e-05 2024-09-16T14:49:42.959 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.959 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.959 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.959 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900105+0000", 2024-09-16T14:49:42.960 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1782e-05 2024-09-16T14:49:42.960 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.960 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.960 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:42.960 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900161+0000", 2024-09-16T14:49:42.960 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5244999999999997e-05 2024-09-16T14:49:42.960 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.960 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.960 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:42.960 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900170+0000", 2024-09-16T14:49:42.960 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4409999999999999e-06 2024-09-16T14:49:42.960 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.961 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.961 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:42.961 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.901930+0000", 2024-09-16T14:49:42.961 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001760246 2024-09-16T14:49:42.961 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.961 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.961 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:42.961 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902046+0000", 2024-09-16T14:49:42.961 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000115882 2024-09-16T14:49:42.961 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.961 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.961 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.961 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.917553+0000", 2024-09-16T14:49:42.962 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015506967999999999 2024-09-16T14:49:42.962 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.962 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.962 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.962 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918732+0000", 2024-09-16T14:49:42.962 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001178733 2024-09-16T14:49:42.962 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.962 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.962 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:42.962 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932379+0000", 2024-09-16T14:49:42.962 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013647276999999999 2024-09-16T14:49:42.962 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.962 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.962 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:42.963 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932420+0000", 2024-09-16T14:49:42.963 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1066999999999998e-05 2024-09-16T14:49:42.963 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:42.963 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:42.963 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:42.963 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.963 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.963 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:703 3.as0 3:572aff57:::smithi04040151-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:42.963 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:42.964 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 19.049356789000001, 2024-09-16T14:49:42.964 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.018566787000000001, 2024-09-16T14:49:42.964 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:42.964 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:42.964 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:42.964 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:42.964 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.964 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 703 2024-09-16T14:49:42.964 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.964 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:42.965 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.965 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:42.965 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:42.965 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.965 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.965 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.965 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:42.965 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:42.965 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.965 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.965 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.966 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:42.966 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900118+0000", 2024-09-16T14:49:42.966 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.966 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.966 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.966 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:42.966 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900122+0000", 2024-09-16T14:49:42.966 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5669999999999999e-06 2024-09-16T14:49:42.966 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.966 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.967 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:42.967 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900129+0000", 2024-09-16T14:49:42.967 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0360000000000001e-06 2024-09-16T14:49:42.967 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.967 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.967 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.967 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900134+0000", 2024-09-16T14:49:42.967 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.916e-06 2024-09-16T14:49:42.967 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.967 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.967 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:42.968 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900217+0000", 2024-09-16T14:49:42.969 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3189000000000006e-05 2024-09-16T14:49:42.969 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.969 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.970 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:42.970 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900222+0000", 2024-09-16T14:49:42.970 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7430000000000002e-06 2024-09-16T14:49:42.970 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.970 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.970 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:42.970 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902209+0000", 2024-09-16T14:49:42.970 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0019871440000000001 2024-09-16T14:49:42.971 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.971 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.971 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:42.971 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902254+0000", 2024-09-16T14:49:42.971 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4969000000000003e-05 2024-09-16T14:49:42.971 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.971 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.971 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:42.971 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918411+0000", 2024-09-16T14:49:42.971 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.016157207999999999 2024-09-16T14:49:42.972 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.972 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.972 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:42.972 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918534+0000", 2024-09-16T14:49:42.972 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000123146 2024-09-16T14:49:42.972 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.972 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.972 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:42.973 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918686+0000", 2024-09-16T14:49:42.973 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015134799999999999 2024-09-16T14:49:42.973 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:42.973 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:42.973 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:42.973 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.973 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.973 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:767 3.as0 3:572aff57:::smithi04040151-20:head [copy-from ver 63] snapc 31=[2e,28] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:49:42.973 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:29.077215+0000", 2024-09-16T14:49:42.974 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.872260476999999, 2024-09-16T14:49:42.974 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.021272019999999999, 2024-09-16T14:49:42.974 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:42.974 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:42.974 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:42.974 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:42.974 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.974 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 767 2024-09-16T14:49:42.974 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.974 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:42.975 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.975 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:42.975 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077215+0000", 2024-09-16T14:49:42.975 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.975 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.975 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.975 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:42.975 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077215+0000", 2024-09-16T14:49:42.975 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.976 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.976 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.976 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:42.976 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077213+0000", 2024-09-16T14:49:42.976 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:42.976 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.976 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.976 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:42.976 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077223+0000", 2024-09-16T14:49:42.976 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9890000000000004e-06 2024-09-16T14:49:42.976 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.976 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.976 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:42.977 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077242+0000", 2024-09-16T14:49:42.977 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8286000000000001e-05 2024-09-16T14:49:42.977 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.977 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.977 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.977 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077253+0000", 2024-09-16T14:49:42.977 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1807999999999999e-05 2024-09-16T14:49:42.977 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.977 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.977 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:42.977 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077321+0000", 2024-09-16T14:49:42.977 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7131999999999994e-05 2024-09-16T14:49:42.977 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.978 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.978 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:42.978 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077431+0000", 2024-09-16T14:49:42.978 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0001102 2024-09-16T14:49:42.978 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.978 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.978 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.978 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.090085+0000", 2024-09-16T14:49:42.978 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012654349 2024-09-16T14:49:42.978 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.978 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.978 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.978 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092962+0000", 2024-09-16T14:49:42.979 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002877133 2024-09-16T14:49:42.979 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.979 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.979 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:42.979 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.098442+0000", 2024-09-16T14:49:42.979 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0054794149999999996 2024-09-16T14:49:42.979 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.979 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.979 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:42.979 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.098487+0000", 2024-09-16T14:49:42.979 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5587000000000002e-05 2024-09-16T14:49:42.979 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:42.979 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:42.980 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:42.980 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.980 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.980 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:768 3.as0 3:572aff57:::smithi04040151-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:49:42.980 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:42.980 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.872209336999999, 2024-09-16T14:49:42.980 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015338218000000001, 2024-09-16T14:49:42.980 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:42.980 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:42.980 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:42.980 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:42.980 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:42.980 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 768 2024-09-16T14:49:42.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.981 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:42.981 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.981 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:42.981 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:42.981 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.981 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.981 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:42.981 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:42.981 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.981 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.982 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:42.982 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:42.982 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.982 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.982 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.982 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:42.982 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077268+0000", 2024-09-16T14:49:42.982 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.266e-06 2024-09-16T14:49:42.982 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.982 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.982 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:42.982 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077275+0000", 2024-09-16T14:49:42.982 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6200000000000001e-06 2024-09-16T14:49:42.982 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.983 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.983 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.983 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077278+0000", 2024-09-16T14:49:42.983 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.055e-06 2024-09-16T14:49:42.983 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.983 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.983 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:42.983 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077612+0000", 2024-09-16T14:49:42.983 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000334321 2024-09-16T14:49:42.983 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.983 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.983 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077660+0000", 2024-09-16T14:49:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8161999999999997e-05 2024-09-16T14:49:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092440+0000", 2024-09-16T14:49:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014779825999999999 2024-09-16T14:49:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092530+0000", 2024-09-16T14:49:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9232000000000002e-05 2024-09-16T14:49:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.984 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.985 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:42.985 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092605+0000", 2024-09-16T14:49:42.985 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5086999999999994e-05 2024-09-16T14:49:42.985 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:42.985 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:42.985 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:42.985 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.985 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.985 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds3 86/65 ECSubWrite(tid=302, reqid=client.4500.0:792, at_version=86'71, trim_to=0'0, roll_forward_to=83'69))", 2024-09-16T14:49:42.985 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:32.881497+0000", 2024-09-16T14:49:42.985 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 10.067979046, 2024-09-16T14:49:42.985 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0055907470000000001, 2024-09-16T14:49:42.985 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:42.986 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.986 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:42.986 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.986 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:42.986 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881497+0000", 2024-09-16T14:49:42.986 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.986 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.986 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.986 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:42.986 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881497+0000", 2024-09-16T14:49:42.986 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.986 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.986 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.987 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:42.987 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881496+0000", 2024-09-16T14:49:42.987 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.987 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.987 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.987 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:42.987 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881544+0000", 2024-09-16T14:49:42.987 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7973000000000003e-05 2024-09-16T14:49:42.987 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.987 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.987 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:42.987 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881555+0000", 2024-09-16T14:49:42.987 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1405e-05 2024-09-16T14:49:42.988 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.988 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.988 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.988 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881564+0000", 2024-09-16T14:49:42.988 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7329999999999992e-06 2024-09-16T14:49:42.988 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.988 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.988 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:42.988 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881635+0000", 2024-09-16T14:49:42.988 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.1067000000000002e-05 2024-09-16T14:49:42.988 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.988 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.989 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.989 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881647+0000", 2024-09-16T14:49:42.989 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1239e-05 2024-09-16T14:49:42.989 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.989 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.989 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.989 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.887060+0000", 2024-09-16T14:49:42.989 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005413691 2024-09-16T14:49:42.989 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.989 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.989 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:42.989 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.887087+0000", 2024-09-16T14:49:42.989 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7234e-05 2024-09-16T14:49:42.990 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:42.990 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:42.990 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:42.990 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.990 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.990 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.cs2 90/65 ECSubWrite(tid=189, reqid=osd.5.0:189, at_version=90'73, trim_to=0'0, roll_forward_to=89'71))", 2024-09-16T14:49:42.990 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.947178+0000", 2024-09-16T14:49:42.990 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 6.0022977429999997, 2024-09-16T14:49:42.990 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014311760999999999, 2024-09-16T14:49:42.990 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:42.990 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.990 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:42.990 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.990 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:42.991 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947178+0000", 2024-09-16T14:49:42.991 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.991 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.991 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.991 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:42.991 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947178+0000", 2024-09-16T14:49:42.991 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.991 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.991 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.991 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:42.991 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947177+0000", 2024-09-16T14:49:42.991 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.991 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.991 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.992 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:42.992 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947210+0000", 2024-09-16T14:49:42.992 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2759999999999998e-05 2024-09-16T14:49:42.992 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.992 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.992 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:42.992 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947224+0000", 2024-09-16T14:49:42.992 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3577000000000001e-05 2024-09-16T14:49:42.992 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.992 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.992 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.992 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947233+0000", 2024-09-16T14:49:42.993 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8169999999999994e-06 2024-09-16T14:49:42.993 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.993 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.993 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:42.993 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947304+0000", 2024-09-16T14:49:42.993 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0809000000000006e-05 2024-09-16T14:49:42.993 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.993 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.993 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.993 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947317+0000", 2024-09-16T14:49:42.993 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.292e-05 2024-09-16T14:49:42.993 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.994 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.994 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:42.994 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.961464+0000", 2024-09-16T14:49:42.994 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014146532 2024-09-16T14:49:42.994 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.994 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.994 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:42.994 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.961490+0000", 2024-09-16T14:49:42.994 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6012000000000001e-05 2024-09-16T14:49:42.994 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:42.994 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:42.995 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:42.995 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.995 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.995 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs7 90/65 ECSubWrite(tid=379, reqid=osd.3.0:379, at_version=90'75, trim_to=0'0, roll_forward_to=83'73))", 2024-09-16T14:49:42.995 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.947459+0000", 2024-09-16T14:49:42.995 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 6.0020171949999996, 2024-09-16T14:49:42.995 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0096387420000000005, 2024-09-16T14:49:42.995 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:42.995 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:42.995 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:42.995 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.996 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:42.996 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947459+0000", 2024-09-16T14:49:42.996 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.996 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.996 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.996 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:42.996 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947459+0000", 2024-09-16T14:49:42.996 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:42.996 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.996 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.996 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:42.996 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947458+0000", 2024-09-16T14:49:42.996 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:42.997 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.997 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.997 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:42.997 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947486+0000", 2024-09-16T14:49:42.997 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8123000000000002e-05 2024-09-16T14:49:42.997 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.997 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.997 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:42.997 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947496+0000", 2024-09-16T14:49:42.998 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7650000000000005e-06 2024-09-16T14:49:42.998 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.998 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.998 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:42.998 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947504+0000", 2024-09-16T14:49:42.998 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9719999999999992e-06 2024-09-16T14:49:42.998 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.998 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.998 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:42.998 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947574+0000", 2024-09-16T14:49:42.998 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0259999999999995e-05 2024-09-16T14:49:42.999 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.999 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.999 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:42.999 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947584+0000", 2024-09-16T14:49:42.999 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7440000000000002e-06 2024-09-16T14:49:42.999 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:42.999 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:42.999 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:43.000 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957064+0000", 2024-09-16T14:49:43.000 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0094797530000000005 2024-09-16T14:49:43.000 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.000 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.000 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:43.000 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957097+0000", 2024-09-16T14:49:43.000 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3615999999999998e-05 2024-09-16T14:49:43.000 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.000 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:43.000 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.001 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.001 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.001 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs7 90/65 ECSubWrite(tid=380, reqid=osd.3.0:380, at_version=90'77, trim_to=0'0, roll_forward_to=83'73))", 2024-09-16T14:49:43.001 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.948118+0000", 2024-09-16T14:49:43.001 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 6.0013573490000001, 2024-09-16T14:49:43.001 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014633293, 2024-09-16T14:49:43.001 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:43.001 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:43.001 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:43.001 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.002 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:43.002 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948118+0000", 2024-09-16T14:49:43.002 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.002 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.002 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.002 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:43.002 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948118+0000", 2024-09-16T14:49:43.002 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.002 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.003 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.003 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:43.003 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948118+0000", 2024-09-16T14:49:43.003 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:43.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.003 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.003 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:43.003 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948141+0000", 2024-09-16T14:49:43.003 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3300999999999999e-05 2024-09-16T14:49:43.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.004 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.004 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:43.004 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948158+0000", 2024-09-16T14:49:43.004 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6999000000000001e-05 2024-09-16T14:49:43.004 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.004 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.004 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:43.004 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948166+0000", 2024-09-16T14:49:43.004 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.551e-06 2024-09-16T14:49:43.004 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.004 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.004 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:43.004 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948701+0000", 2024-09-16T14:49:43.005 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00053460599999999997 2024-09-16T14:49:43.005 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.005 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.005 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:43.005 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948711+0000", 2024-09-16T14:49:43.005 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6889999999999999e-06 2024-09-16T14:49:43.005 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.005 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.005 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:43.005 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.962726+0000", 2024-09-16T14:49:43.005 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014015272 2024-09-16T14:49:43.005 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.006 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.006 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:43.006 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.962752+0000", 2024-09-16T14:49:43.006 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5701999999999999e-05 2024-09-16T14:49:43.006 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.006 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:43.006 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.006 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.006 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.cs2 90/65 ECSubWrite(tid=190, reqid=osd.5.0:190, at_version=90'75, trim_to=0'0, roll_forward_to=89'71))", 2024-09-16T14:49:43.006 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.957165+0000", 2024-09-16T14:49:43.006 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.9923112139999999, 2024-09-16T14:49:43.007 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0094526560000000003, 2024-09-16T14:49:43.007 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:43.007 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:43.007 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:43.007 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.007 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:43.007 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957165+0000", 2024-09-16T14:49:43.007 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.008 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.008 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.008 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:43.008 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957165+0000", 2024-09-16T14:49:43.008 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.008 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.008 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.008 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:43.008 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957164+0000", 2024-09-16T14:49:43.009 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:43.009 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.009 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.009 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:43.009 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957190+0000", 2024-09-16T14:49:43.009 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6451000000000001e-05 2024-09-16T14:49:43.009 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.009 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.009 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:43.010 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957203+0000", 2024-09-16T14:49:43.010 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.237e-05 2024-09-16T14:49:43.010 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.010 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.010 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:43.010 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957211+0000", 2024-09-16T14:49:43.010 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8789999999999992e-06 2024-09-16T14:49:43.010 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.010 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.010 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:43.011 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.961640+0000", 2024-09-16T14:49:43.011 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044288629999999999 2024-09-16T14:49:43.011 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.011 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.011 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:43.011 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.961649+0000", 2024-09-16T14:49:43.011 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9660000000000002e-06 2024-09-16T14:49:43.011 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.011 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.011 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:43.011 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.966600+0000", 2024-09-16T14:49:43.012 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049509059999999997 2024-09-16T14:49:43.012 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.012 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.012 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:43.012 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.966617+0000", 2024-09-16T14:49:43.012 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7010000000000001e-05 2024-09-16T14:49:43.012 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.012 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:43.012 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.012 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.012 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.012 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds3 90/65 ECSubWrite(tid=311, reqid=osd.7.0:311, at_version=90'75, trim_to=0'0, roll_forward_to=89'73))", 2024-09-16T14:49:43.012 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.958625+0000", 2024-09-16T14:49:43.013 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 5.9908504789999997, 2024-09-16T14:49:43.013 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0061203280000000004, 2024-09-16T14:49:43.013 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:43.013 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:43.013 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:43.013 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.013 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:43.013 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958625+0000", 2024-09-16T14:49:43.013 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.013 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.013 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.013 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:43.013 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958625+0000", 2024-09-16T14:49:43.013 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958625+0000", 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958647+0000", 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2073999999999999e-05 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958660+0000", 2024-09-16T14:49:43.014 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2831e-05 2024-09-16T14:49:43.015 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.015 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.015 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:43.015 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958673+0000", 2024-09-16T14:49:43.015 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3169999999999999e-05 2024-09-16T14:49:43.015 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.015 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.015 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:43.015 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958738+0000", 2024-09-16T14:49:43.015 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.4768000000000001e-05 2024-09-16T14:49:43.015 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.015 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.015 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:43.016 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958754+0000", 2024-09-16T14:49:43.016 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6388999999999998e-05 2024-09-16T14:49:43.016 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.016 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.016 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:43.016 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.964722+0000", 2024-09-16T14:49:43.016 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0059672579999999996 2024-09-16T14:49:43.016 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.016 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.016 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:43.016 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.964746+0000", 2024-09-16T14:49:43.016 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4070000000000002e-05 2024-09-16T14:49:43.016 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.016 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:43.017 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.017 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.017 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.017 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.13s3 91/90 ECSubWrite(tid=54, reqid=osd.4.0:54, at_version=91'39, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:43.017 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:37.982933+0000", 2024-09-16T14:49:43.017 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 4.9665426579999998, 2024-09-16T14:49:43.017 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0063814340000000001, 2024-09-16T14:49:43.017 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:43.017 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:43.017 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:43.017 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.017 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:43.017 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.982933+0000", 2024-09-16T14:49:43.018 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.018 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.018 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.018 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:43.018 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.982933+0000", 2024-09-16T14:49:43.018 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.018 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.018 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.018 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:43.018 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.982932+0000", 2024-09-16T14:49:43.018 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:43.018 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.018 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.018 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:43.019 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.982958+0000", 2024-09-16T14:49:43.019 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5984999999999999e-05 2024-09-16T14:49:43.019 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.019 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.019 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:43.019 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.982972+0000", 2024-09-16T14:49:43.019 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3397000000000001e-05 2024-09-16T14:49:43.019 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.019 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.019 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:43.019 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.982988+0000", 2024-09-16T14:49:43.019 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5905000000000001e-05 2024-09-16T14:49:43.019 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.019 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.020 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:43.020 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.983040+0000", 2024-09-16T14:49:43.020 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2039000000000001e-05 2024-09-16T14:49:43.020 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.020 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.020 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:43.020 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.983050+0000", 2024-09-16T14:49:43.020 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0132999999999999e-05 2024-09-16T14:49:43.020 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.020 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.020 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:43.020 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.989291+0000", 2024-09-16T14:49:43.020 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0062410520000000004 2024-09-16T14:49:43.020 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.989315+0000", 2024-09-16T14:49:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3567e-05 2024-09-16T14:49:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:870 3.as0 3:53e83d93:::smithi04040151-22:head [append 0~622592] snapc 40=[3b] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.990043+0000", 2024-09-16T14:49:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.95943246400000004, 2024-09-16T14:49:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0057099860000000002, 2024-09-16T14:49:43.021 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:43.022 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:43.022 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:43.022 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:43.022 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:43.022 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 870 2024-09-16T14:49:43.022 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.022 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:43.022 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.022 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:43.022 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990043+0000", 2024-09-16T14:49:43.022 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.022 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.023 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.023 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:43.023 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990043+0000", 2024-09-16T14:49:43.023 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.023 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.023 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.023 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:43.023 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990042+0000", 2024-09-16T14:49:43.023 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:43.023 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.023 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.023 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:43.023 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990616+0000", 2024-09-16T14:49:43.023 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00057397299999999997 2024-09-16T14:49:43.024 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.024 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.024 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:43.024 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990626+0000", 2024-09-16T14:49:43.024 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0217000000000001e-05 2024-09-16T14:49:43.024 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.024 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.024 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:43.024 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990633+0000", 2024-09-16T14:49:43.024 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.1740000000000003e-06 2024-09-16T14:49:43.024 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.024 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.024 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:43.024 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990867+0000", 2024-09-16T14:49:43.025 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00023375300000000001 2024-09-16T14:49:43.025 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.025 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.025 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:43.025 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990973+0000", 2024-09-16T14:49:43.025 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010624 2024-09-16T14:49:43.025 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.025 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.025 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:43.025 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.991742+0000", 2024-09-16T14:49:43.025 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000768701 2024-09-16T14:49:43.025 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.025 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.026 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:43.026 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.994341+0000", 2024-09-16T14:49:43.026 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025990919999999999 2024-09-16T14:49:43.026 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.026 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.026 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:43.026 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.995696+0000", 2024-09-16T14:49:43.026 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001355532 2024-09-16T14:49:43.026 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.026 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.026 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:43.026 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.995753+0000", 2024-09-16T14:49:43.026 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.6799000000000003e-05 2024-09-16T14:49:43.026 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:871 3.as0 3:53e83d93:::smithi04040151-22:head [append 0~311296] snapc 40=[3b] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.991954+0000", 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.95752126999999998, 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0064516110000000003, 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 871 2024-09-16T14:49:43.027 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.028 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:43.028 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.028 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:43.028 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.991954+0000", 2024-09-16T14:49:43.028 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.028 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.028 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.028 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:43.028 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.991954+0000", 2024-09-16T14:49:43.028 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.028 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.028 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.028 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:43.029 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.991954+0000", 2024-09-16T14:49:43.029 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:43.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.029 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.029 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:43.029 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992242+0000", 2024-09-16T14:49:43.029 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00028799899999999999 2024-09-16T14:49:43.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.029 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.029 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:43.029 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992254+0000", 2024-09-16T14:49:43.029 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1741e-05 2024-09-16T14:49:43.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992262+0000", 2024-09-16T14:49:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6449999999999994e-06 2024-09-16T14:49:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992458+0000", 2024-09-16T14:49:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000195737 2024-09-16T14:49:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992566+0000", 2024-09-16T14:49:43.030 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010817199999999999 2024-09-16T14:49:43.031 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.031 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.031 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:43.031 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.993258+0000", 2024-09-16T14:49:43.031 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000691744 2024-09-16T14:49:43.031 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.031 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.031 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:43.031 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.994449+0000", 2024-09-16T14:49:43.031 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00119136 2024-09-16T14:49:43.031 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.031 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.031 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:43.032 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.998368+0000", 2024-09-16T14:49:43.032 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0039188540000000003 2024-09-16T14:49:43.032 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.032 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.032 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:43.032 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.998406+0000", 2024-09-16T14:49:43.032 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.7953000000000001e-05 2024-09-16T14:49:43.032 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.032 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:43.032 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.032 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.032 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.032 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:872 3.as0 3:53e83d93:::smithi04040151-22:head [setxattr _header (57) in=7b] snapc 40=[3b] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:43.032 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.992270+0000", 2024-09-16T14:49:43.033 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.95720545899999998, 2024-09-16T14:49:43.033 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0062624170000000002, 2024-09-16T14:49:43.033 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:43.033 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:43.033 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:43.033 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:43.033 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:43.033 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 872 2024-09-16T14:49:43.033 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.033 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:43.034 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.034 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:43.034 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992270+0000", 2024-09-16T14:49:43.034 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.034 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.034 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.034 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:43.034 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992270+0000", 2024-09-16T14:49:43.034 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.035 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.035 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.035 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:43.035 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992270+0000", 2024-09-16T14:49:43.035 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:43.035 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.035 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.035 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:43.035 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992273+0000", 2024-09-16T14:49:43.035 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.72e-06 2024-09-16T14:49:43.035 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.036 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.036 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:43.036 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992281+0000", 2024-09-16T14:49:43.036 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2910000000000002e-06 2024-09-16T14:49:43.036 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.036 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.036 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:43.036 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992286+0000", 2024-09-16T14:49:43.036 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5820000000000001e-06 2024-09-16T14:49:43.037 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.037 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.037 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:43.037 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.993679+0000", 2024-09-16T14:49:43.037 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001392627 2024-09-16T14:49:43.037 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.037 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.037 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:43.037 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.993772+0000", 2024-09-16T14:49:43.037 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2819999999999996e-05 2024-09-16T14:49:43.038 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.038 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.038 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:43.038 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.994153+0000", 2024-09-16T14:49:43.038 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000380949 2024-09-16T14:49:43.038 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.038 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.038 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:43.038 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.995398+0000", 2024-09-16T14:49:43.038 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001245803 2024-09-16T14:49:43.039 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.039 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.039 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:43.039 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.998475+0000", 2024-09-16T14:49:43.039 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0030767239999999999 2024-09-16T14:49:43.039 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.039 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.039 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:43.039 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.998533+0000", 2024-09-16T14:49:43.039 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.7595e-05 2024-09-16T14:49:43.040 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.040 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:43.040 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.040 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.040 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.040 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:873 3.as0 3:53e83d93:::smithi04040151-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:43.040 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.992307+0000", 2024-09-16T14:49:43.040 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.957168982, 2024-09-16T14:49:43.040 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0074946509999999997, 2024-09-16T14:49:43.040 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:43.041 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:43.041 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:43.041 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:43.041 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:43.041 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 873 2024-09-16T14:49:43.041 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.041 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:43.041 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.041 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:43.042 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992307+0000", 2024-09-16T14:49:43.042 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.042 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.042 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.042 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:43.042 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992307+0000", 2024-09-16T14:49:43.042 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.042 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.042 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.042 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:43.042 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992306+0000", 2024-09-16T14:49:43.043 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:43.043 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.043 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.043 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:43.043 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992309+0000", 2024-09-16T14:49:43.043 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2059999999999999e-06 2024-09-16T14:49:43.043 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.043 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.043 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:43.044 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992318+0000", 2024-09-16T14:49:43.044 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0189999999999995e-06 2024-09-16T14:49:43.044 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.044 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.044 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:43.044 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992326+0000", 2024-09-16T14:49:43.044 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0550000000000002e-06 2024-09-16T14:49:43.044 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.045 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.045 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:43.045 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.994733+0000", 2024-09-16T14:49:43.045 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002407345 2024-09-16T14:49:43.045 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.045 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.045 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:43.045 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.994843+0000", 2024-09-16T14:49:43.045 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000110263 2024-09-16T14:49:43.045 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.045 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.045 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:43.046 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.998807+0000", 2024-09-16T14:49:43.046 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0039642460000000003 2024-09-16T14:49:43.046 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.046 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.046 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:43.046 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.998890+0000", 2024-09-16T14:49:43.046 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2894999999999998e-05 2024-09-16T14:49:43.046 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.046 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.046 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:43.046 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.999801+0000", 2024-09-16T14:49:43.046 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00091116000000000005 2024-09-16T14:49:43.046 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.047 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:43.047 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.047 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.047 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.047 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.17s5 96/95 ECSubWrite(tid=105, reqid=osd.0.0:105, at_version=96'152, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:43.047 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:42.873248+0000", 2024-09-16T14:49:43.047 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.076227551000000004, 2024-09-16T14:49:43.047 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0071842269999999996, 2024-09-16T14:49:43.047 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:43.047 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:43.047 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:43.048 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.048 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:43.048 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873248+0000", 2024-09-16T14:49:43.048 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.048 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.048 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:43.048 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873248+0000", 2024-09-16T14:49:43.048 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.048 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.048 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:43.048 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873248+0000", 2024-09-16T14:49:43.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:43.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.049 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.049 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:43.049 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873268+0000", 2024-09-16T14:49:43.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0279e-05 2024-09-16T14:49:43.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.049 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.049 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:43.049 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873281+0000", 2024-09-16T14:49:43.049 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3149999999999999e-05 2024-09-16T14:49:43.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.049 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.050 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:43.050 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873290+0000", 2024-09-16T14:49:43.050 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8699999999999998e-06 2024-09-16T14:49:43.050 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.050 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.050 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:43.050 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873321+0000", 2024-09-16T14:49:43.050 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1430999999999998e-05 2024-09-16T14:49:43.051 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.052 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.052 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:43.052 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873329+0000", 2024-09-16T14:49:43.052 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7700000000000001e-06 2024-09-16T14:49:43.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.052 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.052 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:43.052 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.880401+0000", 2024-09-16T14:49:43.052 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0070722299999999997 2024-09-16T14:49:43.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.052 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.052 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:43.053 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.880432+0000", 2024-09-16T14:49:43.053 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1173000000000001e-05 2024-09-16T14:49:43.053 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.053 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:43.053 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.053 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.053 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.053 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.12s6 96/95 ECSubWrite(tid=106, reqid=osd.0.0:106, at_version=96'106, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:43.053 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:42.874711+0000", 2024-09-16T14:49:43.053 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.074764448999999997, 2024-09-16T14:49:43.053 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089858890000000004, 2024-09-16T14:49:43.053 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:43.053 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:43.054 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:43.054 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.054 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:43.054 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874711+0000", 2024-09-16T14:49:43.054 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.054 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.054 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:43.054 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874711+0000", 2024-09-16T14:49:43.054 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.054 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.054 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:43.055 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874711+0000", 2024-09-16T14:49:43.055 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:43.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.055 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.055 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:43.055 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874731+0000", 2024-09-16T14:49:43.055 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0307000000000001e-05 2024-09-16T14:49:43.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.055 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.055 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:43.055 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874746+0000", 2024-09-16T14:49:43.055 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4701e-05 2024-09-16T14:49:43.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.056 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.056 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:43.056 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874754+0000", 2024-09-16T14:49:43.056 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.14e-06 2024-09-16T14:49:43.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.056 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.056 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:43.056 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874796+0000", 2024-09-16T14:49:43.056 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2651999999999998e-05 2024-09-16T14:49:43.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.056 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.056 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:43.056 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874805+0000", 2024-09-16T14:49:43.057 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7900000000000005e-06 2024-09-16T14:49:43.057 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.057 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.057 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:43.057 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.883673+0000", 2024-09-16T14:49:43.057 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0088682129999999998 2024-09-16T14:49:43.057 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.057 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.057 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:43.057 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.883697+0000", 2024-09-16T14:49:43.057 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3787000000000001e-05 2024-09-16T14:49:43.057 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.057 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:43.057 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.058 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.058 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.058 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.17s5 96/95 ECSubWrite(tid=107, reqid=osd.0.0:107, at_version=96'154, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:43.058 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:42.875143+0000", 2024-09-16T14:49:43.058 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.074332766999999994, 2024-09-16T14:49:43.058 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084637489999999996, 2024-09-16T14:49:43.058 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:43.058 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:43.058 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:43.058 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.058 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:43.058 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875143+0000", 2024-09-16T14:49:43.058 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875143+0000", 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875142+0000", 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875164+0000", 2024-09-16T14:49:43.059 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1545000000000001e-05 2024-09-16T14:49:43.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.060 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.060 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:43.060 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875173+0000", 2024-09-16T14:49:43.060 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6320000000000003e-06 2024-09-16T14:49:43.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.060 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.060 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:43.060 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875181+0000", 2024-09-16T14:49:43.060 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5830000000000001e-06 2024-09-16T14:49:43.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.060 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.060 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:43.061 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875241+0000", 2024-09-16T14:49:43.061 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.0374999999999997e-05 2024-09-16T14:49:43.061 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.061 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.061 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:43.061 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875250+0000", 2024-09-16T14:49:43.061 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6000000000000007e-06 2024-09-16T14:49:43.061 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.061 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.061 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:43.061 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.883584+0000", 2024-09-16T14:49:43.061 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083335600000000003 2024-09-16T14:49:43.061 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.061 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.062 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:43.062 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.883607+0000", 2024-09-16T14:49:43.062 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3198999999999999e-05 2024-09-16T14:49:43.062 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.062 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:43.062 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.062 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.062 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.17s5 96/95 ECSubWrite(tid=108, reqid=client.4500.0:866, at_version=96'156, trim_to=0'0, roll_forward_to=96'152))", 2024-09-16T14:49:43.062 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:42.894666+0000", 2024-09-16T14:49:43.062 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 0.054809783000000001, 2024-09-16T14:49:43.062 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0078174909999999993, 2024-09-16T14:49:43.062 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:43.062 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:43.063 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:43.063 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.063 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:43.063 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.894666+0000", 2024-09-16T14:49:43.063 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.063 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.063 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.063 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:43.063 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.894666+0000", 2024-09-16T14:49:43.063 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:43.063 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.063 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.063 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:43.063 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.894665+0000", 2024-09-16T14:49:43.064 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:43.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.064 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.064 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:43.064 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.899248+0000", 2024-09-16T14:49:43.064 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045826900000000004 2024-09-16T14:49:43.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.064 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.064 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:43.064 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.899267+0000", 2024-09-16T14:49:43.064 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9718999999999999e-05 2024-09-16T14:49:43.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.064 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.065 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:43.065 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.899275+0000", 2024-09-16T14:49:43.065 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9500000000000001e-06 2024-09-16T14:49:43.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.065 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.065 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:43.065 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.899315+0000", 2024-09-16T14:49:43.065 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9246999999999999e-05 2024-09-16T14:49:43.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.065 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.065 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:43.065 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.899323+0000", 2024-09-16T14:49:43.065 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2320000000000001e-06 2024-09-16T14:49:43.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.066 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.066 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:43.066 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.902464+0000", 2024-09-16T14:49:43.066 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031410779999999998 2024-09-16T14:49:43.066 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:43.066 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:43.066 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:43.066 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.902483+0000", 2024-09-16T14:49:43.066 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9491999999999999e-05 2024-09-16T14:49:43.067 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.067 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:43.067 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.067 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:43.067 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:43.067 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:43.068 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:43.066+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:43.069 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:49:43.169 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:43.168+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:43.177 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:43.177 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:43.177 INFO:teuthology.orchestra.run.smithi040.stdout: "num_ops": 0 2024-09-16T14:49:43.177 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:43.186 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:49:43.217 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:49:43.218 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:49:43.274 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:43.272+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:43.294 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:49:43.294 INFO:teuthology.orchestra.run.smithi040.stdout: "ops": [], 2024-09-16T14:49:43.294 INFO:teuthology.orchestra.run.smithi040.stdout: "complaint_time": 30, 2024-09-16T14:49:43.295 INFO:teuthology.orchestra.run.smithi040.stdout: "num_blocked_ops": 0 2024-09-16T14:49:43.295 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:49:43.303 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:49:43.355 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:43.355 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:43.371 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:43.370+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:43.372 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:43.372 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:43.388 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:43.388 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:43.406 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:43.406 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:43.422 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:43.422 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:43.439 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:43.439 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:43.453 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:43.453 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:43.456 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:43.456 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:43.462 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:49:43.473 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:43.473 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:43.473 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:43.471+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:43.489 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:43.489 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:43.506 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:43.506 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:43.523 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:43.523 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:43.539 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:43.540 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:43.572 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:43.572+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:43.580 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:43.580 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:43.589 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:49:43.674 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:43.673+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:43.736 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:43.736 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:43.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:49:43.774 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:43.774+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:43.851 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:43.851 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:43.860 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:49:43.863 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is set 2024-09-16T14:49:43.864 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T14:49:43.864 INFO:tasks.rados.rados.0.smithi040.stdout:356: finishing copy_from racing read to smithi04040151-31 2024-09-16T14:49:43.864 INFO:tasks.rados.rados.0.smithi040.stdout:356: finishing copy_from to smithi04040151-31 2024-09-16T14:49:43.864 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 31 v 52 (ObjNum 111 snap 25 seq_num 111) dirty exists 2024-09-16T14:49:43.864 INFO:tasks.rados.rados.0.smithi040.stdout:353: done (2 left) 2024-09-16T14:49:43.865 INFO:tasks.rados.rados.0.smithi040.stdout:356: done (1 left) 2024-09-16T14:49:43.865 INFO:tasks.rados.rados.0.smithi040.stdout:361: done (0 left) 2024-09-16T14:49:43.865 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 37 current snap is 34 2024-09-16T14:49:43.865 INFO:tasks.rados.rados.0.smithi040.stdout:362: seq_num 129 ranges {2162688=622592,2785280=655360,3440640=688128,4128768=212992} 2024-09-16T14:49:43.875 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:43.873+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:43.880 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T14:49:43.880 INFO:tasks.rados.rados.0.smithi040.stdout:362: writing smithi04040151-37 from 2162688 to 2785280 tid 1 2024-09-16T14:49:43.884 INFO:tasks.rados.rados.0.smithi040.stdout:362: writing smithi04040151-37 from 2785280 to 3440640 tid 2 2024-09-16T14:49:43.889 INFO:tasks.rados.rados.0.smithi040.stdout:362: writing smithi04040151-37 from 3440640 to 4128768 tid 3 2024-09-16T14:49:43.890 INFO:tasks.rados.rados.0.smithi040.stdout:362: writing smithi04040151-37 from 4128768 to 4341760 tid 4 2024-09-16T14:49:43.890 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 11 current snap is 34 2024-09-16T14:49:43.891 INFO:tasks.rados.rados.0.smithi040.stdout:363: seq_num 130 ranges {1916928=638976,2555904=671744,3227648=802816,4030464=81920} 2024-09-16T14:49:43.906 INFO:tasks.rados.rados.0.smithi040.stdout:363: writing smithi04040151-11 from 1916928 to 2555904 tid 1 2024-09-16T14:49:43.910 INFO:tasks.rados.rados.0.smithi040.stdout:363: writing smithi04040151-11 from 2555904 to 3227648 tid 2 2024-09-16T14:49:43.915 INFO:tasks.rados.rados.0.smithi040.stdout:363: writing smithi04040151-11 from 3227648 to 4030464 tid 3 2024-09-16T14:49:43.915 INFO:tasks.rados.rados.0.smithi040.stdout:363: writing smithi04040151-11 from 4030464 to 4112384 tid 4 2024-09-16T14:49:43.916 INFO:tasks.rados.rados.0.smithi040.stdout:362: finishing write tid 1 to smithi04040151-37 2024-09-16T14:49:43.916 INFO:tasks.rados.rados.0.smithi040.stdout:362: finishing write tid 2 to smithi04040151-37 2024-09-16T14:49:43.916 INFO:tasks.rados.rados.0.smithi040.stdout:362: finishing write tid 3 to smithi04040151-37 2024-09-16T14:49:43.916 INFO:tasks.rados.rados.0.smithi040.stdout:362: finishing write tid 4 to smithi04040151-37 2024-09-16T14:49:43.916 INFO:tasks.rados.rados.0.smithi040.stdout:362: finishing write tid 5 to smithi04040151-37 2024-09-16T14:49:43.916 INFO:tasks.rados.rados.0.smithi040.stdout:362: finishing write tid 6 to smithi04040151-37 2024-09-16T14:49:43.916 INFO:tasks.rados.rados.0.smithi040.stdout:362: oid 37 updating version 0 to 114 2024-09-16T14:49:43.916 INFO:tasks.rados.rados.0.smithi040.stdout:362: oid 37 version 114 is already newer than 112 2024-09-16T14:49:43.916 INFO:tasks.rados.rados.0.smithi040.stdout:362: oid 37 version 114 is already newer than 113 2024-09-16T14:49:43.916 INFO:tasks.rados.rados.0.smithi040.stdout:362: oid 37 version 114 is already newer than 111 2024-09-16T14:49:43.916 INFO:tasks.rados.rados.0.smithi040.stdout:362: oid 37 version 114 is already newer than 110 2024-09-16T14:49:43.916 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 37 v 114 (ObjNum 129 snap 34 seq_num 129) dirty exists 2024-09-16T14:49:43.916 INFO:tasks.rados.rados.0.smithi040.stdout:362: left oid 37 (ObjNum 129 snap 34 seq_num 129) 2024-09-16T14:49:43.917 INFO:tasks.rados.rados.0.smithi040.stdout:363: finishing write tid 1 to smithi04040151-11 2024-09-16T14:49:43.917 INFO:tasks.rados.rados.0.smithi040.stdout:362: done (1 left) 2024-09-16T14:49:43.917 INFO:tasks.rados.rados.0.smithi040.stdout:364: snap_remove snap 31 2024-09-16T14:49:43.976 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:43.974+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:43.985 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:43.985 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:43.995 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:49:44.077 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:44.076+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:44.145 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:44.146 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:44.154 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:49:44.178 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:44.176+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:44.260 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:44.261 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:44.269 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:49:44.279 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:44.277+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:44.376 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:44.376 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:44.379 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:44.377+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:44.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:49:44.480 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:44.478+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:44.530 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:44.531 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:44.539 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:49:44.581 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:44.581+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:44.684 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:44.681+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:44.686 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:44.686 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:44.695 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:49:44.783 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:44.782+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:44.810 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:44.810 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:44.819 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:49:44.865 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T14:49:44.866 INFO:tasks.rados.rados.0.smithi040.stdout:364: done (1 left) 2024-09-16T14:49:44.866 INFO:tasks.rados.rados.0.smithi040.stdout:365: setattr oid 1 current snap is 34 2024-09-16T14:49:44.866 INFO:tasks.rados.rados.0.smithi040.stdout:363: finishing write tid 2 to smithi04040151-11 2024-09-16T14:49:44.866 INFO:tasks.rados.rados.0.smithi040.stdout:363: finishing write tid 3 to smithi04040151-11 2024-09-16T14:49:44.866 INFO:tasks.rados.rados.0.smithi040.stdout:363: finishing write tid 4 to smithi04040151-11 2024-09-16T14:49:44.866 INFO:tasks.rados.rados.0.smithi040.stdout:363: finishing write tid 5 to smithi04040151-11 2024-09-16T14:49:44.867 INFO:tasks.rados.rados.0.smithi040.stdout:363: finishing write tid 6 to smithi04040151-11 2024-09-16T14:49:44.867 INFO:tasks.rados.rados.0.smithi040.stdout:363: oid 11 updating version 0 to 105 2024-09-16T14:49:44.867 INFO:tasks.rados.rados.0.smithi040.stdout:363: oid 11 updating version 105 to 107 2024-09-16T14:49:44.867 INFO:tasks.rados.rados.0.smithi040.stdout:363: oid 11 version 107 is already newer than 104 2024-09-16T14:49:44.867 INFO:tasks.rados.rados.0.smithi040.stdout:363: oid 11 updating version 107 to 108 2024-09-16T14:49:44.867 INFO:tasks.rados.rados.0.smithi040.stdout:363: oid 11 version 108 is already newer than 106 2024-09-16T14:49:44.867 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 11 v 108 (ObjNum 130 snap 34 seq_num 130) dirty exists 2024-09-16T14:49:44.867 INFO:tasks.rados.rados.0.smithi040.stdout:363: left oid 11 (ObjNum 130 snap 34 seq_num 130) 2024-09-16T14:49:44.867 INFO:tasks.rados.rados.0.smithi040.stdout:363: done (1 left) 2024-09-16T14:49:44.868 INFO:tasks.rados.rados.0.smithi040.stdout:366: snap_create 2024-09-16T14:49:44.879 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 1 v 82 (ObjNum 117 snap 27 seq_num 117) dirty exists 2024-09-16T14:49:44.884 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:44.884+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:44.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T14:49:44.933 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:44.933 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:44.941 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:49:44.985 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:44.982+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:45.085 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:45.085 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:45.086 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:45.083+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:45.093 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:49:45.187 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:45.186+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:45.195 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:45.195 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:45.204 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:49:45.288 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:45.287+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:45.347 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:45.347 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:45.355 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:49:45.388 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:45.386+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:45.490 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:45.489+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:45.497 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:45.497 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:45.506 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:49:45.551 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:49:45.596 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:45.594+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:45.609 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:45.609 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:45.617 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:49:45.676 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:45.676 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:45.691 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:45.689+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:45.692 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:45.692 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:45.708 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:45.708 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:45.718 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:45.718 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:45.724 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:45.727 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:49:45.741 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:45.741 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:45.758 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:45.758 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:45.775 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:45.775 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:45.791 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:45.791 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:45.792 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:45.790+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:45.808 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:45.809 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:45.825 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:45.825 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:45.831 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:45.831 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:45.831 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:45.831 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:45.831 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.831 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:49:45.831 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:45.831 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 83.689243762999993, 2024-09-16T14:49:45.831 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.223181782, 2024-09-16T14:49:45.831 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.832 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:45.832 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.832 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:45.832 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:45.832 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 1 2024-09-16T14:49:45.832 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.832 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.832 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.832 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.832 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:45.832 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.832 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.832 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.833 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.833 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:45.833 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.833 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.833 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.833 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139495+0000", 2024-09-16T14:49:45.833 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:45.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.833 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.833 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.833 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139504+0000", 2024-09-16T14:49:45.833 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6119999999999999e-06 2024-09-16T14:49:45.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.834 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.834 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.834 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139514+0000", 2024-09-16T14:49:45.834 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:49:45.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.834 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.834 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.834 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139533+0000", 2024-09-16T14:49:45.834 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9573e-05 2024-09-16T14:49:45.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.834 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.834 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.834 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139585+0000", 2024-09-16T14:49:45.834 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2170999999999999e-05 2024-09-16T14:49:45.835 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.835 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.835 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:45.835 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139594+0000", 2024-09-16T14:49:45.835 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.884e-06 2024-09-16T14:49:45.835 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.836 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.836 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.836 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359233+0000", 2024-09-16T14:49:45.836 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.21963890899999999 2024-09-16T14:49:45.836 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.836 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.836 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.836 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359437+0000", 2024-09-16T14:49:45.836 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00020374999999999999 2024-09-16T14:49:45.836 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.836 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.837 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:45.837 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359611+0000", 2024-09-16T14:49:45.837 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017445899999999999 2024-09-16T14:49:45.837 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.837 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.837 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:45.837 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.360506+0000", 2024-09-16T14:49:45.837 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00089459299999999995 2024-09-16T14:49:45.837 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.837 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.837 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:45.837 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362594+0000", 2024-09-16T14:49:45.838 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002087776 2024-09-16T14:49:45.838 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.838 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.838 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:45.838 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362636+0000", 2024-09-16T14:49:45.838 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1933e-05 2024-09-16T14:49:45.838 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.838 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.838 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.838 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362679+0000", 2024-09-16T14:49:45.838 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2877999999999998e-05 2024-09-16T14:49:45.838 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.839 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.839 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.839 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.839 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.839 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:535 3.fs0 3:ff6b916c:::smithi04040151-25:head [append 0~786432] snapc 1e=[1e,1c,1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:45.839 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:45.839 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 37.897378687, 2024-09-16T14:49:45.839 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013876462000000001, 2024-09-16T14:49:45.839 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.839 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:45.839 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.839 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.840 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.840 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 535 2024-09-16T14:49:45.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.840 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.840 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.840 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.840 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:45.840 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.840 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.841 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.841 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:45.841 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.841 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.841 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.841 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.841 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931360+0000", 2024-09-16T14:49:45.841 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:45.841 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.841 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.841 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.841 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933014+0000", 2024-09-16T14:49:45.842 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001653715 2024-09-16T14:49:45.842 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.842 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.842 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.842 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933028+0000", 2024-09-16T14:49:45.842 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4239999999999999e-05 2024-09-16T14:49:45.842 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.842 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.843 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.843 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933036+0000", 2024-09-16T14:49:45.843 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8019999999999992e-06 2024-09-16T14:49:45.843 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.843 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.843 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.843 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933084+0000", 2024-09-16T14:49:45.843 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7663000000000001e-05 2024-09-16T14:49:45.843 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.843 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.843 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.843 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933184+0000", 2024-09-16T14:49:45.844 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000100549 2024-09-16T14:49:45.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.844 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.844 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:45.844 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.934324+0000", 2024-09-16T14:49:45.844 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011399229999999999 2024-09-16T14:49:45.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.844 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.844 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:45.844 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941715+0000", 2024-09-16T14:49:45.844 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.007391023 2024-09-16T14:49:45.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.845 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.845 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:45.845 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.945187+0000", 2024-09-16T14:49:45.845 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034717709999999998 2024-09-16T14:49:45.845 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.845 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.845 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.845 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.945238+0000", 2024-09-16T14:49:45.845 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.1459999999999999e-05 2024-09-16T14:49:45.845 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.845 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.845 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.846 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.846 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.846 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:539 3.fs0 3:ff6b916c:::smithi04040151-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:45.846 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:45.846 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 37.890257155, 2024-09-16T14:49:45.846 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01551078, 2024-09-16T14:49:45.846 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.846 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:45.846 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.846 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.846 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.846 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 539 2024-09-16T14:49:45.847 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.847 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.847 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.847 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.847 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:45.847 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.847 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.847 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.847 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.847 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:45.847 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.847 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.848 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.848 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.848 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:45.848 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:45.848 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.848 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.848 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.848 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938486+0000", 2024-09-16T14:49:45.848 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7779999999999999e-06 2024-09-16T14:49:45.848 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.848 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.849 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.849 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938492+0000", 2024-09-16T14:49:45.849 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8120000000000002e-06 2024-09-16T14:49:45.849 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.849 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.849 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.849 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938495+0000", 2024-09-16T14:49:45.849 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.948e-06 2024-09-16T14:49:45.849 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.849 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.849 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.849 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942566+0000", 2024-09-16T14:49:45.850 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0040700839999999999 2024-09-16T14:49:45.850 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.850 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.850 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:45.850 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942659+0000", 2024-09-16T14:49:45.850 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3887000000000004e-05 2024-09-16T14:49:45.850 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.850 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.850 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.850 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.948875+0000", 2024-09-16T14:49:45.850 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0062154630000000001 2024-09-16T14:49:45.850 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.851 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.851 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.851 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.948968+0000", 2024-09-16T14:49:45.851 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2758999999999997e-05 2024-09-16T14:49:45.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.851 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.851 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.851 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.953994+0000", 2024-09-16T14:49:45.851 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050265580000000004 2024-09-16T14:49:45.851 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.851 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.852 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.852 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.852 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:600 3.ds0 3:bcba132a:::smithi04040151-1:head [append 0~507904] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:45.852 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.993737+0000", 2024-09-16T14:49:45.852 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 33.835003792999998, 2024-09-16T14:49:45.852 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.008095819, 2024-09-16T14:49:45.852 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.852 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:45.852 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.852 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.852 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.853 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 600 2024-09-16T14:49:45.853 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.853 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.853 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.853 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.853 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.993737+0000", 2024-09-16T14:49:45.853 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.853 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.853 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.853 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.853 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.993737+0000", 2024-09-16T14:49:45.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.854 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.854 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.993735+0000", 2024-09-16T14:49:45.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:45.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.854 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.854 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994263+0000", 2024-09-16T14:49:45.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00052718199999999995 2024-09-16T14:49:45.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.855 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.855 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994274+0000", 2024-09-16T14:49:45.855 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1535999999999999e-05 2024-09-16T14:49:45.855 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.855 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.855 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.855 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994279+0000", 2024-09-16T14:49:45.855 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0839999999999999e-06 2024-09-16T14:49:45.855 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.855 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.855 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994330+0000", 2024-09-16T14:49:45.856 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0467999999999999e-05 2024-09-16T14:49:45.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:45.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994332+0000", 2024-09-16T14:49:45.856 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:49:45.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980708+0000", 2024-09-16T14:49:45.856 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98637637199999995 2024-09-16T14:49:45.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.857 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:45.857 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.981437+0000", 2024-09-16T14:49:45.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00072860299999999998 2024-09-16T14:49:45.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.857 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.857 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.989211+0000", 2024-09-16T14:49:45.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077741490000000002 2024-09-16T14:49:45.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.858 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.858 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.989340+0000", 2024-09-16T14:49:45.858 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00012908900000000001 2024-09-16T14:49:45.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.858 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.858 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:45.858 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.990969+0000", 2024-09-16T14:49:45.858 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001628612 2024-09-16T14:49:45.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.858 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.858 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:45.858 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996293+0000", 2024-09-16T14:49:45.859 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053247440000000002 2024-09-16T14:49:45.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.859 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.859 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:45.859 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.001434+0000", 2024-09-16T14:49:45.859 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0051404349999999996 2024-09-16T14:49:45.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.859 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.859 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.859 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.001833+0000", 2024-09-16T14:49:45.860 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00039878500000000002 2024-09-16T14:49:45.860 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.860 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.860 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.860 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.860 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.860 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:601 3.ds0 3:bcba132a:::smithi04040151-1:head [append 0~606208] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:45.860 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.998248+0000", 2024-09-16T14:49:45.860 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 33.830492475, 2024-09-16T14:49:45.860 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0062125260000001, 2024-09-16T14:49:45.860 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.860 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:45.861 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.861 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.861 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.861 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 601 2024-09-16T14:49:45.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.861 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.861 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.861 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.861 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998248+0000", 2024-09-16T14:49:45.861 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.861 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998248+0000", 2024-09-16T14:49:45.862 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998246+0000", 2024-09-16T14:49:45.862 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:45.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998747+0000", 2024-09-16T14:49:45.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00050040399999999997 2024-09-16T14:49:45.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.863 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.863 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.863 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998755+0000", 2024-09-16T14:49:45.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3329999999999997e-06 2024-09-16T14:49:45.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.863 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.863 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.863 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998759+0000", 2024-09-16T14:49:45.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8380000000000004e-06 2024-09-16T14:49:45.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.864 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.864 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.864 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998846+0000", 2024-09-16T14:49:45.864 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6456e-05 2024-09-16T14:49:45.864 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.864 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.864 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:45.864 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998847+0000", 2024-09-16T14:49:45.864 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.635e-06 2024-09-16T14:49:45.864 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.864 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984175+0000", 2024-09-16T14:49:45.865 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98532833399999997 2024-09-16T14:49:45.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:45.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984375+0000", 2024-09-16T14:49:45.865 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00019923299999999999 2024-09-16T14:49:45.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992431+0000", 2024-09-16T14:49:45.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0080562129999999996 2024-09-16T14:49:45.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.866 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992552+0000", 2024-09-16T14:49:45.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000120986 2024-09-16T14:49:45.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.866 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:45.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.993913+0000", 2024-09-16T14:49:45.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001361206 2024-09-16T14:49:45.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.867 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:45.867 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996295+0000", 2024-09-16T14:49:45.867 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0023814629999999999 2024-09-16T14:49:45.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.867 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:45.867 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004101+0000", 2024-09-16T14:49:45.868 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0078063500000000001 2024-09-16T14:49:45.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.868 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.868 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.868 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004461+0000", 2024-09-16T14:49:45.868 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00035967799999999999 2024-09-16T14:49:45.868 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.868 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.868 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.869 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:602 3.ds0 3:bcba132a:::smithi04040151-1:head [append 0~557056] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:45.869 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:45.869 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 33.826680256000003, 2024-09-16T14:49:45.869 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0032171249999999, 2024-09-16T14:49:45.869 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.869 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:45.869 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.869 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.869 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.869 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 602 2024-09-16T14:49:45.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.870 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.870 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.870 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.870 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:45.870 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.870 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.870 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.870 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:45.870 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.871 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.871 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.871 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:45.871 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:45.871 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.871 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.871 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.871 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002547+0000", 2024-09-16T14:49:45.871 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00048700700000000002 2024-09-16T14:49:45.871 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.871 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.872 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.872 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002554+0000", 2024-09-16T14:49:45.872 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.435e-06 2024-09-16T14:49:45.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.872 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.872 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.872 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002558+0000", 2024-09-16T14:49:45.872 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.501e-06 2024-09-16T14:49:45.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.872 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.872 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.873 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002641+0000", 2024-09-16T14:49:45.873 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2874999999999995e-05 2024-09-16T14:49:45.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.873 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.873 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:45.873 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002642+0000", 2024-09-16T14:49:45.873 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6670000000000001e-06 2024-09-16T14:49:45.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.873 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.873 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.873 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984450+0000", 2024-09-16T14:49:45.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98180768500000004 2024-09-16T14:49:45.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.874 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:45.874 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984591+0000", 2024-09-16T14:49:45.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014090700000000001 2024-09-16T14:49:45.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.874 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.874 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996586+0000", 2024-09-16T14:49:45.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011994898 2024-09-16T14:49:45.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.875 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.875 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.875 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996695+0000", 2024-09-16T14:49:45.875 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000108962 2024-09-16T14:49:45.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.875 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.875 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:45.875 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.997804+0000", 2024-09-16T14:49:45.875 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011095090000000001 2024-09-16T14:49:45.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.875 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.876 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:45.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000509+0000", 2024-09-16T14:49:45.876 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0027042440000000002 2024-09-16T14:49:45.876 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.876 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:45.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004755+0000", 2024-09-16T14:49:45.876 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0042461269999999997 2024-09-16T14:49:45.876 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.876 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005277+0000", 2024-09-16T14:49:45.877 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00052277599999999997 2024-09-16T14:49:45.877 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.877 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.877 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.877 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.877 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.877 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:603 3.ds0 3:bcba132a:::smithi04040151-1:head [setxattr _header (56) in=7b] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:45.877 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:45.877 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 33.826160667000003, 2024-09-16T14:49:45.877 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.003299656, 2024-09-16T14:49:45.877 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.877 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:45.878 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.878 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.878 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.878 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 603 2024-09-16T14:49:45.878 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.878 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.878 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.878 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.878 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:45.878 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.878 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.879 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.879 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:45.879 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.879 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.879 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.879 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:45.879 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:45.879 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.879 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.880 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002582+0000", 2024-09-16T14:49:45.880 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.345e-06 2024-09-16T14:49:45.880 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.880 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.880 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.880 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002586+0000", 2024-09-16T14:49:45.880 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4710000000000001e-06 2024-09-16T14:49:45.880 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.880 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.880 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.880 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002589+0000", 2024-09-16T14:49:45.881 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.694e-06 2024-09-16T14:49:45.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.881 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.881 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.881 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002673+0000", 2024-09-16T14:49:45.881 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4349000000000002e-05 2024-09-16T14:49:45.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.881 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.881 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:45.881 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002674+0000", 2024-09-16T14:49:45.881 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9300000000000006e-07 2024-09-16T14:49:45.882 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.882 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.882 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.882 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984634+0000", 2024-09-16T14:49:45.882 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98195955099999999 2024-09-16T14:49:45.882 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.882 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.882 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:45.882 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984742+0000", 2024-09-16T14:49:45.882 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000108491 2024-09-16T14:49:45.882 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.882 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.883 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.998850+0000", 2024-09-16T14:49:45.883 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014108005 2024-09-16T14:49:45.883 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.883 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.999006+0000", 2024-09-16T14:49:45.883 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015594799999999999 2024-09-16T14:49:45.883 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:45.883 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.999686+0000", 2024-09-16T14:49:45.884 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00067984999999999996 2024-09-16T14:49:45.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.884 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.884 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:45.884 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000622+0000", 2024-09-16T14:49:45.884 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00093585200000000004 2024-09-16T14:49:45.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.884 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.884 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:45.884 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005500+0000", 2024-09-16T14:49:45.884 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048775049999999999 2024-09-16T14:49:45.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.885 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.885 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.885 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005880+0000", 2024-09-16T14:49:45.885 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00037999399999999998 2024-09-16T14:49:45.885 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.885 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.885 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.885 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.885 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.885 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:604 3.ds0 3:bcba132a:::smithi04040151-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:45.885 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:45.885 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 33.826149125000001, 2024-09-16T14:49:45.886 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.00580336, 2024-09-16T14:49:45.886 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.886 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:45.886 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.886 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.886 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.886 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 604 2024-09-16T14:49:45.886 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.886 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.886 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.886 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.886 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:45.887 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.887 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.887 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.887 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.887 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:45.887 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.887 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.887 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.887 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.887 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:45.887 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:45.887 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.888 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.888 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.888 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002593+0000", 2024-09-16T14:49:45.888 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0559999999999999e-06 2024-09-16T14:49:45.888 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.888 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.888 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.888 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002597+0000", 2024-09-16T14:49:45.888 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.7349999999999998e-06 2024-09-16T14:49:45.888 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.888 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002601+0000", 2024-09-16T14:49:45.889 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6519999999999999e-06 2024-09-16T14:49:45.889 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.889 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002698+0000", 2024-09-16T14:49:45.889 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7378000000000002e-05 2024-09-16T14:49:45.889 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.889 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:45.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002699+0000", 2024-09-16T14:49:45.890 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.5099999999999999e-07 2024-09-16T14:49:45.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.890 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.890 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.890 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984866+0000", 2024-09-16T14:49:45.890 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98216700499999998 2024-09-16T14:49:45.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.890 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.890 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:45.890 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984953+0000", 2024-09-16T14:49:45.890 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6847000000000004e-05 2024-09-16T14:49:45.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.891 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.891 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.891 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000753+0000", 2024-09-16T14:49:45.891 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015800455000000001 2024-09-16T14:49:45.891 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.891 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.891 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:45.891 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000883+0000", 2024-09-16T14:49:45.891 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00013054000000000001 2024-09-16T14:49:45.891 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.891 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.892 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006482+0000", 2024-09-16T14:49:45.892 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0055987839999999999 2024-09-16T14:49:45.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.892 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.892 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006630+0000", 2024-09-16T14:49:45.892 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014820600000000001 2024-09-16T14:49:45.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.892 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.892 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.008395+0000", 2024-09-16T14:49:45.893 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0017643139999999999 2024-09-16T14:49:45.893 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.893 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.893 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.893 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.893 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.893 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:751 3.ds0 3:bcba132a:::smithi04040151-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e80)", 2024-09-16T14:49:45.893 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:27.015103+0000", 2024-09-16T14:49:45.893 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.813637375999999, 2024-09-16T14:49:45.893 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013724121000000001, 2024-09-16T14:49:45.893 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.893 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:45.894 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.894 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.894 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.894 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 751 2024-09-16T14:49:45.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.894 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.894 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.894 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.894 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015103+0000", 2024-09-16T14:49:45.894 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.894 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.895 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.895 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015103+0000", 2024-09-16T14:49:45.895 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.895 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.895 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.895 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.895 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015103+0000", 2024-09-16T14:49:45.895 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:45.895 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.895 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.895 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.895 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015104+0000", 2024-09-16T14:49:45.896 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.584e-06 2024-09-16T14:49:45.896 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.896 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.896 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.896 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015108+0000", 2024-09-16T14:49:45.896 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9389999999999999e-06 2024-09-16T14:49:45.896 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.896 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.896 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.896 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015111+0000", 2024-09-16T14:49:45.896 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7889999999999999e-06 2024-09-16T14:49:45.897 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.897 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.897 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.897 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.017354+0000", 2024-09-16T14:49:45.897 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0022426849999999999 2024-09-16T14:49:45.897 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.897 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.897 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:45.897 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.017423+0000", 2024-09-16T14:49:45.897 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9228999999999997e-05 2024-09-16T14:49:45.897 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.898 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.898 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.898 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.022608+0000", 2024-09-16T14:49:45.898 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0051849920000000002 2024-09-16T14:49:45.898 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.898 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.898 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.898 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.022679+0000", 2024-09-16T14:49:45.898 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0444000000000005e-05 2024-09-16T14:49:45.898 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.898 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.898 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.899 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.028827+0000", 2024-09-16T14:49:45.899 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0061487649999999996 2024-09-16T14:49:45.899 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.899 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.899 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.899 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.899 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.899 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:756 3.fs0 3:f17be48c:::smithi04040151-41:2e [read 0~3588096,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T14:49:45.899 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:27.016084+0000", 2024-09-16T14:49:45.899 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.812656126, 2024-09-16T14:49:45.899 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012061232, 2024-09-16T14:49:45.899 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.900 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:45.900 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.900 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.900 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.900 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 756 2024-09-16T14:49:45.900 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.900 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.900 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.900 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.900 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016084+0000", 2024-09-16T14:49:45.900 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.901 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.901 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.901 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.901 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016084+0000", 2024-09-16T14:49:45.901 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.901 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.901 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.901 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.901 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016084+0000", 2024-09-16T14:49:45.901 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:45.901 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.901 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.902 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.902 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016090+0000", 2024-09-16T14:49:45.902 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7279999999999998e-06 2024-09-16T14:49:45.902 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.902 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.902 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.902 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016099+0000", 2024-09-16T14:49:45.902 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9649999999999997e-06 2024-09-16T14:49:45.902 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.902 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.902 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.902 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016106+0000", 2024-09-16T14:49:45.903 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3670000000000002e-06 2024-09-16T14:49:45.903 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.903 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.903 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.903 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.017376+0000", 2024-09-16T14:49:45.903 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0012697279999999999 2024-09-16T14:49:45.903 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.903 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.903 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.903 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.017477+0000", 2024-09-16T14:49:45.903 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010165700000000001 2024-09-16T14:49:45.903 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.904 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.904 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.904 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.028146+0000", 2024-09-16T14:49:45.904 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010668502999999999 2024-09-16T14:49:45.904 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.904 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.904 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.904 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.904 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.904 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:757 3.fs0 3:f17be48c:::smithi04040151-41:2e [read 0~3588096,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T14:49:45.904 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:27.016111+0000", 2024-09-16T14:49:45.905 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.812629483999999, 2024-09-16T14:49:45.905 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013568108000000001, 2024-09-16T14:49:45.905 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.905 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:45.905 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.905 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.905 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.905 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 757 2024-09-16T14:49:45.905 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.905 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.905 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.905 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.906 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016111+0000", 2024-09-16T14:49:45.906 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.906 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.906 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.906 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.906 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016111+0000", 2024-09-16T14:49:45.906 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.906 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.906 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.906 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.906 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016111+0000", 2024-09-16T14:49:45.906 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:45.907 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.907 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.907 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.907 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016115+0000", 2024-09-16T14:49:45.907 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:49:45.907 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.907 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.907 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.907 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016121+0000", 2024-09-16T14:49:45.907 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9909999999999999e-06 2024-09-16T14:49:45.907 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.908 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.908 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.908 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016125+0000", 2024-09-16T14:49:45.908 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6600000000000003e-06 2024-09-16T14:49:45.908 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.908 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.908 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.908 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.020236+0000", 2024-09-16T14:49:45.908 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0041107019999999999 2024-09-16T14:49:45.908 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.908 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.909 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.909 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.020334+0000", 2024-09-16T14:49:45.909 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8166999999999997e-05 2024-09-16T14:49:45.909 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.909 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.909 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.909 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.029679+0000", 2024-09-16T14:49:45.909 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0093450909999999998 2024-09-16T14:49:45.909 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.909 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.909 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.910 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.910 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.910 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.7s2 82/27 ECSubWrite(tid=342, reqid=osd.3.0:342, at_version=82'110, trim_to=0'0, roll_forward_to=81'108))", 2024-09-16T14:49:45.910 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.933809+0000", 2024-09-16T14:49:45.910 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.894931827000001, 2024-09-16T14:49:45.910 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013532219, 2024-09-16T14:49:45.910 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.910 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:45.910 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.910 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.910 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.910 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933809+0000", 2024-09-16T14:49:45.911 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.911 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.911 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.911 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.911 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933809+0000", 2024-09-16T14:49:45.911 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.911 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.911 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.911 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.911 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933808+0000", 2024-09-16T14:49:45.911 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:45.911 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.912 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.912 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.912 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933834+0000", 2024-09-16T14:49:45.912 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5735e-05 2024-09-16T14:49:45.912 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.912 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.912 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.912 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933844+0000", 2024-09-16T14:49:45.912 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0081999999999999e-05 2024-09-16T14:49:45.912 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.912 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.912 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.913 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933851+0000", 2024-09-16T14:49:45.913 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8070000000000002e-06 2024-09-16T14:49:45.913 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.913 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.913 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.913 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934134+0000", 2024-09-16T14:49:45.913 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00028271999999999998 2024-09-16T14:49:45.914 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.914 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.914 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:45.914 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934143+0000", 2024-09-16T14:49:45.914 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4120000000000007e-06 2024-09-16T14:49:45.914 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.914 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.914 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:45.914 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.947319+0000", 2024-09-16T14:49:45.914 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013176155 2024-09-16T14:49:45.914 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.915 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.915 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.915 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.947341+0000", 2024-09-16T14:49:45.915 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1603e-05 2024-09-16T14:49:45.915 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.915 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.915 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.915 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.915 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.915 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs5 82/65 ECSubWrite(tid=345, reqid=osd.3.0:345, at_version=82'69, trim_to=0'0, roll_forward_to=81'65))", 2024-09-16T14:49:45.915 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.933907+0000", 2024-09-16T14:49:45.915 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.894833985999998, 2024-09-16T14:49:45.916 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012550997, 2024-09-16T14:49:45.916 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.916 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:45.916 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.916 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.916 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.916 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933907+0000", 2024-09-16T14:49:45.916 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.916 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.917 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.917 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.917 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933907+0000", 2024-09-16T14:49:45.917 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.917 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.917 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.917 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.917 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933906+0000", 2024-09-16T14:49:45.917 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:45.917 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.917 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.918 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.918 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933922+0000", 2024-09-16T14:49:45.918 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5930000000000002e-05 2024-09-16T14:49:45.918 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.918 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.918 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.918 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933929+0000", 2024-09-16T14:49:45.918 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6730000000000003e-06 2024-09-16T14:49:45.918 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.918 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.918 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.918 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933934+0000", 2024-09-16T14:49:45.919 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:49:45.919 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.919 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.919 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.919 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.938626+0000", 2024-09-16T14:49:45.919 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0046912070000000002 2024-09-16T14:49:45.919 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.919 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.919 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:45.919 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.938635+0000", 2024-09-16T14:49:45.919 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3589999999999997e-06 2024-09-16T14:49:45.920 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.920 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.920 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:45.920 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.946337+0000", 2024-09-16T14:49:45.920 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077023760000000004 2024-09-16T14:49:45.920 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.920 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.920 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.920 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.946458+0000", 2024-09-16T14:49:45.920 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000120277 2024-09-16T14:49:45.920 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.921 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.921 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.921 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.921 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.921 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.7s2 82/27 ECSubWrite(tid=344, reqid=osd.3.0:344, at_version=82'112, trim_to=0'0, roll_forward_to=81'108))", 2024-09-16T14:49:45.921 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.933939+0000", 2024-09-16T14:49:45.921 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.894801413, 2024-09-16T14:49:45.921 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014103087, 2024-09-16T14:49:45.921 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.921 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:45.921 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.921 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.922 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.922 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933939+0000", 2024-09-16T14:49:45.922 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.922 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.922 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.922 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.922 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933939+0000", 2024-09-16T14:49:45.922 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.922 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.922 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.922 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.923 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933939+0000", 2024-09-16T14:49:45.923 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:45.923 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.923 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.923 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.923 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933954+0000", 2024-09-16T14:49:45.923 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.491e-05 2024-09-16T14:49:45.923 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.923 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.923 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.923 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933960+0000", 2024-09-16T14:49:45.923 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6109999999999999e-06 2024-09-16T14:49:45.924 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.924 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.924 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.924 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933965+0000", 2024-09-16T14:49:45.924 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9599999999999999e-06 2024-09-16T14:49:45.924 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.924 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.924 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.924 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.941466+0000", 2024-09-16T14:49:45.924 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0075007529999999998 2024-09-16T14:49:45.924 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.924 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.925 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:45.925 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.941478+0000", 2024-09-16T14:49:45.925 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1501999999999999e-05 2024-09-16T14:49:45.925 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.925 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.925 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:45.925 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.948023+0000", 2024-09-16T14:49:45.925 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0065449870000000004 2024-09-16T14:49:45.925 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.925 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.925 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.926 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.948042+0000", 2024-09-16T14:49:45.926 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9545999999999999e-05 2024-09-16T14:49:45.926 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.926 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.926 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.926 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.926 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.926 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:860 3.ds0 3:bd112605:::smithi04040151-44:3b [read 0~737280,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:45.926 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.862879+0000", 2024-09-16T14:49:45.926 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.965861426, 2024-09-16T14:49:45.926 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013219919, 2024-09-16T14:49:45.926 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.927 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:45.927 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.927 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.927 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.927 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 860 2024-09-16T14:49:45.927 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.927 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.927 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.927 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.927 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862879+0000", 2024-09-16T14:49:45.927 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.927 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.928 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.928 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.928 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862879+0000", 2024-09-16T14:49:45.928 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.928 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.928 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.928 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.928 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862877+0000", 2024-09-16T14:49:45.928 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:45.928 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.928 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.929 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.929 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862886+0000", 2024-09-16T14:49:45.929 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4190000000000002e-06 2024-09-16T14:49:45.929 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.929 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.929 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.929 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862898+0000", 2024-09-16T14:49:45.929 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1994000000000001e-05 2024-09-16T14:49:45.929 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.929 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.929 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.929 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862906+0000", 2024-09-16T14:49:45.930 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.1459999999999996e-06 2024-09-16T14:49:45.930 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.930 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.930 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.930 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862955+0000", 2024-09-16T14:49:45.930 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9147999999999998e-05 2024-09-16T14:49:45.930 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.930 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.930 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:45.930 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862965+0000", 2024-09-16T14:49:45.930 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5519999999999993e-06 2024-09-16T14:49:45.930 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.931 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.931 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.931 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866479+0000", 2024-09-16T14:49:45.931 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0035142139999999999 2024-09-16T14:49:45.931 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.931 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.931 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.931 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866601+0000", 2024-09-16T14:49:45.931 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00012230699999999999 2024-09-16T14:49:45.931 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.931 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.932 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.932 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.876099+0000", 2024-09-16T14:49:45.932 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0094979060000000004 2024-09-16T14:49:45.932 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.932 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.932 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.932 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.932 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.932 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:861 3.ds0 3:bd112605:::smithi04040151-44:3b [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:45.932 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.862920+0000", 2024-09-16T14:49:45.932 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.9658209599999998, 2024-09-16T14:49:45.932 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013865529, 2024-09-16T14:49:45.933 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.933 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:45.933 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.933 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.933 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.933 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 861 2024-09-16T14:49:45.933 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.933 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.933 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.933 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.933 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862920+0000", 2024-09-16T14:49:45.934 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.934 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.934 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.934 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.934 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862920+0000", 2024-09-16T14:49:45.934 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.934 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.934 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.934 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.934 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862919+0000", 2024-09-16T14:49:45.934 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:45.934 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.935 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.935 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.935 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862922+0000", 2024-09-16T14:49:45.935 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1489999999999998e-06 2024-09-16T14:49:45.935 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.935 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.935 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.935 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862926+0000", 2024-09-16T14:49:45.936 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0969999999999999e-06 2024-09-16T14:49:45.936 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.936 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.936 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.936 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862929+0000", 2024-09-16T14:49:45.936 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.061e-06 2024-09-16T14:49:45.936 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.936 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.936 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.936 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863026+0000", 2024-09-16T14:49:45.936 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6824999999999995e-05 2024-09-16T14:49:45.937 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.937 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.937 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:45.937 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863031+0000", 2024-09-16T14:49:45.937 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1919999999999998e-06 2024-09-16T14:49:45.937 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.937 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.937 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.937 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866928+0000", 2024-09-16T14:49:45.937 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0038978099999999998 2024-09-16T14:49:45.937 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.938 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.938 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.938 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867034+0000", 2024-09-16T14:49:45.938 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010561299999999999 2024-09-16T14:49:45.938 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.938 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.938 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.938 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.876785+0000", 2024-09-16T14:49:45.938 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0097512020000000005 2024-09-16T14:49:45.938 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.938 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.938 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.939 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.939 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.939 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:862 3.ds0 3:bd112605:::smithi04040151-44:3b [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:45.939 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.862941+0000", 2024-09-16T14:49:45.939 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.9657997009999999, 2024-09-16T14:49:45.939 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014724180999999999, 2024-09-16T14:49:45.939 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.939 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:45.939 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.939 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.939 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.939 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 862 2024-09-16T14:49:45.940 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.940 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.940 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.940 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.940 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862941+0000", 2024-09-16T14:49:45.940 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.940 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.940 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.940 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.940 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862941+0000", 2024-09-16T14:49:45.940 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.941 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.941 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.941 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.941 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862941+0000", 2024-09-16T14:49:45.941 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:45.941 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.941 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.941 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.941 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862943+0000", 2024-09-16T14:49:45.941 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3180000000000001e-06 2024-09-16T14:49:45.941 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.942 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.942 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.942 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862946+0000", 2024-09-16T14:49:45.942 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6160000000000002e-06 2024-09-16T14:49:45.942 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.942 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.942 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.942 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862949+0000", 2024-09-16T14:49:45.942 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6400000000000001e-06 2024-09-16T14:49:45.942 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.942 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.942 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.943 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863054+0000", 2024-09-16T14:49:45.943 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010529100000000001 2024-09-16T14:49:45.943 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.943 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.943 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:45.943 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863061+0000", 2024-09-16T14:49:45.943 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6590000000000001e-06 2024-09-16T14:49:45.943 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.943 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.943 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.943 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867329+0000", 2024-09-16T14:49:45.943 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0042681100000000003 2024-09-16T14:49:45.944 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.944 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.944 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.944 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867443+0000", 2024-09-16T14:49:45.944 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000113596 2024-09-16T14:49:45.944 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.944 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.944 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.944 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.877665+0000", 2024-09-16T14:49:45.944 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01022227 2024-09-16T14:49:45.944 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.945 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.945 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.945 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.945 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.945 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:863 3.13s0 3:c9d06351:::smithi04040151-33:3b [read 0~1654784,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:45.945 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.862960+0000", 2024-09-16T14:49:45.945 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.9657806309999999, 2024-09-16T14:49:45.945 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013471337999999999, 2024-09-16T14:49:45.945 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.945 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:45.945 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.945 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.946 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.946 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 863 2024-09-16T14:49:45.946 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.946 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.946 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.946 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.946 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862960+0000", 2024-09-16T14:49:45.946 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.946 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.946 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.947 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.947 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862960+0000", 2024-09-16T14:49:45.947 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.947 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.947 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.947 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.947 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862960+0000", 2024-09-16T14:49:45.947 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:45.947 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.947 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.947 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.947 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862962+0000", 2024-09-16T14:49:45.948 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4420000000000001e-06 2024-09-16T14:49:45.948 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.948 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.948 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.948 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862966+0000", 2024-09-16T14:49:45.948 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5810000000000001e-06 2024-09-16T14:49:45.948 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.948 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.948 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.948 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862968+0000", 2024-09-16T14:49:45.948 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7369999999999998e-06 2024-09-16T14:49:45.949 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.949 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.949 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.949 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862999+0000", 2024-09-16T14:49:45.949 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1031000000000002e-05 2024-09-16T14:49:45.949 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.949 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.949 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:45.949 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863008+0000", 2024-09-16T14:49:45.949 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4200000000000007e-06 2024-09-16T14:49:45.949 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.949 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.950 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.950 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.865719+0000", 2024-09-16T14:49:45.950 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002710877 2024-09-16T14:49:45.950 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.950 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.950 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.950 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866147+0000", 2024-09-16T14:49:45.950 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000428124 2024-09-16T14:49:45.950 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.950 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.950 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.950 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.876431+0000", 2024-09-16T14:49:45.951 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010284422 2024-09-16T14:49:45.951 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.951 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.951 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.951 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.951 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.951 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:864 3.13s0 3:c9d06351:::smithi04040151-33:3b [read 0~1654784,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:45.951 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.863012+0000", 2024-09-16T14:49:45.951 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.9657284810000002, 2024-09-16T14:49:45.951 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014700533999999999, 2024-09-16T14:49:45.951 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.952 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:45.952 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.952 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.952 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.952 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 864 2024-09-16T14:49:45.952 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.952 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.952 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.952 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.952 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863012+0000", 2024-09-16T14:49:45.952 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.952 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.953 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.953 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.953 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863012+0000", 2024-09-16T14:49:45.953 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.953 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.953 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.953 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.953 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863011+0000", 2024-09-16T14:49:45.953 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:45.953 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.953 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.954 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.954 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863018+0000", 2024-09-16T14:49:45.954 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.5679999999999996e-06 2024-09-16T14:49:45.954 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.954 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.954 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.954 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863026+0000", 2024-09-16T14:49:45.954 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8980000000000002e-06 2024-09-16T14:49:45.954 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.954 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.954 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.954 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863032+0000", 2024-09-16T14:49:45.955 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:49:45.955 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.955 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.955 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.955 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863063+0000", 2024-09-16T14:49:45.955 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0474e-05 2024-09-16T14:49:45.955 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.955 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.955 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:45.955 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863069+0000", 2024-09-16T14:49:45.955 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:49:45.955 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.956 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.956 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.956 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866510+0000", 2024-09-16T14:49:45.956 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00344123 2024-09-16T14:49:45.956 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.956 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.956 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.956 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866610+0000", 2024-09-16T14:49:45.956 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000100385 2024-09-16T14:49:45.956 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.956 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.956 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.957 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.877713+0000", 2024-09-16T14:49:45.957 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011102386000000001 2024-09-16T14:49:45.957 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.957 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.957 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.957 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.957 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.957 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:865 3.13s0 3:c9d06351:::smithi04040151-33:3b [read 0~1654784,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:45.957 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.863041+0000", 2024-09-16T14:49:45.957 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 3.9656995469999998, 2024-09-16T14:49:45.957 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015790143999999999, 2024-09-16T14:49:45.957 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:45.958 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:45.958 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:45.958 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:45.958 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:45.958 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 865 2024-09-16T14:49:45.958 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.958 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:45.958 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.958 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:45.958 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863041+0000", 2024-09-16T14:49:45.958 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.959 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.959 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.959 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:45.959 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863041+0000", 2024-09-16T14:49:45.959 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:45.959 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.959 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.959 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:45.959 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863041+0000", 2024-09-16T14:49:45.959 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:45.959 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.959 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.960 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:45.960 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863043+0000", 2024-09-16T14:49:45.960 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0549999999999998e-06 2024-09-16T14:49:45.960 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.960 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.960 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:45.960 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863047+0000", 2024-09-16T14:49:45.960 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.941e-06 2024-09-16T14:49:45.960 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.960 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.960 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:45.961 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863050+0000", 2024-09-16T14:49:45.961 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:49:45.961 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.961 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.961 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.961 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863083+0000", 2024-09-16T14:49:45.961 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3009999999999997e-05 2024-09-16T14:49:45.961 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.961 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.961 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:45.961 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863085+0000", 2024-09-16T14:49:45.961 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8849999999999999e-06 2024-09-16T14:49:45.962 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.962 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.962 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:45.962 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867224+0000", 2024-09-16T14:49:45.962 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0041382240000000002 2024-09-16T14:49:45.962 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.962 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.962 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:45.962 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867316+0000", 2024-09-16T14:49:45.962 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2559000000000006e-05 2024-09-16T14:49:45.962 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:45.963 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:45.963 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:45.963 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.878831+0000", 2024-09-16T14:49:45.963 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011514912 2024-09-16T14:49:45.963 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.963 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.963 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.963 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:45.963 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:45.963 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:45.964 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T14:49:45.965 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:45.965 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:45.966 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:45.967 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:49:45.967 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:45.964+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:45.967 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T14:49:45.968 INFO:tasks.rados.rados.0.smithi040.stdout:365: done (1 left) 2024-09-16T14:49:45.968 INFO:tasks.rados.rados.0.smithi040.stdout:366: done (0 left) 2024-09-16T14:49:45.968 INFO:tasks.rados.rados.0.smithi040.stdout:367: setattr oid 25 current snap is 35 2024-09-16T14:49:45.969 INFO:tasks.rados.rados.0.smithi040.stdout:368: rollback oid 43 current snap is 35 2024-09-16T14:49:45.970 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 43 to 34 2024-09-16T14:49:45.970 INFO:tasks.rados.rados.0.smithi040.stdout:369: read oid 42 snap -1 2024-09-16T14:49:45.970 INFO:tasks.rados.rados.0.smithi040.stdout:369: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-16T14:49:45.970 INFO:tasks.rados.rados.0.smithi040.stdout:370: read oid 17 snap 34 2024-09-16T14:49:45.970 INFO:tasks.rados.rados.0.smithi040.stdout:370: expect (ObjNum 31 snap 0 seq_num 31) 2024-09-16T14:49:45.970 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 35 current snap is 35 2024-09-16T14:49:45.970 INFO:tasks.rados.rados.0.smithi040.stdout:371: seq_num 131 ranges {4866048=655360,5521408=425984,5947392=638976,6586368=491520} 2024-09-16T14:49:45.971 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T14:49:45.974 INFO:tasks.rados.rados.0.smithi040.stdout:371: writing smithi04040151-35 from 4866048 to 5521408 tid 1 2024-09-16T14:49:45.974 INFO:tasks.rados.rados.0.smithi040.stdout:371: writing smithi04040151-35 from 5521408 to 5947392 tid 2 2024-09-16T14:49:45.974 INFO:tasks.rados.rados.0.smithi040.stdout:371: writing smithi04040151-35 from 5947392 to 6586368 tid 3 2024-09-16T14:49:45.974 INFO:tasks.rados.rados.0.smithi040.stdout:371: writing smithi04040151-35 from 6586368 to 7077888 tid 4 2024-09-16T14:49:45.974 INFO:tasks.rados.rados.0.smithi040.stdout:372: setattr oid 29 current snap is 35 2024-09-16T14:49:45.974 INFO:tasks.rados.rados.0.smithi040.stdout:370: expect (ObjNum 31 snap 0 seq_num 31) 2024-09-16T14:49:45.974 INFO:tasks.rados.rados.0.smithi040.stdout:369: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-16T14:49:45.974 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 25 v 56 (ObjNum 92 snap 17 seq_num 92) dirty exists 2024-09-16T14:49:45.974 INFO:tasks.rados.rados.0.smithi040.stdout:368: finishing rollback tid 0 to smithi04040151-43 2024-09-16T14:49:45.974 INFO:tasks.rados.rados.0.smithi040.stdout:368: finishing rollback tid 1 to smithi04040151-43 2024-09-16T14:49:45.974 INFO:tasks.rados.rados.0.smithi040.stdout:368: finishing rollback tid 2 to smithi04040151-43 2024-09-16T14:49:45.975 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 43 v 120 (ObjNum 62 snap 3 seq_num 62) dirty exists 2024-09-16T14:49:45.975 INFO:tasks.rados.rados.0.smithi040.stdout:371: finishing write tid 1 to smithi04040151-35 2024-09-16T14:49:45.975 INFO:tasks.rados.rados.0.smithi040.stdout:371: finishing write tid 2 to smithi04040151-35 2024-09-16T14:49:45.975 INFO:tasks.rados.rados.0.smithi040.stdout:371: finishing write tid 3 to smithi04040151-35 2024-09-16T14:49:45.975 INFO:tasks.rados.rados.0.smithi040.stdout:371: finishing write tid 4 to smithi04040151-35 2024-09-16T14:49:45.975 INFO:tasks.rados.rados.0.smithi040.stdout:371: finishing write tid 5 to smithi04040151-35 2024-09-16T14:49:45.975 INFO:tasks.rados.rados.0.smithi040.stdout:371: finishing write tid 6 to smithi04040151-35 2024-09-16T14:49:45.975 INFO:tasks.rados.rados.0.smithi040.stdout:371: oid 35 updating version 0 to 73 2024-09-16T14:49:45.975 INFO:tasks.rados.rados.0.smithi040.stdout:371: oid 35 updating version 73 to 75 2024-09-16T14:49:45.975 INFO:tasks.rados.rados.0.smithi040.stdout:371: oid 35 version 75 is already newer than 74 2024-09-16T14:49:45.975 INFO:tasks.rados.rados.0.smithi040.stdout:371: oid 35 updating version 75 to 76 2024-09-16T14:49:45.975 INFO:tasks.rados.rados.0.smithi040.stdout:371: oid 35 version 76 is already newer than 72 2024-09-16T14:49:45.976 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 35 v 76 (ObjNum 131 snap 35 seq_num 131) dirty exists 2024-09-16T14:49:45.976 INFO:tasks.rados.rados.0.smithi040.stdout:371: left oid 35 (ObjNum 131 snap 35 seq_num 131) 2024-09-16T14:49:45.976 INFO:tasks.rados.rados.0.smithi040.stdout:367: done (5 left) 2024-09-16T14:49:45.976 INFO:tasks.rados.rados.0.smithi040.stdout:368: done (4 left) 2024-09-16T14:49:45.976 INFO:tasks.rados.rados.0.smithi040.stdout:369: done (3 left) 2024-09-16T14:49:45.976 INFO:tasks.rados.rados.0.smithi040.stdout:370: done (2 left) 2024-09-16T14:49:45.976 INFO:tasks.rados.rados.0.smithi040.stdout:371: done (1 left) 2024-09-16T14:49:45.976 INFO:tasks.rados.rados.0.smithi040.stdout:373: read oid 46 snap 34 2024-09-16T14:49:45.976 INFO:tasks.rados.rados.0.smithi040.stdout:373: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-16T14:49:45.976 INFO:tasks.rados.rados.0.smithi040.stdout:374: rollback oid 20 current snap is 35 2024-09-16T14:49:45.976 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 20 to 34 2024-09-16T14:49:45.977 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 2 current snap is 35 2024-09-16T14:49:45.977 INFO:tasks.rados.rados.0.smithi040.stdout:375: seq_num 132 ranges {3801088=671744,4472832=573440,5046272=753664,5799936=229376} 2024-09-16T14:49:45.977 INFO:tasks.rados.rados.0.smithi040.stdout:375: writing smithi04040151-2 from 3801088 to 4472832 tid 1 2024-09-16T14:49:45.978 INFO:tasks.rados.rados.0.smithi040.stdout:375: writing smithi04040151-2 from 4472832 to 5046272 tid 2 2024-09-16T14:49:45.980 INFO:tasks.rados.rados.0.smithi040.stdout:375: writing smithi04040151-2 from 5046272 to 5799936 tid 3 2024-09-16T14:49:45.981 INFO:tasks.rados.rados.0.smithi040.stdout:375: writing smithi04040151-2 from 5799936 to 6029312 tid 4 2024-09-16T14:49:45.982 INFO:tasks.rados.rados.0.smithi040.stdout:376: delete oid 24 current snap is 35 2024-09-16T14:49:45.982 INFO:tasks.rados.rados.0.smithi040.stdout:374: finishing rollback tid 0 to smithi04040151-20 2024-09-16T14:49:45.982 INFO:tasks.rados.rados.0.smithi040.stdout:373: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-16T14:49:46.005 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 29 v 81 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-16T14:49:46.005 INFO:tasks.rados.rados.0.smithi040.stdout:374: finishing rollback tid 1 to smithi04040151-20 2024-09-16T14:49:46.005 INFO:tasks.rados.rados.0.smithi040.stdout:374: finishing rollback tid 2 to smithi04040151-20 2024-09-16T14:49:46.005 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 20 v 71 (ObjNum 117 snap 27 seq_num 117) dirty exists 2024-09-16T14:49:46.005 INFO:tasks.rados.rados.0.smithi040.stdout:375: finishing write tid 1 to smithi04040151-2 2024-09-16T14:49:46.006 INFO:tasks.rados.rados.0.smithi040.stdout:375: finishing write tid 2 to smithi04040151-2 2024-09-16T14:49:46.006 INFO:tasks.rados.rados.0.smithi040.stdout:375: finishing write tid 3 to smithi04040151-2 2024-09-16T14:49:46.006 INFO:tasks.rados.rados.0.smithi040.stdout:375: finishing write tid 4 to smithi04040151-2 2024-09-16T14:49:46.006 INFO:tasks.rados.rados.0.smithi040.stdout:375: finishing write tid 5 to smithi04040151-2 2024-09-16T14:49:46.006 INFO:tasks.rados.rados.0.smithi040.stdout:375: finishing write tid 6 to smithi04040151-2 2024-09-16T14:49:46.006 INFO:tasks.rados.rados.0.smithi040.stdout:375: oid 2 updating version 0 to 82 2024-09-16T14:49:46.006 INFO:tasks.rados.rados.0.smithi040.stdout:375: oid 2 version 82 is already newer than 81 2024-09-16T14:49:46.006 INFO:tasks.rados.rados.0.smithi040.stdout:375: oid 2 version 82 is already newer than 79 2024-09-16T14:49:46.006 INFO:tasks.rados.rados.0.smithi040.stdout:375: oid 2 version 82 is already newer than 80 2024-09-16T14:49:46.006 INFO:tasks.rados.rados.0.smithi040.stdout:375: oid 2 version 82 is already newer than 78 2024-09-16T14:49:46.006 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 2 v 82 (ObjNum 132 snap 35 seq_num 132) dirty exists 2024-09-16T14:49:46.007 INFO:tasks.rados.rados.0.smithi040.stdout:375: left oid 2 (ObjNum 132 snap 35 seq_num 132) 2024-09-16T14:49:46.007 INFO:tasks.rados.rados.0.smithi040.stdout:372: done (4 left) 2024-09-16T14:49:46.007 INFO:tasks.rados.rados.0.smithi040.stdout:373: done (3 left) 2024-09-16T14:49:46.007 INFO:tasks.rados.rados.0.smithi040.stdout:374: done (2 left) 2024-09-16T14:49:46.007 INFO:tasks.rados.rados.0.smithi040.stdout:375: done (1 left) 2024-09-16T14:49:46.008 INFO:tasks.rados.rados.0.smithi040.stdout:376: done (0 left) 2024-09-16T14:49:46.008 INFO:tasks.rados.rados.0.smithi040.stdout:377: rollback oid 33 current snap is 35 2024-09-16T14:49:46.008 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 33 to 34 2024-09-16T14:49:46.008 INFO:tasks.rados.rados.0.smithi040.stdout:378: snap_create 2024-09-16T14:49:46.010 INFO:tasks.rados.rados.0.smithi040.stdout:377: finishing rollback tid 0 to smithi04040151-33 2024-09-16T14:49:46.012 INFO:tasks.rados.rados.0.smithi040.stdout:377: finishing rollback tid 1 to smithi04040151-33 2024-09-16T14:49:46.015 INFO:tasks.rados.rados.0.smithi040.stdout:377: finishing rollback tid 2 to smithi04040151-33 2024-09-16T14:49:46.015 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 33 v 43 (ObjNum 98 snap 18 seq_num 98) dirty exists 2024-09-16T14:49:46.066 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:46.065+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:46.083 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:46.083 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:46.083 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:46.083 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:46.092 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:49:46.167 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:46.166+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:46.193 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:46.193 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:46.194 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:46.194 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:46.202 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:49:46.267 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:46.266+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:46.306 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:46.306 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:46.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:46.306 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:46.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.306 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:295 3.9s0 3:96d2b183:::smithi04040151-29:head [copy-from ver 0] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T14:49:46.307 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:46.307 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.388485297000003, 2024-09-16T14:49:46.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010216596999999999, 2024-09-16T14:49:46.307 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.307 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:46.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:46.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:46.307 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:46.307 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 295 2024-09-16T14:49:46.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.307 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:46.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:46.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915430+0000", 2024-09-16T14:49:46.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:46.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915441+0000", 2024-09-16T14:49:46.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0789e-05 2024-09-16T14:49:46.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915462+0000", 2024-09-16T14:49:46.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0845000000000001e-05 2024-09-16T14:49:46.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915473+0000", 2024-09-16T14:49:46.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0805000000000001e-05 2024-09-16T14:49:46.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915528+0000", 2024-09-16T14:49:46.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4772999999999999e-05 2024-09-16T14:49:46.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:46.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915537+0000", 2024-09-16T14:49:46.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6700000000000006e-06 2024-09-16T14:49:46.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.919813+0000", 2024-09-16T14:49:46.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042757840000000004 2024-09-16T14:49:46.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:46.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.919907+0000", 2024-09-16T14:49:46.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504999999999993e-05 2024-09-16T14:49:46.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925649+0000", 2024-09-16T14:49:46.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057427870000000001 2024-09-16T14:49:46.312 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.312 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.312 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.312 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:296 3.9s0 3:96d2b183:::smithi04040151-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-16T14:49:46.312 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:46.312 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.388433380000002, 2024-09-16T14:49:46.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010423623, 2024-09-16T14:49:46.312 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.312 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:46.312 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:46.312 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:46.313 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:46.313 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 296 2024-09-16T14:49:46.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.313 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:46.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:46.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915484+0000", 2024-09-16T14:49:46.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915489+0000", 2024-09-16T14:49:46.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6079999999999998e-06 2024-09-16T14:49:46.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915496+0000", 2024-09-16T14:49:46.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7589999999999997e-06 2024-09-16T14:49:46.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915506+0000", 2024-09-16T14:49:46.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0340000000000002e-06 2024-09-16T14:49:46.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915588+0000", 2024-09-16T14:49:46.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2198000000000001e-05 2024-09-16T14:49:46.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:46.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915594+0000", 2024-09-16T14:49:46.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:49:46.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920099+0000", 2024-09-16T14:49:46.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045048700000000002 2024-09-16T14:49:46.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:46.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920147+0000", 2024-09-16T14:49:46.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8239999999999999e-05 2024-09-16T14:49:46.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925726+0000", 2024-09-16T14:49:46.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055787529999999997 2024-09-16T14:49:46.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:46.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925816+0000", 2024-09-16T14:49:46.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0185000000000005e-05 2024-09-16T14:49:46.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925908+0000", 2024-09-16T14:49:46.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1948999999999997e-05 2024-09-16T14:49:46.319 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.319 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.320 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.320 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as4 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:46.320 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:46.320 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.379878728999998, 2024-09-16T14:49:46.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088260140000000001, 2024-09-16T14:49:46.320 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.320 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.320 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:46.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:46.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:46.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924062+0000", 2024-09-16T14:49:46.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2892e-05 2024-09-16T14:49:46.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924070+0000", 2024-09-16T14:49:46.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8690000000000004e-06 2024-09-16T14:49:46.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924076+0000", 2024-09-16T14:49:46.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0739999999999998e-06 2024-09-16T14:49:46.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924119+0000", 2024-09-16T14:49:46.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2549999999999997e-05 2024-09-16T14:49:46.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924126+0000", 2024-09-16T14:49:46.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8960000000000003e-06 2024-09-16T14:49:46.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932835+0000", 2024-09-16T14:49:46.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087089200000000002 2024-09-16T14:49:46.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932865+0000", 2024-09-16T14:49:46.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0054000000000001e-05 2024-09-16T14:49:46.324 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.324 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.324 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.325 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es1 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:46.325 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:46.325 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.379764141000003, 2024-09-16T14:49:46.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087127249999999993, 2024-09-16T14:49:46.325 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.325 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.325 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:46.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:46.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:46.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924168+0000", 2024-09-16T14:49:46.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4800000000000001e-05 2024-09-16T14:49:46.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924176+0000", 2024-09-16T14:49:46.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.25e-06 2024-09-16T14:49:46.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924182+0000", 2024-09-16T14:49:46.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3640000000000004e-06 2024-09-16T14:49:46.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924229+0000", 2024-09-16T14:49:46.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6477999999999998e-05 2024-09-16T14:49:46.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924238+0000", 2024-09-16T14:49:46.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4429999999999992e-06 2024-09-16T14:49:46.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932848+0000", 2024-09-16T14:49:46.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086097910000000003 2024-09-16T14:49:46.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.329 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.329 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932867+0000", 2024-09-16T14:49:46.329 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8927999999999999e-05 2024-09-16T14:49:46.329 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.329 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.329 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.329 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:394 3.9s0 3:93498a0b:::smithi04040151-32:head [append 0~491520] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:46.329 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:46.329 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.434345659999998, 2024-09-16T14:49:46.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016461614999999999, 2024-09-16T14:49:46.330 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.330 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:46.330 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:46.330 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:46.330 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:46.330 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 394 2024-09-16T14:49:46.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.330 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.330 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.330 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:46.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:46.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869570+0000", 2024-09-16T14:49:46.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:46.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872427+0000", 2024-09-16T14:49:46.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028569070000000001 2024-09-16T14:49:46.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872450+0000", 2024-09-16T14:49:46.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2943e-05 2024-09-16T14:49:46.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872461+0000", 2024-09-16T14:49:46.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0882e-05 2024-09-16T14:49:46.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872521+0000", 2024-09-16T14:49:46.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9639000000000003e-05 2024-09-16T14:49:46.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:46.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872530+0000", 2024-09-16T14:49:46.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8810000000000002e-06 2024-09-16T14:49:46.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.878309+0000", 2024-09-16T14:49:46.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057788520000000001 2024-09-16T14:49:46.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:46.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.878434+0000", 2024-09-16T14:49:46.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012550899999999999 2024-09-16T14:49:46.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.334 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880126+0000", 2024-09-16T14:49:46.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016919019999999999 2024-09-16T14:49:46.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885887+0000", 2024-09-16T14:49:46.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057608959999999997 2024-09-16T14:49:46.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:46.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885957+0000", 2024-09-16T14:49:46.335 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0504000000000001e-05 2024-09-16T14:49:46.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.335 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.335 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886034+0000", 2024-09-16T14:49:46.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6661000000000003e-05 2024-09-16T14:49:46.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.336 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.336 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.336 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:395 3.9s0 3:93498a0b:::smithi04040151-32:head [append 0~770048] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:46.336 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:46.336 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.429618245, 2024-09-16T14:49:46.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.018911853999999999, 2024-09-16T14:49:46.336 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.336 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:46.337 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:46.337 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:46.337 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:46.337 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 395 2024-09-16T14:49:46.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.337 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:46.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874298+0000", 2024-09-16T14:49:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875671+0000", 2024-09-16T14:49:46.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0013727069999999999 2024-09-16T14:49:46.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875689+0000", 2024-09-16T14:49:46.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7832000000000001e-05 2024-09-16T14:49:46.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875700+0000", 2024-09-16T14:49:46.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1267000000000001e-05 2024-09-16T14:49:46.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.339 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875758+0000", 2024-09-16T14:49:46.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8060999999999999e-05 2024-09-16T14:49:46.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:46.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875766+0000", 2024-09-16T14:49:46.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9580000000000007e-06 2024-09-16T14:49:46.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.340 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.340 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.340 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881004+0000", 2024-09-16T14:49:46.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052375169999999997 2024-09-16T14:49:46.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:46.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881098+0000", 2024-09-16T14:49:46.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3875999999999994e-05 2024-09-16T14:49:46.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882500+0000", 2024-09-16T14:49:46.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014019600000000001 2024-09-16T14:49:46.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886041+0000", 2024-09-16T14:49:46.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035415210000000002 2024-09-16T14:49:46.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:46.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893136+0000", 2024-09-16T14:49:46.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070949330000000003 2024-09-16T14:49:46.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893212+0000", 2024-09-16T14:49:46.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5739000000000001e-05 2024-09-16T14:49:46.342 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.342 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.343 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.343 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:396 3.9s0 3:93498a0b:::smithi04040151-32:head [setxattr _header (57) in=7b] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:46.343 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:46.343 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.428201090000002, 2024-09-16T14:49:46.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017781597999999999, 2024-09-16T14:49:46.343 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.343 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:46.343 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:46.343 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:46.343 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:46.343 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 396 2024-09-16T14:49:46.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.344 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:46.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:46.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875715+0000", 2024-09-16T14:49:46.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:46.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875721+0000", 2024-09-16T14:49:46.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6989999999999997e-06 2024-09-16T14:49:46.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875728+0000", 2024-09-16T14:49:46.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9040000000000003e-06 2024-09-16T14:49:46.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875734+0000", 2024-09-16T14:49:46.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4519999999999996e-06 2024-09-16T14:49:46.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875800+0000", 2024-09-16T14:49:46.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6538000000000006e-05 2024-09-16T14:49:46.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:46.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875804+0000", 2024-09-16T14:49:46.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.771e-06 2024-09-16T14:49:46.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883232+0000", 2024-09-16T14:49:46.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074282009999999997 2024-09-16T14:49:46.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:46.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883323+0000", 2024-09-16T14:49:46.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0740000000000002e-05 2024-09-16T14:49:46.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884057+0000", 2024-09-16T14:49:46.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00073422299999999999 2024-09-16T14:49:46.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893020+0000", 2024-09-16T14:49:46.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089623210000000005 2024-09-16T14:49:46.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:46.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893403+0000", 2024-09-16T14:49:46.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00038375799999999999 2024-09-16T14:49:46.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893499+0000", 2024-09-16T14:49:46.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5284000000000005e-05 2024-09-16T14:49:46.349 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.349 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.349 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.349 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:397 3.9s0 3:93498a0b:::smithi04040151-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:46.349 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:46.350 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.428169965000002, 2024-09-16T14:49:46.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019838090999999999, 2024-09-16T14:49:46.350 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.350 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:46.350 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:46.350 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:46.350 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:46.350 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 397 2024-09-16T14:49:46.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.350 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:46.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:46.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:46.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875750+0000", 2024-09-16T14:49:46.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3939999999999999e-06 2024-09-16T14:49:46.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875755+0000", 2024-09-16T14:49:46.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.31e-06 2024-09-16T14:49:46.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875760+0000", 2024-09-16T14:49:46.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4769999999999997e-06 2024-09-16T14:49:46.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875824+0000", 2024-09-16T14:49:46.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3893000000000006e-05 2024-09-16T14:49:46.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:46.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875829+0000", 2024-09-16T14:49:46.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3489999999999999e-06 2024-09-16T14:49:46.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.353 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.353 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885652+0000", 2024-09-16T14:49:46.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0098232530000000005 2024-09-16T14:49:46.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:46.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885748+0000", 2024-09-16T14:49:46.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5366999999999997e-05 2024-09-16T14:49:46.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893876+0000", 2024-09-16T14:49:46.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081279479999999994 2024-09-16T14:49:46.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:46.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893967+0000", 2024-09-16T14:49:46.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0777000000000002e-05 2024-09-16T14:49:46.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895586+0000", 2024-09-16T14:49:46.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016197029999999999 2024-09-16T14:49:46.355 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.355 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.355 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.355 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(osd.3.17:4 3.9s0 3:9b721ec3:::smithi04040151-12:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:46.356 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:46.356 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 52.419947671000003, 2024-09-16T14:49:46.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011299052, 2024-09-16T14:49:46.356 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.356 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:46.356 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:46.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:46.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883969+0000", 2024-09-16T14:49:46.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:46.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883976+0000", 2024-09-16T14:49:46.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3730000000000004e-06 2024-09-16T14:49:46.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883999+0000", 2024-09-16T14:49:46.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3390000000000001e-05 2024-09-16T14:49:46.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884007+0000", 2024-09-16T14:49:46.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1489999999999997e-06 2024-09-16T14:49:46.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886141+0000", 2024-09-16T14:49:46.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0021347240000000002 2024-09-16T14:49:46.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:46.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886230+0000", 2024-09-16T14:49:46.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8234999999999998e-05 2024-09-16T14:49:46.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.359 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.359 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895269+0000", 2024-09-16T14:49:46.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090399230000000001 2024-09-16T14:49:46.360 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.360 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.360 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.360 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:440 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:46.360 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:46.360 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.427894350000003, 2024-09-16T14:49:46.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01253772, 2024-09-16T14:49:46.360 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.360 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:46.360 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:46.360 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:46.361 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:46.361 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 440 2024-09-16T14:49:46.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.361 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:46.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:46.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876022+0000", 2024-09-16T14:49:46.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:46.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876033+0000", 2024-09-16T14:49:46.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1474000000000001e-05 2024-09-16T14:49:46.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876051+0000", 2024-09-16T14:49:46.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7628000000000001e-05 2024-09-16T14:49:46.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876060+0000", 2024-09-16T14:49:46.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339999999999996e-06 2024-09-16T14:49:46.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876128+0000", 2024-09-16T14:49:46.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7460999999999997e-05 2024-09-16T14:49:46.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:46.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876138+0000", 2024-09-16T14:49:46.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8649999999999996e-06 2024-09-16T14:49:46.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882141+0000", 2024-09-16T14:49:46.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060036550000000001 2024-09-16T14:49:46.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:46.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882233+0000", 2024-09-16T14:49:46.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1626000000000007e-05 2024-09-16T14:49:46.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.888561+0000", 2024-09-16T14:49:46.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063284769999999999 2024-09-16T14:49:46.366 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.366 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.366 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.366 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:441 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:46.366 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:46.366 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.427847292999999, 2024-09-16T14:49:46.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.013066019999999999, 2024-09-16T14:49:46.366 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.366 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:46.367 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:46.367 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:46.367 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:46.367 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 441 2024-09-16T14:49:46.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.367 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.367 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:46.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.367 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:46.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876070+0000", 2024-09-16T14:49:46.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.368 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.368 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876074+0000", 2024-09-16T14:49:46.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1330000000000001e-06 2024-09-16T14:49:46.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876083+0000", 2024-09-16T14:49:46.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2519999999999999e-06 2024-09-16T14:49:46.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876088+0000", 2024-09-16T14:49:46.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5509999999999999e-06 2024-09-16T14:49:46.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876235+0000", 2024-09-16T14:49:46.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014688900000000001 2024-09-16T14:49:46.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:46.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876243+0000", 2024-09-16T14:49:46.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5490000000000001e-06 2024-09-16T14:49:46.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.884127+0000", 2024-09-16T14:49:46.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078848489999999993 2024-09-16T14:49:46.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:46.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.884238+0000", 2024-09-16T14:49:46.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111007 2024-09-16T14:49:46.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889137+0000", 2024-09-16T14:49:46.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048983739999999996 2024-09-16T14:49:46.371 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.371 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.371 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.372 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:442 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:46.372 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:46.372 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.427808634999998, 2024-09-16T14:49:46.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014710668, 2024-09-16T14:49:46.372 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.372 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:46.372 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:46.372 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:46.372 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:46.372 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 442 2024-09-16T14:49:46.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.373 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:46.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:46.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:46.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876113+0000", 2024-09-16T14:49:46.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0759999999999996e-06 2024-09-16T14:49:46.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876119+0000", 2024-09-16T14:49:46.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3339999999999998e-06 2024-09-16T14:49:46.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876126+0000", 2024-09-16T14:49:46.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8410000000000002e-06 2024-09-16T14:49:46.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876273+0000", 2024-09-16T14:49:46.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014714900000000001 2024-09-16T14:49:46.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876279+0000", 2024-09-16T14:49:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7819999999999999e-06 2024-09-16T14:49:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.885397+0000", 2024-09-16T14:49:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091178149999999996 2024-09-16T14:49:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.885492+0000", 2024-09-16T14:49:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5384999999999996e-05 2024-09-16T14:49:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.890820+0000", 2024-09-16T14:49:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053278559999999997 2024-09-16T14:49:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs3 61/27 ECSubWrite(tid=238, reqid=osd.3.0:238, at_version=61'42, trim_to=0'0, roll_forward_to=60'38))", 2024-09-16T14:49:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:46.378 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.377049851000002, 2024-09-16T14:49:46.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089143629999999998, 2024-09-16T14:49:46.378 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.378 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.378 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:46.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:46.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:46.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926896+0000", 2024-09-16T14:49:46.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7764999999999999e-05 2024-09-16T14:49:46.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926909+0000", 2024-09-16T14:49:46.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3454e-05 2024-09-16T14:49:46.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926918+0000", 2024-09-16T14:49:46.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7099999999999996e-06 2024-09-16T14:49:46.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926968+0000", 2024-09-16T14:49:46.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0778000000000001e-05 2024-09-16T14:49:46.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926989+0000", 2024-09-16T14:49:46.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0069999999999999e-05 2024-09-16T14:49:46.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935750+0000", 2024-09-16T14:49:46.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.008761876 2024-09-16T14:49:46.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935783+0000", 2024-09-16T14:49:46.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2202000000000001e-05 2024-09-16T14:49:46.382 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.382 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.382 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.382 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s6 61/27 ECSubWrite(tid=239, reqid=osd.3.0:239, at_version=61'79, trim_to=0'0, roll_forward_to=60'75))", 2024-09-16T14:49:46.382 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:46.382 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.376987053999997, 2024-09-16T14:49:46.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.009451849, 2024-09-16T14:49:46.382 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.383 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.383 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:46.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.383 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:46.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:46.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926951+0000", 2024-09-16T14:49:46.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9959999999999999e-05 2024-09-16T14:49:46.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926962+0000", 2024-09-16T14:49:46.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1498000000000001e-05 2024-09-16T14:49:46.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926970+0000", 2024-09-16T14:49:46.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5410000000000003e-06 2024-09-16T14:49:46.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.927857+0000", 2024-09-16T14:49:46.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00088758200000000004 2024-09-16T14:49:46.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.927868+0000", 2024-09-16T14:49:46.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1234e-05 2024-09-16T14:49:46.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936355+0000", 2024-09-16T14:49:46.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0084864520000000002 2024-09-16T14:49:46.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936383+0000", 2024-09-16T14:49:46.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8068e-05 2024-09-16T14:49:46.386 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.386 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.387 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.387 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:709 3.9s0 3:96d2b183:::smithi04040151-29:head [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:46.387 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.900189+0000", 2024-09-16T14:49:46.387 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.40372958, 2024-09-16T14:49:46.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011070020999999999, 2024-09-16T14:49:46.387 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.387 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:46.387 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:46.387 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:46.387 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:46.387 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 709 2024-09-16T14:49:46.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.388 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900189+0000", 2024-09-16T14:49:46.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900189+0000", 2024-09-16T14:49:46.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900188+0000", 2024-09-16T14:49:46.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900192+0000", 2024-09-16T14:49:46.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7670000000000001e-06 2024-09-16T14:49:46.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900196+0000", 2024-09-16T14:49:46.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5480000000000003e-06 2024-09-16T14:49:46.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900201+0000", 2024-09-16T14:49:46.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5310000000000003e-06 2024-09-16T14:49:46.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900321+0000", 2024-09-16T14:49:46.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120096 2024-09-16T14:49:46.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:46.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900327+0000", 2024-09-16T14:49:46.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3389999999999998e-06 2024-09-16T14:49:46.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.907572+0000", 2024-09-16T14:49:46.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0072446180000000004 2024-09-16T14:49:46.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:46.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.907672+0000", 2024-09-16T14:49:46.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9617000000000005e-05 2024-09-16T14:49:46.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.911259+0000", 2024-09-16T14:49:46.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035870350000000001 2024-09-16T14:49:46.392 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.392 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.392 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.392 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs3 81/65 ECSubWrite(tid=337, reqid=osd.3.0:337, at_version=81'65, trim_to=0'0, roll_forward_to=80'61))", 2024-09-16T14:49:46.392 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:27.927767+0000", 2024-09-16T14:49:46.392 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.376151565000001, 2024-09-16T14:49:46.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088756419999999996, 2024-09-16T14:49:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927767+0000", 2024-09-16T14:49:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927767+0000", 2024-09-16T14:49:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927766+0000", 2024-09-16T14:49:46.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:46.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927792+0000", 2024-09-16T14:49:46.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5914999999999998e-05 2024-09-16T14:49:46.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927809+0000", 2024-09-16T14:49:46.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7926000000000001e-05 2024-09-16T14:49:46.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927820+0000", 2024-09-16T14:49:46.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0277e-05 2024-09-16T14:49:46.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930629+0000", 2024-09-16T14:49:46.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00280919 2024-09-16T14:49:46.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930637+0000", 2024-09-16T14:49:46.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3000000000000002e-06 2024-09-16T14:49:46.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.936614+0000", 2024-09-16T14:49:46.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059770400000000003 2024-09-16T14:49:46.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.396 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.936642+0000", 2024-09-16T14:49:46.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7911000000000001e-05 2024-09-16T14:49:46.397 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.397 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.397 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.397 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.397 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:822 3.19s0 3:9b72282b:::smithi04040151-27:head [copy-from ver 52] snapc 3b=[3b,3a,38,37] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T14:49:46.397 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.944334+0000", 2024-09-16T14:49:46.397 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3595836380000001, 2024-09-16T14:49:46.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.073023423, 2024-09-16T14:49:46.397 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.397 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:46.397 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:46.397 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:46.398 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:46.398 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 822 2024-09-16T14:49:46.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.398 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944334+0000", 2024-09-16T14:49:46.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944334+0000", 2024-09-16T14:49:46.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944332+0000", 2024-09-16T14:49:46.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:46.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944343+0000", 2024-09-16T14:49:46.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0324e-05 2024-09-16T14:49:46.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944361+0000", 2024-09-16T14:49:46.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8354000000000001e-05 2024-09-16T14:49:46.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944372+0000", 2024-09-16T14:49:46.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1123e-05 2024-09-16T14:49:46.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958524+0000", 2024-09-16T14:49:46.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014151403999999999 2024-09-16T14:49:46.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:46.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958527+0000", 2024-09-16T14:49:46.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.101e-06 2024-09-16T14:49:46.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.970154+0000", 2024-09-16T14:49:46.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0116273200000001 2024-09-16T14:49:46.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:46.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.970333+0000", 2024-09-16T14:49:46.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017934799999999999 2024-09-16T14:49:46.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.005943+0000", 2024-09-16T14:49:46.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035610044 2024-09-16T14:49:46.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.014246+0000", 2024-09-16T14:49:46.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083026430000000002 2024-09-16T14:49:46.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:46.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017305+0000", 2024-09-16T14:49:46.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003059291 2024-09-16T14:49:46.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017358+0000", 2024-09-16T14:49:46.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2552000000000001e-05 2024-09-16T14:49:46.403 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.403 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.403 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.404 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:823 3.19s0 3:9b72282b:::smithi04040151-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90)", 2024-09-16T14:49:46.404 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:46.404 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3595316420000003, 2024-09-16T14:49:46.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0666982119999999, 2024-09-16T14:49:46.404 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.404 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:46.404 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:46.404 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:46.404 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:46.404 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 823 2024-09-16T14:49:46.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.405 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:46.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:46.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:46.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944389+0000", 2024-09-16T14:49:46.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9960000000000002e-06 2024-09-16T14:49:46.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944399+0000", 2024-09-16T14:49:46.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8179999999999999e-06 2024-09-16T14:49:46.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944407+0000", 2024-09-16T14:49:46.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.456e-06 2024-09-16T14:49:46.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958549+0000", 2024-09-16T14:49:46.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014142050999999999 2024-09-16T14:49:46.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:46.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958550+0000", 2024-09-16T14:49:46.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1200000000000002e-07 2024-09-16T14:49:46.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971413+0000", 2024-09-16T14:49:46.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.012862798 2024-09-16T14:49:46.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:46.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971466+0000", 2024-09-16T14:49:46.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3282999999999998e-05 2024-09-16T14:49:46.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010860+0000", 2024-09-16T14:49:46.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.039393862000000002 2024-09-16T14:49:46.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:46.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010968+0000", 2024-09-16T14:49:46.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108378 2024-09-16T14:49:46.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.409 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.409 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.011085+0000", 2024-09-16T14:49:46.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116402 2024-09-16T14:49:46.409 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.410 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.410 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.410 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es1 90/27 ECSubWrite(tid=301, reqid=osd.11.0:301, at_version=90'62, trim_to=0'0, roll_forward_to=83'60))", 2024-09-16T14:49:46.410 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.956242+0000", 2024-09-16T14:49:46.410 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3476765910000008, 2024-09-16T14:49:46.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083679859999999991, 2024-09-16T14:49:46.410 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.410 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.410 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956242+0000", 2024-09-16T14:49:46.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956242+0000", 2024-09-16T14:49:46.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956240+0000", 2024-09-16T14:49:46.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:46.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956267+0000", 2024-09-16T14:49:46.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6101999999999998e-05 2024-09-16T14:49:46.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956279+0000", 2024-09-16T14:49:46.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2118e-05 2024-09-16T14:49:46.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956290+0000", 2024-09-16T14:49:46.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1311000000000001e-05 2024-09-16T14:49:46.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956492+0000", 2024-09-16T14:49:46.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000201795 2024-09-16T14:49:46.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956501+0000", 2024-09-16T14:49:46.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1060000000000006e-06 2024-09-16T14:49:46.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.964580+0000", 2024-09-16T14:49:46.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0080795469999999994 2024-09-16T14:49:46.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.964609+0000", 2024-09-16T14:49:46.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9043e-05 2024-09-16T14:49:46.414 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.414 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.414 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.415 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.18s4 99/92 ECSubWrite(tid=441, reqid=client.4500.0:905, at_version=99'119, trim_to=0'0, roll_forward_to=98'116))", 2024-09-16T14:49:46.415 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:45.876953+0000", 2024-09-16T14:49:46.415 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.42696493299999999, 2024-09-16T14:49:46.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083132399999999995, 2024-09-16T14:49:46.415 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.415 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.415 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.415 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.415 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876953+0000", 2024-09-16T14:49:46.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876953+0000", 2024-09-16T14:49:46.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876952+0000", 2024-09-16T14:49:46.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876973+0000", 2024-09-16T14:49:46.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0709000000000001e-05 2024-09-16T14:49:46.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876995+0000", 2024-09-16T14:49:46.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1634e-05 2024-09-16T14:49:46.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.877003+0000", 2024-09-16T14:49:46.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1270000000000003e-06 2024-09-16T14:49:46.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.877447+0000", 2024-09-16T14:49:46.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044436600000000002 2024-09-16T14:49:46.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.877459+0000", 2024-09-16T14:49:46.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1367e-05 2024-09-16T14:49:46.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.885241+0000", 2024-09-16T14:49:46.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007781924 2024-09-16T14:49:46.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.885266+0000", 2024-09-16T14:49:46.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5795999999999998e-05 2024-09-16T14:49:46.419 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.419 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.419 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.419 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.419 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.419 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:46.421 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:49:46.421 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:46.420+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:46.521 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:46.521+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:46.565 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:46.565 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:46.565 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:46.565 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:46.573 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:49:46.622 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:46.621+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:46.676 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:46.676 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:46.676 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:46.684 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:49:46.727 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:46.726+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:46.788 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:46.788 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:46.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:46.789 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:46.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.789 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:49:46.789 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:46.789 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 65.876694893000007, 2024-09-16T14:49:46.789 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007542137, 2024-09-16T14:49:46.789 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.789 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.789 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.789 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.789 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:46.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:46.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.790 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.790 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:46.790 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.790 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909917+0000", 2024-09-16T14:49:46.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6513000000000001e-05 2024-09-16T14:49:46.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:49:46.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:49:46.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.791 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.791 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.791 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909941+0000", 2024-09-16T14:49:46.791 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1859999999999992e-06 2024-09-16T14:49:46.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910004+0000", 2024-09-16T14:49:46.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2706999999999994e-05 2024-09-16T14:49:46.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910012+0000", 2024-09-16T14:49:46.792 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.157e-06 2024-09-16T14:49:46.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.792 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.792 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.792 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917395+0000", 2024-09-16T14:49:46.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073827659999999998 2024-09-16T14:49:46.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.793 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.793 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917423+0000", 2024-09-16T14:49:46.793 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8568000000000002e-05 2024-09-16T14:49:46.793 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.793 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.793 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.793 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.793 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=37'32, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:46.793 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:46.793 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.865584548999998, 2024-09-16T14:49:46.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070991120000000003, 2024-09-16T14:49:46.794 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.794 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.794 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:46.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.794 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.794 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.794 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:46.794 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920991+0000", 2024-09-16T14:49:46.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921010+0000", 2024-09-16T14:49:46.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8529000000000002e-05 2024-09-16T14:49:46.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.795 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.795 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921018+0000", 2024-09-16T14:49:46.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4670000000000004e-06 2024-09-16T14:49:46.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921025+0000", 2024-09-16T14:49:46.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9330000000000004e-06 2024-09-16T14:49:46.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921076+0000", 2024-09-16T14:49:46.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0662999999999999e-05 2024-09-16T14:49:46.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921084+0000", 2024-09-16T14:49:46.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4780000000000008e-06 2024-09-16T14:49:46.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928073+0000", 2024-09-16T14:49:46.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069883940000000002 2024-09-16T14:49:46.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928091+0000", 2024-09-16T14:49:46.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7992999999999999e-05 2024-09-16T14:49:46.797 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.798 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.798 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.798 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:46.798 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:46.798 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.861376178999997, 2024-09-16T14:49:46.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079457759999999999, 2024-09-16T14:49:46.798 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.798 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.798 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:46.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:46.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:46.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925222+0000", 2024-09-16T14:49:46.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1871000000000001e-05 2024-09-16T14:49:46.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925238+0000", 2024-09-16T14:49:46.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6719999999999999e-05 2024-09-16T14:49:46.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925244+0000", 2024-09-16T14:49:46.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3630000000000002e-06 2024-09-16T14:49:46.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925291+0000", 2024-09-16T14:49:46.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6944000000000003e-05 2024-09-16T14:49:46.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.801 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925299+0000", 2024-09-16T14:49:46.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:49:46.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933129+0000", 2024-09-16T14:49:46.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078297330000000002 2024-09-16T14:49:46.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933146+0000", 2024-09-16T14:49:46.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.664e-05 2024-09-16T14:49:46.802 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.802 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.802 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.803 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es5 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:46.803 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:46.803 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.861324543999999, 2024-09-16T14:49:46.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078358790000000005, 2024-09-16T14:49:46.803 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.803 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.803 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:46.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:46.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925251+0000", 2024-09-16T14:49:46.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925265+0000", 2024-09-16T14:49:46.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4078e-05 2024-09-16T14:49:46.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925273+0000", 2024-09-16T14:49:46.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1069999999999999e-06 2024-09-16T14:49:46.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925279+0000", 2024-09-16T14:49:46.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4409999999999998e-06 2024-09-16T14:49:46.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925330+0000", 2024-09-16T14:49:46.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1208999999999997e-05 2024-09-16T14:49:46.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925339+0000", 2024-09-16T14:49:46.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8109999999999992e-06 2024-09-16T14:49:46.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933064+0000", 2024-09-16T14:49:46.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077252149999999997 2024-09-16T14:49:46.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933088+0000", 2024-09-16T14:49:46.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3317999999999999e-05 2024-09-16T14:49:46.807 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.807 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.807 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.807 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:46.807 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933425+0000", 2024-09-16T14:49:46.807 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.853151365999999, 2024-09-16T14:49:46.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090001820000000007, 2024-09-16T14:49:46.807 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.807 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.808 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933425+0000", 2024-09-16T14:49:46.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933425+0000", 2024-09-16T14:49:46.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933424+0000", 2024-09-16T14:49:46.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:46.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933478+0000", 2024-09-16T14:49:46.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3735e-05 2024-09-16T14:49:46.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933493+0000", 2024-09-16T14:49:46.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4978e-05 2024-09-16T14:49:46.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933502+0000", 2024-09-16T14:49:46.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6509999999999996e-06 2024-09-16T14:49:46.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933761+0000", 2024-09-16T14:49:46.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025924500000000002 2024-09-16T14:49:46.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933772+0000", 2024-09-16T14:49:46.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0694e-05 2024-09-16T14:49:46.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942409+0000", 2024-09-16T14:49:46.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086371539999999993 2024-09-16T14:49:46.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942425+0000", 2024-09-16T14:49:46.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5781e-05 2024-09-16T14:49:46.811 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.811 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.811 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.812 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.cs4 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:46.812 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:46.812 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.85278976, 2024-09-16T14:49:46.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086187320000000005, 2024-09-16T14:49:46.812 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.812 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.812 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:46.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933807+0000", 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1107e-05 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933817+0000", 2024-09-16T14:49:46.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8870000000000004e-06 2024-09-16T14:49:46.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933826+0000", 2024-09-16T14:49:46.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5319999999999995e-06 2024-09-16T14:49:46.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933899+0000", 2024-09-16T14:49:46.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2294999999999998e-05 2024-09-16T14:49:46.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933908+0000", 2024-09-16T14:49:46.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4110000000000002e-06 2024-09-16T14:49:46.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942383+0000", 2024-09-16T14:49:46.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0084747099999999999 2024-09-16T14:49:46.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942405+0000", 2024-09-16T14:49:46.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2399e-05 2024-09-16T14:49:46.815 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.816 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.816 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.816 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:46.816 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.934497+0000", 2024-09-16T14:49:46.816 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.852079607, 2024-09-16T14:49:46.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078697109999999997, 2024-09-16T14:49:46.816 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.816 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.816 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934497+0000", 2024-09-16T14:49:46.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934497+0000", 2024-09-16T14:49:46.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934496+0000", 2024-09-16T14:49:46.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:46.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935463+0000", 2024-09-16T14:49:46.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00096685900000000003 2024-09-16T14:49:46.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935498+0000", 2024-09-16T14:49:46.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5179000000000003e-05 2024-09-16T14:49:46.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935509+0000", 2024-09-16T14:49:46.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1223000000000001e-05 2024-09-16T14:49:46.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935573+0000", 2024-09-16T14:49:46.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4114000000000003e-05 2024-09-16T14:49:46.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935583+0000", 2024-09-16T14:49:46.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.008e-05 2024-09-16T14:49:46.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942344+0000", 2024-09-16T14:49:46.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067610760000000004 2024-09-16T14:49:46.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942366+0000", 2024-09-16T14:49:46.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2059e-05 2024-09-16T14:49:46.820 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.820 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.820 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.820 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 61/27 ECSubWrite(tid=242, reqid=osd.3.0:242, at_version=61'79, trim_to=0'0, roll_forward_to=60'77))", 2024-09-16T14:49:46.821 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.944331+0000", 2024-09-16T14:49:46.821 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.842245476999999, 2024-09-16T14:49:46.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083214770000000007, 2024-09-16T14:49:46.821 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.821 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.821 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944331+0000", 2024-09-16T14:49:46.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944331+0000", 2024-09-16T14:49:46.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944330+0000", 2024-09-16T14:49:46.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:46.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944354+0000", 2024-09-16T14:49:46.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3702000000000001e-05 2024-09-16T14:49:46.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944367+0000", 2024-09-16T14:49:46.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3485999999999999e-05 2024-09-16T14:49:46.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944375+0000", 2024-09-16T14:49:46.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3219999999999993e-06 2024-09-16T14:49:46.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944425+0000", 2024-09-16T14:49:46.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9289000000000002e-05 2024-09-16T14:49:46.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944436+0000", 2024-09-16T14:49:46.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1381e-05 2024-09-16T14:49:46.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.952629+0000", 2024-09-16T14:49:46.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081932840000000003 2024-09-16T14:49:46.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.952652+0000", 2024-09-16T14:49:46.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.296e-05 2024-09-16T14:49:46.825 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.825 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.825 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.825 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 77/27 ECSubWrite(tid=311, reqid=osd.3.0:311, at_version=77'100, trim_to=0'0, roll_forward_to=76'98))", 2024-09-16T14:49:46.825 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.915451+0000", 2024-09-16T14:49:46.825 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.871125512999999, 2024-09-16T14:49:46.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079661820000000005, 2024-09-16T14:49:46.825 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.825 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.825 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915451+0000", 2024-09-16T14:49:46.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915451+0000", 2024-09-16T14:49:46.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915450+0000", 2024-09-16T14:49:46.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:46.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915534+0000", 2024-09-16T14:49:46.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4160999999999996e-05 2024-09-16T14:49:46.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915547+0000", 2024-09-16T14:49:46.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2764e-05 2024-09-16T14:49:46.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915555+0000", 2024-09-16T14:49:46.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2090000000000005e-06 2024-09-16T14:49:46.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915659+0000", 2024-09-16T14:49:46.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103879 2024-09-16T14:49:46.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915685+0000", 2024-09-16T14:49:46.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5959999999999999e-05 2024-09-16T14:49:46.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923392+0000", 2024-09-16T14:49:46.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077072379999999999 2024-09-16T14:49:46.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923417+0000", 2024-09-16T14:49:46.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5000000000000001e-05 2024-09-16T14:49:46.829 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.829 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.829 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.829 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.0s1 77/65 ECSubWrite(tid=279, reqid=osd.11.0:279, at_version=77'35, trim_to=0'0, roll_forward_to=74'31))", 2024-09-16T14:49:46.829 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.916327+0000", 2024-09-16T14:49:46.829 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.870249450999999, 2024-09-16T14:49:46.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070483610000000004, 2024-09-16T14:49:46.830 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.830 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.830 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916327+0000", 2024-09-16T14:49:46.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916327+0000", 2024-09-16T14:49:46.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916326+0000", 2024-09-16T14:49:46.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:46.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916348+0000", 2024-09-16T14:49:46.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2327000000000002e-05 2024-09-16T14:49:46.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916363+0000", 2024-09-16T14:49:46.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4295e-05 2024-09-16T14:49:46.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916371+0000", 2024-09-16T14:49:46.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9519999999999994e-06 2024-09-16T14:49:46.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916417+0000", 2024-09-16T14:49:46.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6279000000000002e-05 2024-09-16T14:49:46.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916427+0000", 2024-09-16T14:49:46.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0164e-05 2024-09-16T14:49:46.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923355+0000", 2024-09-16T14:49:46.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069278569999999999 2024-09-16T14:49:46.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923375+0000", 2024-09-16T14:49:46.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0341999999999999e-05 2024-09-16T14:49:46.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.833 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.834 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 77/72 ECSubWrite(tid=14, reqid=client.4500.0:702, at_version=77'33, trim_to=0'0, roll_forward_to=77'32))", 2024-09-16T14:49:46.834 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:46.834 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 22.868885751000001, 2024-09-16T14:49:46.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070951110000000003, 2024-09-16T14:49:46.834 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.834 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.834 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:46.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:46.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:46.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:46.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918213+0000", 2024-09-16T14:49:46.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000523721 2024-09-16T14:49:46.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918232+0000", 2024-09-16T14:49:46.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8743999999999999e-05 2024-09-16T14:49:46.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918241+0000", 2024-09-16T14:49:46.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9539999999999993e-06 2024-09-16T14:49:46.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918356+0000", 2024-09-16T14:49:46.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011514999999999999 2024-09-16T14:49:46.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918367+0000", 2024-09-16T14:49:46.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0933e-05 2024-09-16T14:49:46.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.924758+0000", 2024-09-16T14:49:46.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063913499999999996 2024-09-16T14:49:46.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.924786+0000", 2024-09-16T14:49:46.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7129000000000001e-05 2024-09-16T14:49:46.838 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.838 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.838 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.838 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 81/27 ECSubWrite(tid=339, reqid=osd.3.0:339, at_version=81'108, trim_to=0'0, roll_forward_to=80'104))", 2024-09-16T14:49:46.838 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:27.930359+0000", 2024-09-16T14:49:46.838 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.856217434000001, 2024-09-16T14:49:46.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0085286319999999995, 2024-09-16T14:49:46.838 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.838 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.839 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930359+0000", 2024-09-16T14:49:46.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930359+0000", 2024-09-16T14:49:46.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930358+0000", 2024-09-16T14:49:46.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930382+0000", 2024-09-16T14:49:46.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4034999999999999e-05 2024-09-16T14:49:46.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930393+0000", 2024-09-16T14:49:46.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1062e-05 2024-09-16T14:49:46.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930403+0000", 2024-09-16T14:49:46.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8779999999999993e-06 2024-09-16T14:49:46.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930675+0000", 2024-09-16T14:49:46.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027152900000000001 2024-09-16T14:49:46.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930681+0000", 2024-09-16T14:49:46.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.229e-06 2024-09-16T14:49:46.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.938865+0000", 2024-09-16T14:49:46.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081845249999999998 2024-09-16T14:49:46.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.938887+0000", 2024-09-16T14:49:46.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1946e-05 2024-09-16T14:49:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 81/65 ECSubWrite(tid=83, reqid=osd.1.0:83, at_version=81'66, trim_to=0'0, roll_forward_to=80'64))", 2024-09-16T14:49:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:27.931996+0000", 2024-09-16T14:49:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 18.854580678000001, 2024-09-16T14:49:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074671399999999997, 2024-09-16T14:49:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.931996+0000", 2024-09-16T14:49:46.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.931996+0000", 2024-09-16T14:49:46.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.931995+0000", 2024-09-16T14:49:46.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.932025+0000", 2024-09-16T14:49:46.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9884999999999999e-05 2024-09-16T14:49:46.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.932035+0000", 2024-09-16T14:49:46.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0182e-05 2024-09-16T14:49:46.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.932043+0000", 2024-09-16T14:49:46.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0900000000000005e-06 2024-09-16T14:49:46.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.933159+0000", 2024-09-16T14:49:46.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011161980000000001 2024-09-16T14:49:46.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.933170+0000", 2024-09-16T14:49:46.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0925e-05 2024-09-16T14:49:46.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.939441+0000", 2024-09-16T14:49:46.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062703339999999998 2024-09-16T14:49:46.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.939463+0000", 2024-09-16T14:49:46.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2005e-05 2024-09-16T14:49:46.847 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.847 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.847 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.847 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 90/27 ECSubWrite(tid=310, reqid=osd.7.0:310, at_version=90'48, trim_to=0'0, roll_forward_to=89'46))", 2024-09-16T14:49:46.847 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:46.848 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.8323432309999994, 2024-09-16T14:49:46.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011963438999999999, 2024-09-16T14:49:46.848 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.848 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.848 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:46.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:46.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:46.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954255+0000", 2024-09-16T14:49:46.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2704000000000001e-05 2024-09-16T14:49:46.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954273+0000", 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7609999999999999e-05 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954283+0000", 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9539999999999999e-06 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954340+0000", 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6968000000000001e-05 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954352+0000", 2024-09-16T14:49:46.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1548e-05 2024-09-16T14:49:46.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.966177+0000", 2024-09-16T14:49:46.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011825293000000001 2024-09-16T14:49:46.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.966196+0000", 2024-09-16T14:49:46.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9636999999999999e-05 2024-09-16T14:49:46.851 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.851 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.852 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.852 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 91/90 ECSubWrite(tid=383, reqid=client.4500.0:825, at_version=91'144, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:46.852 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:46.852 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.8106218819999995, 2024-09-16T14:49:46.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079391419999999997, 2024-09-16T14:49:46.852 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.852 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.852 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976430+0000", 2024-09-16T14:49:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000476222 2024-09-16T14:49:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976441+0000", 2024-09-16T14:49:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1457000000000001e-05 2024-09-16T14:49:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976449+0000", 2024-09-16T14:49:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9719999999999992e-06 2024-09-16T14:49:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976957+0000", 2024-09-16T14:49:46.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050779500000000001 2024-09-16T14:49:46.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976967+0000", 2024-09-16T14:49:46.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0424e-05 2024-09-16T14:49:46.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.983864+0000", 2024-09-16T14:49:46.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068963369999999998 2024-09-16T14:49:46.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.983894+0000", 2024-09-16T14:49:46.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9784000000000001e-05 2024-09-16T14:49:46.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.856 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.856 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.856 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.17s5 91/90 ECSubWrite(tid=394, reqid=osd.3.0:391, at_version=0'0, trim_to=0'0, roll_forward_to=91'144))", 2024-09-16T14:49:46.856 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.006074+0000", 2024-09-16T14:49:46.856 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.7805021889999999, 2024-09-16T14:49:46.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010644525, 2024-09-16T14:49:46.856 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.856 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.857 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006074+0000", 2024-09-16T14:49:46.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006074+0000", 2024-09-16T14:49:46.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006073+0000", 2024-09-16T14:49:46.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:46.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006086+0000", 2024-09-16T14:49:46.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2862e-05 2024-09-16T14:49:46.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006102+0000", 2024-09-16T14:49:46.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6331e-05 2024-09-16T14:49:46.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006113+0000", 2024-09-16T14:49:46.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0518e-05 2024-09-16T14:49:46.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006171+0000", 2024-09-16T14:49:46.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8326999999999999e-05 2024-09-16T14:49:46.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006182+0000", 2024-09-16T14:49:46.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1134999999999999e-05 2024-09-16T14:49:46.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016684+0000", 2024-09-16T14:49:46.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010501664000000001 2024-09-16T14:49:46.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016719+0000", 2024-09-16T14:49:46.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4480000000000002e-05 2024-09-16T14:49:46.861 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.861 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.861 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.861 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.19s3 91/90 ECSubWrite(tid=102, reqid=client.4500.0:822, at_version=91'77, trim_to=0'0, roll_forward_to=91'75))", 2024-09-16T14:49:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.006461+0000", 2024-09-16T14:49:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.7801150099999994, 2024-09-16T14:49:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010302113, 2024-09-16T14:49:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006461+0000", 2024-09-16T14:49:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006461+0000", 2024-09-16T14:49:46.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006460+0000", 2024-09-16T14:49:46.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:46.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007931+0000", 2024-09-16T14:49:46.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014713079999999999 2024-09-16T14:49:46.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007954+0000", 2024-09-16T14:49:46.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2308999999999999e-05 2024-09-16T14:49:46.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007970+0000", 2024-09-16T14:49:46.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5880999999999999e-05 2024-09-16T14:49:46.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008058+0000", 2024-09-16T14:49:46.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8506999999999998e-05 2024-09-16T14:49:46.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008071+0000", 2024-09-16T14:49:46.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2434e-05 2024-09-16T14:49:46.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016742+0000", 2024-09-16T14:49:46.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086718799999999999 2024-09-16T14:49:46.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016763+0000", 2024-09-16T14:49:46.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0890000000000002e-05 2024-09-16T14:49:46.866 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.866 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.866 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.866 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.17s5 91/90 ECSubWrite(tid=395, reqid=osd.3.0:395, at_version=91'146, trim_to=0'0, roll_forward_to=91'144))", 2024-09-16T14:49:46.866 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.008031+0000", 2024-09-16T14:49:46.866 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.7785448210000006, 2024-09-16T14:49:46.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087059660000000007, 2024-09-16T14:49:46.866 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.866 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.866 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008031+0000", 2024-09-16T14:49:46.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008031+0000", 2024-09-16T14:49:46.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008030+0000", 2024-09-16T14:49:46.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:46.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008049+0000", 2024-09-16T14:49:46.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9230999999999999e-05 2024-09-16T14:49:46.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008059+0000", 2024-09-16T14:49:46.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7729999999999994e-06 2024-09-16T14:49:46.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008066+0000", 2024-09-16T14:49:46.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4849999999999997e-06 2024-09-16T14:49:46.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008113+0000", 2024-09-16T14:49:46.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7154999999999998e-05 2024-09-16T14:49:46.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008124+0000", 2024-09-16T14:49:46.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0933e-05 2024-09-16T14:49:46.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016723+0000", 2024-09-16T14:49:46.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0085989640000000006 2024-09-16T14:49:46.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016737+0000", 2024-09-16T14:49:46.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4635e-05 2024-09-16T14:49:46.870 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.870 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.871 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.871 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.14s2 91/90 ECSubWrite(tid=103, reqid=osd.1.0:102, at_version=0'0, trim_to=0'0, roll_forward_to=91'76))", 2024-09-16T14:49:46.871 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.009355+0000", 2024-09-16T14:49:46.871 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 8.7772207439999992, 2024-09-16T14:49:46.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074028089999999998, 2024-09-16T14:49:46.871 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.871 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.871 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009355+0000", 2024-09-16T14:49:46.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009355+0000", 2024-09-16T14:49:46.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009354+0000", 2024-09-16T14:49:46.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:46.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009370+0000", 2024-09-16T14:49:46.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6163000000000001e-05 2024-09-16T14:49:46.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009384+0000", 2024-09-16T14:49:46.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3655000000000001e-05 2024-09-16T14:49:46.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009393+0000", 2024-09-16T14:49:46.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1919999999999994e-06 2024-09-16T14:49:46.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009453+0000", 2024-09-16T14:49:46.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9626000000000001e-05 2024-09-16T14:49:46.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009467+0000", 2024-09-16T14:49:46.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4123e-05 2024-09-16T14:49:46.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016732+0000", 2024-09-16T14:49:46.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007265395 2024-09-16T14:49:46.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016758+0000", 2024-09-16T14:49:46.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6157e-05 2024-09-16T14:49:46.875 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.875 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.875 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.875 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.ds2 95/65 ECSubWrite(tid=335, reqid=osd.7.0:333, at_version=0'0, trim_to=0'0, roll_forward_to=95'81))", 2024-09-16T14:49:46.875 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:41.883865+0000", 2024-09-16T14:49:46.875 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 4.9027109700000002, 2024-09-16T14:49:46.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0072428129999999999, 2024-09-16T14:49:46.875 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:46.876 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:46.876 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:46.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:46.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883865+0000", 2024-09-16T14:49:46.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:46.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883865+0000", 2024-09-16T14:49:46.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:46.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:46.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883865+0000", 2024-09-16T14:49:46.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:46.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:46.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883874+0000", 2024-09-16T14:49:46.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7290000000000006e-06 2024-09-16T14:49:46.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:46.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883882+0000", 2024-09-16T14:49:46.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4859999999999997e-06 2024-09-16T14:49:46.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:46.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883889+0000", 2024-09-16T14:49:46.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3309999999999998e-06 2024-09-16T14:49:46.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:46.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883986+0000", 2024-09-16T14:49:46.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6576999999999994e-05 2024-09-16T14:49:46.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:46.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883994+0000", 2024-09-16T14:49:46.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0139999999999998e-06 2024-09-16T14:49:46.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:46.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.891085+0000", 2024-09-16T14:49:46.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070908899999999999 2024-09-16T14:49:46.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:46.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:46.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:46.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.891108+0000", 2024-09-16T14:49:46.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3181999999999999e-05 2024-09-16T14:49:46.879 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.879 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.879 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.879 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:46.880 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:46.880 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:46.881 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:49:46.882 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:46.880+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:46.981 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:46.980+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:47.000 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is set 2024-09-16T14:49:47.000 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T14:49:47.001 INFO:tasks.rados.rados.0.smithi040.stdout:377: done (1 left) 2024-09-16T14:49:47.001 INFO:tasks.rados.rados.0.smithi040.stdout:378: done (0 left) 2024-09-16T14:49:47.001 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 9 current snap is 36 2024-09-16T14:49:47.001 INFO:tasks.rados.rados.0.smithi040.stdout:379: seq_num 134 ranges {442368=704512,1146880=475136,1622016=573440,2195456=507904} 2024-09-16T14:49:47.006 INFO:tasks.rados.rados.0.smithi040.stdout:379: writing smithi04040151-9 from 442368 to 1146880 tid 1 2024-09-16T14:49:47.009 INFO:tasks.rados.rados.0.smithi040.stdout:379: writing smithi04040151-9 from 1146880 to 1622016 tid 2 2024-09-16T14:49:47.014 INFO:tasks.rados.rados.0.smithi040.stdout:379: writing smithi04040151-9 from 1622016 to 2195456 tid 3 2024-09-16T14:49:47.017 INFO:tasks.rados.rados.0.smithi040.stdout:379: writing smithi04040151-9 from 2195456 to 2703360 tid 4 2024-09-16T14:49:47.018 INFO:tasks.rados.rados.0.smithi040.stdout:380: read oid 21 snap 35 2024-09-16T14:49:47.018 INFO:tasks.rados.rados.0.smithi040.stdout:380: expect deleted 2024-09-16T14:49:47.018 INFO:tasks.rados.rados.0.smithi040.stdout:381: rmattr oid 20 current snap is 36 2024-09-16T14:49:47.018 INFO:tasks.rados.rados.0.smithi040.stdout:381: done (2 left) 2024-09-16T14:49:47.018 INFO:tasks.rados.rados.0.smithi040.stdout:382: snap_create 2024-09-16T14:49:47.023 INFO:tasks.rados.rados.0.smithi040.stdout:379: finishing write tid 1 to smithi04040151-9 2024-09-16T14:49:47.023 INFO:tasks.rados.rados.0.smithi040.stdout:379: finishing write tid 2 to smithi04040151-9 2024-09-16T14:49:47.024 INFO:tasks.rados.rados.0.smithi040.stdout:379: finishing write tid 3 to smithi04040151-9 2024-09-16T14:49:47.025 INFO:tasks.rados.rados.0.smithi040.stdout:379: finishing write tid 4 to smithi04040151-9 2024-09-16T14:49:47.025 INFO:tasks.rados.rados.0.smithi040.stdout:379: finishing write tid 5 to smithi04040151-9 2024-09-16T14:49:47.026 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T14:49:47.028 INFO:tasks.rados.rados.0.smithi040.stdout:379: finishing write tid 6 to smithi04040151-9 2024-09-16T14:49:47.028 INFO:tasks.rados.rados.0.smithi040.stdout:379: oid 9 updating version 0 to 164 2024-09-16T14:49:47.028 INFO:tasks.rados.rados.0.smithi040.stdout:379: oid 9 version 164 is already newer than 160 2024-09-16T14:49:47.028 INFO:tasks.rados.rados.0.smithi040.stdout:379: oid 9 version 164 is already newer than 161 2024-09-16T14:49:47.028 INFO:tasks.rados.rados.0.smithi040.stdout:379: oid 9 version 164 is already newer than 163 2024-09-16T14:49:47.028 INFO:tasks.rados.rados.0.smithi040.stdout:379: oid 9 version 164 is already newer than 162 2024-09-16T14:49:47.029 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 9 v 164 (ObjNum 134 snap 36 seq_num 134) dirty exists 2024-09-16T14:49:47.029 INFO:tasks.rados.rados.0.smithi040.stdout:379: left oid 9 (ObjNum 134 snap 36 seq_num 134) 2024-09-16T14:49:47.029 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:47.029 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:47.029 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:47.029 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:47.037 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:49:47.082 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:47.081+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:47.140 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:47.140 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:47.140 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:47.140 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:47.140 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:47.148 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:49:47.192 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:47.188+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:47.285 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:47.284+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:47.296 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:47.296 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:47.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:47.296 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:47.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.297 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:49:47.297 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:47.297 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 66.383102817999998, 2024-09-16T14:49:47.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079855010000000008, 2024-09-16T14:49:47.298 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.298 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.298 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:47.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:47.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909483+0000", 2024-09-16T14:49:47.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:47.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909521+0000", 2024-09-16T14:49:47.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7185999999999999e-05 2024-09-16T14:49:47.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909543+0000", 2024-09-16T14:49:47.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2802999999999998e-05 2024-09-16T14:49:47.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909551+0000", 2024-09-16T14:49:47.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5039999999999996e-06 2024-09-16T14:49:47.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909927+0000", 2024-09-16T14:49:47.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037637400000000001 2024-09-16T14:49:47.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:49:47.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2040000000000002e-06 2024-09-16T14:49:47.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917442+0000", 2024-09-16T14:49:47.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007508659 2024-09-16T14:49:47.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917469+0000", 2024-09-16T14:49:47.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7197999999999999e-05 2024-09-16T14:49:47.307 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.307 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.307 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.307 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:49:47.308 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:47.308 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 66.379030471999997, 2024-09-16T14:49:47.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067701159999999996, 2024-09-16T14:49:47.308 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.308 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.308 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:47.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:47.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:47.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:47.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913579+0000", 2024-09-16T14:49:47.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2829000000000001e-05 2024-09-16T14:49:47.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913591+0000", 2024-09-16T14:49:47.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2264e-05 2024-09-16T14:49:47.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913600+0000", 2024-09-16T14:49:47.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9959999999999999e-06 2024-09-16T14:49:47.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913648+0000", 2024-09-16T14:49:47.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7759000000000001e-05 2024-09-16T14:49:47.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913657+0000", 2024-09-16T14:49:47.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6709999999999994e-06 2024-09-16T14:49:47.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920296+0000", 2024-09-16T14:49:47.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066385120000000001 2024-09-16T14:49:47.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920326+0000", 2024-09-16T14:49:47.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0405000000000001e-05 2024-09-16T14:49:47.316 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.316 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.316 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.316 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 34/27 ECSubWrite(tid=74, reqid=client.4500.0:229, at_version=34'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:49:47.316 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 66.378214514000007, 2024-09-16T14:49:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060056090000000003, 2024-09-16T14:49:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:47.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:47.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:47.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914468+0000", 2024-09-16T14:49:47.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6398999999999996e-05 2024-09-16T14:49:47.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914483+0000", 2024-09-16T14:49:47.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5238e-05 2024-09-16T14:49:47.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.319 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914494+0000", 2024-09-16T14:49:47.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0192e-05 2024-09-16T14:49:47.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914622+0000", 2024-09-16T14:49:47.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012855599999999999 2024-09-16T14:49:47.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914635+0000", 2024-09-16T14:49:47.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2510000000000001e-05 2024-09-16T14:49:47.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920339+0000", 2024-09-16T14:49:47.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057041549999999998 2024-09-16T14:49:47.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920378+0000", 2024-09-16T14:49:47.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8918999999999998e-05 2024-09-16T14:49:47.321 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.321 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.321 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.322 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:49:47.322 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:47.322 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 66.376201738000006, 2024-09-16T14:49:47.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069011319999999999, 2024-09-16T14:49:47.322 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.322 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.322 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:47.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:47.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:47.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:49:47.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.324 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916410+0000", 2024-09-16T14:49:47.324 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5343000000000001e-05 2024-09-16T14:49:47.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.324 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.324 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916418+0000", 2024-09-16T14:49:47.325 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2360000000000004e-06 2024-09-16T14:49:47.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.325 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.325 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.325 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916428+0000", 2024-09-16T14:49:47.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6779999999999995e-06 2024-09-16T14:49:47.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.326 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.326 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916492+0000", 2024-09-16T14:49:47.326 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4219000000000003e-05 2024-09-16T14:49:47.326 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.327 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916502+0000", 2024-09-16T14:49:47.327 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6690000000000001e-06 2024-09-16T14:49:47.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.327 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.327 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923254+0000", 2024-09-16T14:49:47.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067518070000000003 2024-09-16T14:49:47.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.328 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.328 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.328 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923286+0000", 2024-09-16T14:49:47.328 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2367999999999999e-05 2024-09-16T14:49:47.329 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.329 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.329 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.329 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.329 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s7 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:47.330 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:47.330 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 63.370897354, 2024-09-16T14:49:47.330 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006972692, 2024-09-16T14:49:47.330 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.330 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.330 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.330 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:47.331 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.331 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.331 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.331 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:47.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.332 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.332 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.332 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:47.332 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:47.332 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.333 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.333 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921709+0000", 2024-09-16T14:49:47.333 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0411000000000002e-05 2024-09-16T14:49:47.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.333 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.334 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.334 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921719+0000", 2024-09-16T14:49:47.334 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2650000000000002e-06 2024-09-16T14:49:47.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.335 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921726+0000", 2024-09-16T14:49:47.336 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8770000000000004e-06 2024-09-16T14:49:47.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.336 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.336 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.336 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921773+0000", 2024-09-16T14:49:47.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7435999999999998e-05 2024-09-16T14:49:47.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.337 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.337 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.337 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921782+0000", 2024-09-16T14:49:47.337 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6039999999999993e-06 2024-09-16T14:49:47.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.338 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928635+0000", 2024-09-16T14:49:47.338 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068533129999999998 2024-09-16T14:49:47.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.338 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.338 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.339 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928662+0000", 2024-09-16T14:49:47.339 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7089000000000001e-05 2024-09-16T14:49:47.339 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.339 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.339 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.339 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.340 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:47.340 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:47.340 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 63.370400865999997, 2024-09-16T14:49:47.340 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064783530000000001, 2024-09-16T14:49:47.340 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.340 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.341 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.341 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.341 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:47.341 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.341 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:47.342 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.342 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.342 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.342 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922185+0000", 2024-09-16T14:49:47.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:47.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.343 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.343 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.343 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922205+0000", 2024-09-16T14:49:47.343 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9871e-05 2024-09-16T14:49:47.343 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.344 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.344 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922219+0000", 2024-09-16T14:49:47.344 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4266e-05 2024-09-16T14:49:47.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922229+0000", 2024-09-16T14:49:47.345 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5619999999999993e-06 2024-09-16T14:49:47.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.345 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.345 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.345 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922270+0000", 2024-09-16T14:49:47.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0917000000000001e-05 2024-09-16T14:49:47.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.346 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.346 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922278+0000", 2024-09-16T14:49:47.346 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3890000000000005e-06 2024-09-16T14:49:47.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.346 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928630+0000", 2024-09-16T14:49:47.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063524439999999996 2024-09-16T14:49:47.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.347 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.347 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928664+0000", 2024-09-16T14:49:47.347 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3651e-05 2024-09-16T14:49:47.347 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.347 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.347 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.347 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.348 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s5 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:49:47.348 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:47.348 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 63.368952462000003, 2024-09-16T14:49:47.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069615850000000002, 2024-09-16T14:49:47.348 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.348 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.348 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.348 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.348 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.348 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:47.348 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:47.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.349 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923633+0000", 2024-09-16T14:49:47.349 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:47.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.349 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.349 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923644+0000", 2024-09-16T14:49:47.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.08e-05 2024-09-16T14:49:47.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923657+0000", 2024-09-16T14:49:47.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2836999999999999e-05 2024-09-16T14:49:47.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.350 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.350 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.350 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923664+0000", 2024-09-16T14:49:47.350 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1470000000000004e-06 2024-09-16T14:49:47.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923885+0000", 2024-09-16T14:49:47.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022155800000000001 2024-09-16T14:49:47.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.351 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.351 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.351 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923894+0000", 2024-09-16T14:49:47.351 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9260000000000006e-06 2024-09-16T14:49:47.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930571+0000", 2024-09-16T14:49:47.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066770420000000002 2024-09-16T14:49:47.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.352 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.352 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.352 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930596+0000", 2024-09-16T14:49:47.352 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4383e-05 2024-09-16T14:49:47.352 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.352 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.353 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.353 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as3 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:47.353 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:47.353 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 63.368206225999998, 2024-09-16T14:49:47.353 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062471519999999997, 2024-09-16T14:49:47.353 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.353 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.353 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.353 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:47.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:47.354 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.354 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.354 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.354 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:47.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:47.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924405+0000", 2024-09-16T14:49:47.355 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5147000000000002e-05 2024-09-16T14:49:47.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.355 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.355 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.355 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924414+0000", 2024-09-16T14:49:47.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4810000000000006e-06 2024-09-16T14:49:47.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924421+0000", 2024-09-16T14:49:47.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0899999999999999e-06 2024-09-16T14:49:47.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.356 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.356 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924468+0000", 2024-09-16T14:49:47.356 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.727e-05 2024-09-16T14:49:47.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924477+0000", 2024-09-16T14:49:47.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7360000000000007e-06 2024-09-16T14:49:47.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.357 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.357 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930601+0000", 2024-09-16T14:49:47.357 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061242529999999996 2024-09-16T14:49:47.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.358 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.358 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.358 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930628+0000", 2024-09-16T14:49:47.358 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6517e-05 2024-09-16T14:49:47.358 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.358 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.358 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.359 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.359 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:47.359 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:47.359 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 63.367956022000001, 2024-09-16T14:49:47.359 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006019328, 2024-09-16T14:49:47.359 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.359 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.360 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.360 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.360 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:47.360 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.360 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:47.361 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.361 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.361 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.361 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924630+0000", 2024-09-16T14:49:47.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:47.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.362 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.362 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.362 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924650+0000", 2024-09-16T14:49:47.362 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0024e-05 2024-09-16T14:49:47.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.363 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924660+0000", 2024-09-16T14:49:47.363 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9219999999999992e-06 2024-09-16T14:49:47.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.363 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.363 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924668+0000", 2024-09-16T14:49:47.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6820000000000003e-06 2024-09-16T14:49:47.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.364 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.364 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.364 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924711+0000", 2024-09-16T14:49:47.364 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3009000000000001e-05 2024-09-16T14:49:47.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.365 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.365 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924720+0000", 2024-09-16T14:49:47.365 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9290000000000004e-06 2024-09-16T14:49:47.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.365 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930634+0000", 2024-09-16T14:49:47.366 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059144810000000001 2024-09-16T14:49:47.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.366 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.366 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.366 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930650+0000", 2024-09-16T14:49:47.367 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5688999999999998e-05 2024-09-16T14:49:47.367 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.367 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.367 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.367 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.367 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.8s6 51/27 ECSubWrite(tid=173, reqid=osd.11.0:173, at_version=51'54, trim_to=0'0, roll_forward_to=50'50))", 2024-09-16T14:49:47.368 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.881283+0000", 2024-09-16T14:49:47.368 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.411303934000003, 2024-09-16T14:49:47.368 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057129700000000004, 2024-09-16T14:49:47.368 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.368 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.368 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.368 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881283+0000", 2024-09-16T14:49:47.369 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.369 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.369 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.369 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881283+0000", 2024-09-16T14:49:47.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.370 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.370 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.370 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881282+0000", 2024-09-16T14:49:47.370 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:47.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.371 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881333+0000", 2024-09-16T14:49:47.371 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0413e-05 2024-09-16T14:49:47.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.371 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.371 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881345+0000", 2024-09-16T14:49:47.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1933000000000001e-05 2024-09-16T14:49:47.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.372 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.372 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.372 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881353+0000", 2024-09-16T14:49:47.372 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0469999999999994e-06 2024-09-16T14:49:47.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.373 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.373 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882512+0000", 2024-09-16T14:49:47.373 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001159806 2024-09-16T14:49:47.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.374 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882521+0000", 2024-09-16T14:49:47.374 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9639999999999992e-06 2024-09-16T14:49:47.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.374 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.374 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.886958+0000", 2024-09-16T14:49:47.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0044365170000000001 2024-09-16T14:49:47.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.375 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.375 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.375 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.886996+0000", 2024-09-16T14:49:47.375 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7808999999999997e-05 2024-09-16T14:49:47.376 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.376 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.376 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.376 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.376 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:47.376 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:47.377 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 39.359586903, 2024-09-16T14:49:47.377 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062137989999999999, 2024-09-16T14:49:47.377 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.377 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.377 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.377 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:47.378 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.378 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.378 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.378 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:47.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.379 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.379 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.379 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.932999+0000", 2024-09-16T14:49:47.379 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:47.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933022+0000", 2024-09-16T14:49:47.380 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3518000000000001e-05 2024-09-16T14:49:47.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.380 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.380 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.380 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933035+0000", 2024-09-16T14:49:47.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2565000000000001e-05 2024-09-16T14:49:47.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933044+0000", 2024-09-16T14:49:47.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3009999999999995e-06 2024-09-16T14:49:47.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.381 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.381 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.381 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934016+0000", 2024-09-16T14:49:47.381 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00097159700000000004 2024-09-16T14:49:47.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934022+0000", 2024-09-16T14:49:47.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4869999999999999e-06 2024-09-16T14:49:47.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.382 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939202+0000", 2024-09-16T14:49:47.382 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051795820000000003 2024-09-16T14:49:47.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.382 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.382 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.383 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939214+0000", 2024-09-16T14:49:47.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1663e-05 2024-09-16T14:49:47.383 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.383 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.383 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.383 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.383 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.cs3 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:47.383 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:47.383 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 39.359120367999999, 2024-09-16T14:49:47.383 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057306709999999997, 2024-09-16T14:49:47.384 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.384 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.384 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:47.384 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.384 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.384 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.384 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:47.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933465+0000", 2024-09-16T14:49:47.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:47.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.385 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.385 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.385 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933489+0000", 2024-09-16T14:49:47.385 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3940000000000001e-05 2024-09-16T14:49:47.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933501+0000", 2024-09-16T14:49:47.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1932e-05 2024-09-16T14:49:47.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.386 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.386 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.386 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933510+0000", 2024-09-16T14:49:47.386 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5820000000000002e-06 2024-09-16T14:49:47.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936680+0000", 2024-09-16T14:49:47.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003169893 2024-09-16T14:49:47.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.387 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.387 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936689+0000", 2024-09-16T14:49:47.387 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5810000000000003e-06 2024-09-16T14:49:47.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.387 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939176+0000", 2024-09-16T14:49:47.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002486238 2024-09-16T14:49:47.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.388 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.388 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.388 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939197+0000", 2024-09-16T14:49:47.388 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1325e-05 2024-09-16T14:49:47.388 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.388 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.388 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.389 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 62/27 ECSubWrite(tid=244, reqid=osd.7.0:244, at_version=62'27, trim_to=0'0, roll_forward_to=61'25))", 2024-09-16T14:49:47.389 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:47.389 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.349757361000002, 2024-09-16T14:49:47.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007111081, 2024-09-16T14:49:47.389 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.389 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.389 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.389 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.389 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.389 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:47.389 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.390 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:47.390 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.390 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.390 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:47.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:47.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.391 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.391 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.391 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942855+0000", 2024-09-16T14:49:47.391 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5999999999999998e-05 2024-09-16T14:49:47.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.392 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.392 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942867+0000", 2024-09-16T14:49:47.392 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2481e-05 2024-09-16T14:49:47.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.392 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.393 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942879+0000", 2024-09-16T14:49:47.393 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1838000000000001e-05 2024-09-16T14:49:47.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.393 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.393 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942938+0000", 2024-09-16T14:49:47.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8282000000000002e-05 2024-09-16T14:49:47.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.394 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.394 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.394 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942948+0000", 2024-09-16T14:49:47.394 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0497e-05 2024-09-16T14:49:47.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.395 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949905+0000", 2024-09-16T14:49:47.395 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069572940000000002 2024-09-16T14:49:47.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.395 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.395 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.396 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949940+0000", 2024-09-16T14:49:47.396 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4965999999999997e-05 2024-09-16T14:49:47.396 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.396 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.396 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.396 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.397 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 62/27 ECSubWrite(tid=234, reqid=osd.11.0:234, at_version=62'39, trim_to=0'0, roll_forward_to=61'37))", 2024-09-16T14:49:47.397 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:47.397 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 38.349323877000003, 2024-09-16T14:49:47.397 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066952440000000004, 2024-09-16T14:49:47.397 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.397 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.397 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.398 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:47.398 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.398 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.398 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:47.399 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.399 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.399 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.399 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943262+0000", 2024-09-16T14:49:47.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:47.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.400 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.400 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.400 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943285+0000", 2024-09-16T14:49:47.400 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2650000000000002e-05 2024-09-16T14:49:47.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943301+0000", 2024-09-16T14:49:47.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5398000000000001e-05 2024-09-16T14:49:47.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.401 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.401 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.401 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943309+0000", 2024-09-16T14:49:47.401 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9269999999999994e-06 2024-09-16T14:49:47.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943373+0000", 2024-09-16T14:49:47.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3194999999999994e-05 2024-09-16T14:49:47.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.402 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943383+0000", 2024-09-16T14:49:47.402 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0778000000000001e-05 2024-09-16T14:49:47.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.402 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.402 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949934+0000", 2024-09-16T14:49:47.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065509790000000002 2024-09-16T14:49:47.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.403 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.403 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.403 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949958+0000", 2024-09-16T14:49:47.403 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3598999999999999e-05 2024-09-16T14:49:47.403 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.403 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.403 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.404 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 76/27 ECSubWrite(tid=303, reqid=client.4500.0:699, at_version=76'98, trim_to=0'0, roll_forward_to=76'95))", 2024-09-16T14:49:47.404 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:47.404 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.382553332000001, 2024-09-16T14:49:47.404 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006028032, 2024-09-16T14:49:47.404 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.404 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.404 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.404 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:47.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.405 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.405 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:47.405 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.405 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910032+0000", 2024-09-16T14:49:47.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:47.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910053+0000", 2024-09-16T14:49:47.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0665e-05 2024-09-16T14:49:47.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.406 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.406 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.406 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910067+0000", 2024-09-16T14:49:47.406 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4416000000000001e-05 2024-09-16T14:49:47.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910077+0000", 2024-09-16T14:49:47.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4509999999999998e-06 2024-09-16T14:49:47.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.407 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910367+0000", 2024-09-16T14:49:47.407 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00029006799999999999 2024-09-16T14:49:47.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.407 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.407 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910378+0000", 2024-09-16T14:49:47.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0628000000000001e-05 2024-09-16T14:49:47.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.916036+0000", 2024-09-16T14:49:47.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0056580270000000004 2024-09-16T14:49:47.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.408 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.408 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.408 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.916061+0000", 2024-09-16T14:49:47.408 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5658000000000001e-05 2024-09-16T14:49:47.409 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.409 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.409 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.409 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.409 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 77/27 ECSubWrite(tid=277, reqid=osd.11.0:277, at_version=77'54, trim_to=0'0, roll_forward_to=75'52))", 2024-09-16T14:49:47.409 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:47.409 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.382065963999999, 2024-09-16T14:49:47.409 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064553950000000001, 2024-09-16T14:49:47.409 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.409 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.410 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:47.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.410 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.410 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.410 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:47.410 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910520+0000", 2024-09-16T14:49:47.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:49:47.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.411 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.411 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910554+0000", 2024-09-16T14:49:47.411 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3111999999999997e-05 2024-09-16T14:49:47.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.411 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910578+0000", 2024-09-16T14:49:47.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4757999999999999e-05 2024-09-16T14:49:47.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.412 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910590+0000", 2024-09-16T14:49:47.412 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2156e-05 2024-09-16T14:49:47.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.412 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910646+0000", 2024-09-16T14:49:47.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5979000000000001e-05 2024-09-16T14:49:47.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.413 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910656+0000", 2024-09-16T14:49:47.413 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9410000000000002e-06 2024-09-16T14:49:47.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.413 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.413 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916943+0000", 2024-09-16T14:49:47.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062870260000000002 2024-09-16T14:49:47.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.414 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.414 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916976+0000", 2024-09-16T14:49:47.414 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2641999999999997e-05 2024-09-16T14:49:47.414 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.414 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.414 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.414 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.415 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 91/27 ECSubWrite(tid=316, reqid=osd.7.0:315, at_version=0'0, trim_to=0'0, roll_forward_to=91'50))", 2024-09-16T14:49:47.415 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.963325+0000", 2024-09-16T14:49:47.415 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.3292618730000001, 2024-09-16T14:49:47.415 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060303650000000002, 2024-09-16T14:49:47.415 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.415 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.415 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.415 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.416 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963325+0000", 2024-09-16T14:49:47.416 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.416 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.416 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963325+0000", 2024-09-16T14:49:47.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963324+0000", 2024-09-16T14:49:47.417 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:47.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.417 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.417 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.417 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963338+0000", 2024-09-16T14:49:47.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3813e-05 2024-09-16T14:49:47.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963350+0000", 2024-09-16T14:49:47.418 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2512e-05 2024-09-16T14:49:47.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.418 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.418 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.418 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963360+0000", 2024-09-16T14:49:47.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0141e-05 2024-09-16T14:49:47.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963425+0000", 2024-09-16T14:49:47.419 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4579999999999995e-05 2024-09-16T14:49:47.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.419 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.419 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.419 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963437+0000", 2024-09-16T14:49:47.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1780000000000001e-05 2024-09-16T14:49:47.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.420 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.969335+0000", 2024-09-16T14:49:47.420 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0058980500000000002 2024-09-16T14:49:47.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.420 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.420 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.421 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.969355+0000", 2024-09-16T14:49:47.421 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0489e-05 2024-09-16T14:49:47.421 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.421 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.421 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.422 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 91/90 ECSubWrite(tid=383, reqid=client.4500.0:825, at_version=91'144, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:47.422 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.976070+0000", 2024-09-16T14:49:47.422 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.316516665, 2024-09-16T14:49:47.422 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066451510000000002, 2024-09-16T14:49:47.422 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.422 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.422 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.423 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976070+0000", 2024-09-16T14:49:47.423 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.423 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.423 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976070+0000", 2024-09-16T14:49:47.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.424 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.424 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.424 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976069+0000", 2024-09-16T14:49:47.424 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:47.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.425 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.425 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976953+0000", 2024-09-16T14:49:47.425 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00088349499999999998 2024-09-16T14:49:47.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.425 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977007+0000", 2024-09-16T14:49:47.426 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4676000000000003e-05 2024-09-16T14:49:47.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.426 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.426 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.426 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977083+0000", 2024-09-16T14:49:47.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5831e-05 2024-09-16T14:49:47.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.427 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.427 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.427 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977120+0000", 2024-09-16T14:49:47.427 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6887000000000001e-05 2024-09-16T14:49:47.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.428 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977129+0000", 2024-09-16T14:49:47.428 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9120000000000004e-06 2024-09-16T14:49:47.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.428 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.428 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.982689+0000", 2024-09-16T14:49:47.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055597590000000001 2024-09-16T14:49:47.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.429 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.429 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.429 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.982715+0000", 2024-09-16T14:49:47.429 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6478999999999998e-05 2024-09-16T14:49:47.430 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.430 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.430 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.430 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.430 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.19s7 91/90 ECSubWrite(tid=102, reqid=client.4500.0:822, at_version=91'77, trim_to=0'0, roll_forward_to=91'75))", 2024-09-16T14:49:47.431 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.007185+0000", 2024-09-16T14:49:47.431 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.2854019309999991, 2024-09-16T14:49:47.431 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068803600000000003, 2024-09-16T14:49:47.431 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.431 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.431 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.431 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007185+0000", 2024-09-16T14:49:47.432 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.432 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.432 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.432 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007185+0000", 2024-09-16T14:49:47.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.433 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.433 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.433 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007183+0000", 2024-09-16T14:49:47.433 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:47.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.434 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008457+0000", 2024-09-16T14:49:47.434 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001274025 2024-09-16T14:49:47.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.434 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.434 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008473+0000", 2024-09-16T14:49:47.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5605999999999999e-05 2024-09-16T14:49:47.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.435 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.435 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.435 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008483+0000", 2024-09-16T14:49:47.435 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8430000000000005e-06 2024-09-16T14:49:47.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008541+0000", 2024-09-16T14:49:47.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7911999999999997e-05 2024-09-16T14:49:47.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.436 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.436 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.436 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008551+0000", 2024-09-16T14:49:47.436 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0353999999999999e-05 2024-09-16T14:49:47.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.014044+0000", 2024-09-16T14:49:47.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054927439999999999 2024-09-16T14:49:47.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.437 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.437 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.437 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.014065+0000", 2024-09-16T14:49:47.437 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1124999999999998e-05 2024-09-16T14:49:47.437 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.437 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.438 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.438 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.438 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.17s4 91/90 ECSubWrite(tid=395, reqid=osd.3.0:395, at_version=91'146, trim_to=0'0, roll_forward_to=91'144))", 2024-09-16T14:49:47.438 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:47.438 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.2831323129999994, 2024-09-16T14:49:47.438 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0076634049999999999, 2024-09-16T14:49:47.438 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.439 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.439 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:47.439 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.439 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.439 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.439 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:47.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:47.440 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:47.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.440 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.440 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.440 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009478+0000", 2024-09-16T14:49:47.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4235000000000001e-05 2024-09-16T14:49:47.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009486+0000", 2024-09-16T14:49:47.441 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7940000000000003e-06 2024-09-16T14:49:47.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.441 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.441 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.441 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009492+0000", 2024-09-16T14:49:47.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7980000000000002e-06 2024-09-16T14:49:47.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010420+0000", 2024-09-16T14:49:47.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00092807100000000004 2024-09-16T14:49:47.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.442 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.442 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.442 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010429+0000", 2024-09-16T14:49:47.442 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8389999999999996e-06 2024-09-16T14:49:47.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017094+0000", 2024-09-16T14:49:47.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066650879999999996 2024-09-16T14:49:47.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.443 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.443 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.443 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017118+0000", 2024-09-16T14:49:47.443 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3941e-05 2024-09-16T14:49:47.444 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.444 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.444 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.444 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.444 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.444 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:47.446 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:49:47.446 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:47.445+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:47.546 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:47.545+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:47.591 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:47.591 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:47.591 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:47.599 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:49:47.648 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:47.647+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:47.702 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:47.702 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:47.702 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:47.702 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:47.710 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:49:47.749 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:47.748+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:47.814 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:47.814 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:47.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:47.814 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:47.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.814 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:229 3.es0 3:7ecfd044:::smithi04040151-35:head [copy-from ver 4] snapc 4=[4,2] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:49:47.814 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:47.814 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 66.907629165000003, 2024-09-16T14:49:47.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016506461, 2024-09-16T14:49:47.815 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.815 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:47.815 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.815 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.815 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.815 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 229 2024-09-16T14:49:47.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.815 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:47.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:47.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904168+0000", 2024-09-16T14:49:47.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:47.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904187+0000", 2024-09-16T14:49:47.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9449999999999998e-05 2024-09-16T14:49:47.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904206+0000", 2024-09-16T14:49:47.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8698e-05 2024-09-16T14:49:47.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904215+0000", 2024-09-16T14:49:47.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:49:47.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904276+0000", 2024-09-16T14:49:47.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1130999999999995e-05 2024-09-16T14:49:47.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:47.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904284+0000", 2024-09-16T14:49:47.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0609999999999996e-06 2024-09-16T14:49:47.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906272+0000", 2024-09-16T14:49:47.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019884210000000002 2024-09-16T14:49:47.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906391+0000", 2024-09-16T14:49:47.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000118843 2024-09-16T14:49:47.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.912595+0000", 2024-09-16T14:49:47.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062038190000000002 2024-09-16T14:49:47.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917584+0000", 2024-09-16T14:49:47.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049889069999999999 2024-09-16T14:49:47.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:47.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920592+0000", 2024-09-16T14:49:47.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030081639999999998 2024-09-16T14:49:47.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920676+0000", 2024-09-16T14:49:47.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3974999999999995e-05 2024-09-16T14:49:47.821 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.821 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.821 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.821 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:298 3.es0 3:7ecfd044:::smithi04040151-35:head [append 0~720896] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:47.821 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:47.821 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.882965349999999, 2024-09-16T14:49:47.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014187808, 2024-09-16T14:49:47.821 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.822 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:47.822 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.822 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.822 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.822 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 298 2024-09-16T14:49:47.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.822 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.822 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.822 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:47.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:47.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928832+0000", 2024-09-16T14:49:47.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:47.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929746+0000", 2024-09-16T14:49:47.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00091434199999999995 2024-09-16T14:49:47.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929760+0000", 2024-09-16T14:49:47.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3519e-05 2024-09-16T14:49:47.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929767+0000", 2024-09-16T14:49:47.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3180000000000001e-06 2024-09-16T14:49:47.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929849+0000", 2024-09-16T14:49:47.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2020000000000004e-05 2024-09-16T14:49:47.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:47.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929854+0000", 2024-09-16T14:49:47.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1209999999999998e-06 2024-09-16T14:49:47.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931714+0000", 2024-09-16T14:49:47.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001859989 2024-09-16T14:49:47.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931845+0000", 2024-09-16T14:49:47.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130901 2024-09-16T14:49:47.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933089+0000", 2024-09-16T14:49:47.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001243535 2024-09-16T14:49:47.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936370+0000", 2024-09-16T14:49:47.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032817660000000002 2024-09-16T14:49:47.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:47.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942945+0000", 2024-09-16T14:49:47.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065741560000000003 2024-09-16T14:49:47.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.828 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.828 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943021+0000", 2024-09-16T14:49:47.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6734000000000001e-05 2024-09-16T14:49:47.829 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.829 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.829 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.829 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:299 3.es0 3:7ecfd044:::smithi04040151-35:head [append 0~376832] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:47.829 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:47.829 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.880675549999999, 2024-09-16T14:49:47.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012149627, 2024-09-16T14:49:47.829 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.830 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:47.830 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.830 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.830 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.830 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 299 2024-09-16T14:49:47.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.830 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:47.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:47.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931122+0000", 2024-09-16T14:49:47.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:47.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931722+0000", 2024-09-16T14:49:47.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059944100000000004 2024-09-16T14:49:47.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931732+0000", 2024-09-16T14:49:47.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9369999999999999e-06 2024-09-16T14:49:47.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931739+0000", 2024-09-16T14:49:47.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2819999999999999e-06 2024-09-16T14:49:47.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933835+0000", 2024-09-16T14:49:47.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020961140000000001 2024-09-16T14:49:47.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933929+0000", 2024-09-16T14:49:47.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3683000000000004e-05 2024-09-16T14:49:47.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.934917+0000", 2024-09-16T14:49:47.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00098803200000000002 2024-09-16T14:49:47.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.939558+0000", 2024-09-16T14:49:47.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00464175 2024-09-16T14:49:47.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:47.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943205+0000", 2024-09-16T14:49:47.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036465820000000002 2024-09-16T14:49:47.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943273+0000", 2024-09-16T14:49:47.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7917999999999996e-05 2024-09-16T14:49:47.835 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.835 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.835 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.836 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:300 3.es0 3:7ecfd044:::smithi04040151-35:head [setxattr _header (56) in=7b] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:47.836 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:47.836 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.880049456000002, 2024-09-16T14:49:47.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011740911, 2024-09-16T14:49:47.836 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.836 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:47.836 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.836 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.836 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.836 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 300 2024-09-16T14:49:47.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.836 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:47.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:47.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:47.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:47.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931753+0000", 2024-09-16T14:49:47.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0380000000000001e-06 2024-09-16T14:49:47.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931759+0000", 2024-09-16T14:49:47.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3430000000000001e-06 2024-09-16T14:49:47.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931765+0000", 2024-09-16T14:49:47.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-16T14:49:47.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.935623+0000", 2024-09-16T14:49:47.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038583049999999998 2024-09-16T14:49:47.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.935717+0000", 2024-09-16T14:49:47.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4587999999999996e-05 2024-09-16T14:49:47.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936150+0000", 2024-09-16T14:49:47.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043270599999999999 2024-09-16T14:49:47.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.939576+0000", 2024-09-16T14:49:47.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034257620000000002 2024-09-16T14:49:47.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:47.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943399+0000", 2024-09-16T14:49:47.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003823145 2024-09-16T14:49:47.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943490+0000", 2024-09-16T14:49:47.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1241000000000003e-05 2024-09-16T14:49:47.842 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.842 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.842 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.842 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:301 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:47.842 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:47.842 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.880028156000002, 2024-09-16T14:49:47.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014215955000000001, 2024-09-16T14:49:47.842 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.842 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:47.842 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.842 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.843 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.843 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 301 2024-09-16T14:49:47.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.843 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:47.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:47.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931770+0000", 2024-09-16T14:49:47.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:47.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931775+0000", 2024-09-16T14:49:47.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4530000000000004e-06 2024-09-16T14:49:47.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931782+0000", 2024-09-16T14:49:47.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0550000000000002e-06 2024-09-16T14:49:47.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931787+0000", 2024-09-16T14:49:47.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2310000000000004e-06 2024-09-16T14:49:47.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936401+0000", 2024-09-16T14:49:47.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046143419999999996 2024-09-16T14:49:47.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:47.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936482+0000", 2024-09-16T14:49:47.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0407000000000005e-05 2024-09-16T14:49:47.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943851+0000", 2024-09-16T14:49:47.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073695000000000002 2024-09-16T14:49:47.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943951+0000", 2024-09-16T14:49:47.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9327000000000006e-05 2024-09-16T14:49:47.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.945987+0000", 2024-09-16T14:49:47.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002036057 2024-09-16T14:49:47.847 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.847 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.847 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.847 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:369 3.8s0 3:1f96bdfc:::smithi04040151-37:head [append 0~475136] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:47.847 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:47.847 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.854086162999998, 2024-09-16T14:49:47.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010979078999999999, 2024-09-16T14:49:47.848 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.848 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:47.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.848 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.848 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 369 2024-09-16T14:49:47.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.848 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:47.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:47.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957711+0000", 2024-09-16T14:49:47.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:47.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958323+0000", 2024-09-16T14:49:47.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000612439 2024-09-16T14:49:47.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958339+0000", 2024-09-16T14:49:47.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5985999999999999e-05 2024-09-16T14:49:47.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.850 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.850 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958348+0000", 2024-09-16T14:49:47.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8699999999999998e-06 2024-09-16T14:49:47.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958393+0000", 2024-09-16T14:49:47.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4981000000000002e-05 2024-09-16T14:49:47.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958478+0000", 2024-09-16T14:49:47.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4845999999999994e-05 2024-09-16T14:49:47.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.959659+0000", 2024-09-16T14:49:47.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011808999999999999 2024-09-16T14:49:47.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962687+0000", 2024-09-16T14:49:47.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030278330000000002 2024-09-16T14:49:47.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:47.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968651+0000", 2024-09-16T14:49:47.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005964764 2024-09-16T14:49:47.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968692+0000", 2024-09-16T14:49:47.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0374999999999999e-05 2024-09-16T14:49:47.853 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.853 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.853 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.853 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:398 3.as0 3:572aff57:::smithi04040151-20:head [append 0~507904] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:47.853 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:47.853 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 53.932721397000002, 2024-09-16T14:49:47.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010447988, 2024-09-16T14:49:47.854 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.854 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:47.854 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.854 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.854 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.854 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 398 2024-09-16T14:49:47.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.854 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:47.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:47.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.855 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879076+0000", 2024-09-16T14:49:47.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:47.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.855 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881851+0000", 2024-09-16T14:49:47.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0027753420000000001 2024-09-16T14:49:47.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881863+0000", 2024-09-16T14:49:47.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2203e-05 2024-09-16T14:49:47.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881871+0000", 2024-09-16T14:49:47.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4059999999999999e-06 2024-09-16T14:49:47.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881924+0000", 2024-09-16T14:49:47.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2932000000000001e-05 2024-09-16T14:49:47.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882118+0000", 2024-09-16T14:49:47.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019413500000000001 2024-09-16T14:49:47.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884114+0000", 2024-09-16T14:49:47.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001996154 2024-09-16T14:49:47.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889295+0000", 2024-09-16T14:49:47.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051809899999999999 2024-09-16T14:49:47.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:47.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889480+0000", 2024-09-16T14:49:47.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018519700000000001 2024-09-16T14:49:47.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889525+0000", 2024-09-16T14:49:47.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5399999999999999e-05 2024-09-16T14:49:47.859 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.859 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.859 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.859 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:400 3.as0 3:572aff57:::smithi04040151-20:head [append 0~262144] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:47.859 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:47.860 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 53.928212367, 2024-09-16T14:49:47.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010619327, 2024-09-16T14:49:47.860 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.860 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:47.860 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.860 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.860 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.860 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 400 2024-09-16T14:49:47.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.860 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:47.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:47.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883585+0000", 2024-09-16T14:49:47.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:47.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883868+0000", 2024-09-16T14:49:47.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028385599999999997 2024-09-16T14:49:47.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883880+0000", 2024-09-16T14:49:47.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1181e-05 2024-09-16T14:49:47.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883888+0000", 2024-09-16T14:49:47.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.926e-06 2024-09-16T14:49:47.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887739+0000", 2024-09-16T14:49:47.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003851658 2024-09-16T14:49:47.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887849+0000", 2024-09-16T14:49:47.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109616 2024-09-16T14:49:47.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.888684+0000", 2024-09-16T14:49:47.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083518000000000004 2024-09-16T14:49:47.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894010+0000", 2024-09-16T14:49:47.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053260670000000003 2024-09-16T14:49:47.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:47.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894143+0000", 2024-09-16T14:49:47.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000133077 2024-09-16T14:49:47.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894206+0000", 2024-09-16T14:49:47.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2658999999999998e-05 2024-09-16T14:49:47.865 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.865 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.865 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.865 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:401 3.as0 3:572aff57:::smithi04040151-20:head [setxattr _header (57) in=7b] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:47.865 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:47.865 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 53.927902443000001, 2024-09-16T14:49:47.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010524774000000001, 2024-09-16T14:49:47.865 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.866 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:47.866 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.866 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.866 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.866 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 401 2024-09-16T14:49:47.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.866 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:47.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:47.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:47.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:47.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883900+0000", 2024-09-16T14:49:47.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1400000000000002e-06 2024-09-16T14:49:47.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883908+0000", 2024-09-16T14:49:47.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2689999999999999e-06 2024-09-16T14:49:47.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883913+0000", 2024-09-16T14:49:47.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-16T14:49:47.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889560+0000", 2024-09-16T14:49:47.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005647165 2024-09-16T14:49:47.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889627+0000", 2024-09-16T14:49:47.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6951000000000003e-05 2024-09-16T14:49:47.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889898+0000", 2024-09-16T14:49:47.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270709 2024-09-16T14:49:47.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894012+0000", 2024-09-16T14:49:47.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041137810000000004 2024-09-16T14:49:47.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:47.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894344+0000", 2024-09-16T14:49:47.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000332121 2024-09-16T14:49:47.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894421+0000", 2024-09-16T14:49:47.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7406999999999999e-05 2024-09-16T14:49:47.871 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.871 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.871 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.871 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:402 3.as0 3:572aff57:::smithi04040151-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:47.871 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:47.871 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 53.927880567000003, 2024-09-16T14:49:47.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011864743000000001, 2024-09-16T14:49:47.871 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.872 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:47.872 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.872 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.872 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.872 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 402 2024-09-16T14:49:47.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.872 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:47.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:47.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:47.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:47.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883921+0000", 2024-09-16T14:49:47.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7520000000000001e-06 2024-09-16T14:49:47.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883927+0000", 2024-09-16T14:49:47.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4679999999999997e-06 2024-09-16T14:49:47.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883932+0000", 2024-09-16T14:49:47.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1440000000000002e-06 2024-09-16T14:49:47.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890477+0000", 2024-09-16T14:49:47.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065446430000000002 2024-09-16T14:49:47.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:47.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890586+0000", 2024-09-16T14:49:47.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010909 2024-09-16T14:49:47.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894750+0000", 2024-09-16T14:49:47.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00416359 2024-09-16T14:49:47.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894834+0000", 2024-09-16T14:49:47.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4596000000000001e-05 2024-09-16T14:49:47.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895783+0000", 2024-09-16T14:49:47.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00094892000000000002 2024-09-16T14:49:47.876 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.876 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.876 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:443 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.935669994000001, 2024-09-16T14:49:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.013766494000000001, 2024-09-16T14:49:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 443 2024-09-16T14:49:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876127+0000", 2024-09-16T14:49:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876138+0000", 2024-09-16T14:49:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1344e-05 2024-09-16T14:49:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876159+0000", 2024-09-16T14:49:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1157000000000001e-05 2024-09-16T14:49:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876170+0000", 2024-09-16T14:49:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0651e-05 2024-09-16T14:49:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876234+0000", 2024-09-16T14:49:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4114000000000003e-05 2024-09-16T14:49:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876243+0000", 2024-09-16T14:49:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0049999999999993e-06 2024-09-16T14:49:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879445+0000", 2024-09-16T14:49:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032019269999999998 2024-09-16T14:49:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879550+0000", 2024-09-16T14:49:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010536500000000001 2024-09-16T14:49:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889895+0000", 2024-09-16T14:49:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010345043 2024-09-16T14:49:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:444 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.935580287000001, 2024-09-16T14:49:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014716241, 2024-09-16T14:49:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 444 2024-09-16T14:49:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876218+0000", 2024-09-16T14:49:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876224+0000", 2024-09-16T14:49:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:49:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876235+0000", 2024-09-16T14:49:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.025e-05 2024-09-16T14:49:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876242+0000", 2024-09-16T14:49:47.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8790000000000003e-06 2024-09-16T14:49:47.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876315+0000", 2024-09-16T14:49:47.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2260000000000003e-05 2024-09-16T14:49:47.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:47.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876320+0000", 2024-09-16T14:49:47.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5690000000000004e-06 2024-09-16T14:49:47.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880078+0000", 2024-09-16T14:49:47.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0037576430000000002 2024-09-16T14:49:47.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880168+0000", 2024-09-16T14:49:47.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0353999999999996e-05 2024-09-16T14:49:47.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.890935+0000", 2024-09-16T14:49:47.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010766556 2024-09-16T14:49:47.887 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.887 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.887 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.887 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:445 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:47.887 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:47.887 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.935536304999999, 2024-09-16T14:49:47.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016529716999999999, 2024-09-16T14:49:47.887 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.887 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:47.887 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.888 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.888 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.888 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 445 2024-09-16T14:49:47.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.888 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:47.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:47.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876262+0000", 2024-09-16T14:49:47.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:47.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876269+0000", 2024-09-16T14:49:47.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9469999999999997e-06 2024-09-16T14:49:47.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876276+0000", 2024-09-16T14:49:47.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5950000000000004e-06 2024-09-16T14:49:47.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876281+0000", 2024-09-16T14:49:47.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:49:47.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876343+0000", 2024-09-16T14:49:47.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1685000000000003e-05 2024-09-16T14:49:47.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:47.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876348+0000", 2024-09-16T14:49:47.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7269999999999998e-06 2024-09-16T14:49:47.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881091+0000", 2024-09-16T14:49:47.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047434599999999997 2024-09-16T14:49:47.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881181+0000", 2024-09-16T14:49:47.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9838000000000003e-05 2024-09-16T14:49:47.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.892792+0000", 2024-09-16T14:49:47.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011611316 2024-09-16T14:49:47.892 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.892 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.892 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.892 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s1 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:47.893 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:47.893 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 39.878726055000001, 2024-09-16T14:49:47.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011783765999999999, 2024-09-16T14:49:47.893 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.893 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.893 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:47.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:47.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933072+0000", 2024-09-16T14:49:47.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:47.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933092+0000", 2024-09-16T14:49:47.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0000000000000002e-05 2024-09-16T14:49:47.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933119+0000", 2024-09-16T14:49:47.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6877e-05 2024-09-16T14:49:47.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933127+0000", 2024-09-16T14:49:47.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4600000000000003e-06 2024-09-16T14:49:47.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.937087+0000", 2024-09-16T14:49:47.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0039599129999999998 2024-09-16T14:49:47.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.937096+0000", 2024-09-16T14:49:47.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8910000000000001e-06 2024-09-16T14:49:47.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944834+0000", 2024-09-16T14:49:47.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077376809999999997 2024-09-16T14:49:47.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944857+0000", 2024-09-16T14:49:47.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2785999999999998e-05 2024-09-16T14:49:47.897 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.897 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.897 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.897 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:591 3.0s0 3:060eef9f:::smithi04040151-16:head [copy-from ver 27] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:47.897 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:47.897 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.835927046999998, 2024-09-16T14:49:47.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0279670919999999, 2024-09-16T14:49:47.897 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.897 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:47.897 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.898 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.898 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.898 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 591 2024-09-16T14:49:47.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.898 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:47.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:47.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975870+0000", 2024-09-16T14:49:47.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:47.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975878+0000", 2024-09-16T14:49:47.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2619999999999998e-06 2024-09-16T14:49:47.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975902+0000", 2024-09-16T14:49:47.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4036000000000002e-05 2024-09-16T14:49:47.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975912+0000", 2024-09-16T14:49:47.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-16T14:49:47.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975986+0000", 2024-09-16T14:49:47.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3548999999999997e-05 2024-09-16T14:49:47.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:47.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975990+0000", 2024-09-16T14:49:47.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7670000000000001e-06 2024-09-16T14:49:47.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.975658+0000", 2024-09-16T14:49:47.901 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99966882800000001 2024-09-16T14:49:47.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.901 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.901 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for missing object", 2024-09-16T14:49:47.901 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.975885+0000", 2024-09-16T14:49:47.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000226682 2024-09-16T14:49:47.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.982217+0000", 2024-09-16T14:49:47.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063321530000000001 2024-09-16T14:49:47.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.902 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.982350+0000", 2024-09-16T14:49:47.902 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013316700000000001 2024-09-16T14:49:47.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.902 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.902 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.999798+0000", 2024-09-16T14:49:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017447608 2024-09-16T14:49:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001884+0000", 2024-09-16T14:49:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020859889999999999 2024-09-16T14:49:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.003629+0000", 2024-09-16T14:49:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001745106 2024-09-16T14:49:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.903 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.904 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.904 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.003839+0000", 2024-09-16T14:49:47.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020980099999999999 2024-09-16T14:49:47.904 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.904 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.904 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.904 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.904 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:592 3.0s0 3:060eef9f:::smithi04040151-16:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:47.904 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:47.904 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 35.835870122999999, 2024-09-16T14:49:47.904 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0256284010000001, 2024-09-16T14:49:47.904 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.905 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:47.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.905 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.905 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 592 2024-09-16T14:49:47.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.905 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.905 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.905 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.905 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:47.905 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:47.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975928+0000", 2024-09-16T14:49:47.906 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:47.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.906 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.906 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.906 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975936+0000", 2024-09-16T14:49:47.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6090000000000003e-06 2024-09-16T14:49:47.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975943+0000", 2024-09-16T14:49:47.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:47.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.907 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975949+0000", 2024-09-16T14:49:47.907 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8300000000000001e-06 2024-09-16T14:49:47.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.907 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.907 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.976030+0000", 2024-09-16T14:49:47.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1267000000000006e-05 2024-09-16T14:49:47.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:47.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.976033+0000", 2024-09-16T14:49:47.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7590000000000002e-06 2024-09-16T14:49:47.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.908 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.908 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.908 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.977041+0000", 2024-09-16T14:49:47.908 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0010081639999999 2024-09-16T14:49:47.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for missing object", 2024-09-16T14:49:47.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.977086+0000", 2024-09-16T14:49:47.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4879999999999997e-05 2024-09-16T14:49:47.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.909 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.983015+0000", 2024-09-16T14:49:47.909 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059287480000000002 2024-09-16T14:49:47.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.909 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.909 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:47.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.983090+0000", 2024-09-16T14:49:47.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5081999999999997e-05 2024-09-16T14:49:47.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001193+0000", 2024-09-16T14:49:47.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.018102713999999999 2024-09-16T14:49:47.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.910 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.910 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001306+0000", 2024-09-16T14:49:47.910 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011330799999999999 2024-09-16T14:49:47.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.910 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.911 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.911 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001557+0000", 2024-09-16T14:49:47.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025100900000000001 2024-09-16T14:49:47.911 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.911 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.911 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.911 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.911 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs4 90/65 ECSubWrite(tid=380, reqid=osd.3.0:380, at_version=90'77, trim_to=0'0, roll_forward_to=83'73))", 2024-09-16T14:49:47.911 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.950581+0000", 2024-09-16T14:49:47.911 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.861217929, 2024-09-16T14:49:47.911 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010219577000000001, 2024-09-16T14:49:47.911 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.912 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.912 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.950581+0000", 2024-09-16T14:49:47.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.912 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.912 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.950581+0000", 2024-09-16T14:49:47.912 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.912 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.950580+0000", 2024-09-16T14:49:47.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:47.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.950608+0000", 2024-09-16T14:49:47.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7637e-05 2024-09-16T14:49:47.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.913 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.913 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.913 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.951627+0000", 2024-09-16T14:49:47.913 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001019439 2024-09-16T14:49:47.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.951638+0000", 2024-09-16T14:49:47.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0930999999999999e-05 2024-09-16T14:49:47.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.951676+0000", 2024-09-16T14:49:47.914 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7883999999999998e-05 2024-09-16T14:49:47.914 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.914 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.914 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.914 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.951686+0000", 2024-09-16T14:49:47.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0675e-05 2024-09-16T14:49:47.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.960787+0000", 2024-09-16T14:49:47.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091004250000000005 2024-09-16T14:49:47.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.915 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.915 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.915 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.960801+0000", 2024-09-16T14:49:47.915 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3613e-05 2024-09-16T14:49:47.915 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.915 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.915 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.916 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(osd.8.17:4 3.es0 3:7ecfd044:::smithi04040151-35:head [assert-version v52,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:49:47.916 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.971941+0000", 2024-09-16T14:49:47.916 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.839857748, 2024-09-16T14:49:47.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016070661, 2024-09-16T14:49:47.916 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.916 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:47.916 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.916 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.916 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.916 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971941+0000", 2024-09-16T14:49:47.916 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971941+0000", 2024-09-16T14:49:47.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971939+0000", 2024-09-16T14:49:47.917 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:47.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.917 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.917 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.917 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971949+0000", 2024-09-16T14:49:47.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2420000000000006e-06 2024-09-16T14:49:47.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971959+0000", 2024-09-16T14:49:47.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0827e-05 2024-09-16T14:49:47.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.918 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971969+0000", 2024-09-16T14:49:47.918 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6579999999999997e-06 2024-09-16T14:49:47.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.918 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.918 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.972028+0000", 2024-09-16T14:49:47.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9457999999999999e-05 2024-09-16T14:49:47.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.972120+0000", 2024-09-16T14:49:47.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1174999999999994e-05 2024-09-16T14:49:47.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.919 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.919 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.919 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.988012+0000", 2024-09-16T14:49:47.919 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015892125 2024-09-16T14:49:47.919 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.919 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.920 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.920 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.19s1 91/90 ECSubWrite(tid=102, reqid=client.4500.0:822, at_version=91'77, trim_to=0'0, roll_forward_to=91'75))", 2024-09-16T14:49:47.920 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.006520+0000", 2024-09-16T14:49:47.920 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 9.8052790390000002, 2024-09-16T14:49:47.920 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01031002, 2024-09-16T14:49:47.920 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.920 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:47.920 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.920 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.920 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006520+0000", 2024-09-16T14:49:47.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006520+0000", 2024-09-16T14:49:47.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.921 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.921 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006519+0000", 2024-09-16T14:49:47.921 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:47.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.921 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006954+0000", 2024-09-16T14:49:47.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043463699999999998 2024-09-16T14:49:47.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006973+0000", 2024-09-16T14:49:47.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9389e-05 2024-09-16T14:49:47.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.922 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.922 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.922 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006994+0000", 2024-09-16T14:49:47.922 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1229999999999998e-05 2024-09-16T14:49:47.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007063+0000", 2024-09-16T14:49:47.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8979999999999993e-05 2024-09-16T14:49:47.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007077+0000", 2024-09-16T14:49:47.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4119e-05 2024-09-16T14:49:47.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.923 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.923 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.923 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016803+0000", 2024-09-16T14:49:47.923 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0097257460000000004 2024-09-16T14:49:47.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.924 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.924 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016830+0000", 2024-09-16T14:49:47.924 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6835999999999999e-05 2024-09-16T14:49:47.924 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.924 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.924 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.924 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.924 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:927 3.es0 3:7b667bcb:::smithi04040151-2:head [append 0~573440] snapc 46=[46] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-16T14:49:47.924 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:45.975604+0000", 2024-09-16T14:49:47.924 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.836194745, 2024-09-16T14:49:47.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010575327000000001, 2024-09-16T14:49:47.925 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:47.925 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:47.925 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:47.925 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:47.925 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:47.925 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 927 2024-09-16T14:49:47.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.925 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:47.925 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.925 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:47.925 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.975604+0000", 2024-09-16T14:49:47.925 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:47.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.975604+0000", 2024-09-16T14:49:47.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:47.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:47.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.975602+0000", 2024-09-16T14:49:47.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:47.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.926 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.926 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:47.926 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.976305+0000", 2024-09-16T14:49:47.926 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00070255000000000003 2024-09-16T14:49:47.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:47.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.976332+0000", 2024-09-16T14:49:47.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7155e-05 2024-09-16T14:49:47.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:47.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.976337+0000", 2024-09-16T14:49:47.927 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4790000000000004e-06 2024-09-16T14:49:47.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.927 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.927 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:47.927 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.976392+0000", 2024-09-16T14:49:47.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4843999999999999e-05 2024-09-16T14:49:47.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:47.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.976512+0000", 2024-09-16T14:49:47.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119578 2024-09-16T14:49:47.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.928 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.928 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:47.928 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.977488+0000", 2024-09-16T14:49:47.928 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00097599399999999999 2024-09-16T14:49:47.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:47.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.978514+0000", 2024-09-16T14:49:47.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010261109999999999 2024-09-16T14:49:47.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:47.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.986106+0000", 2024-09-16T14:49:47.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0075921909999999999 2024-09-16T14:49:47.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:47.929 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:47.929 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:47.929 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.986179+0000", 2024-09-16T14:49:47.929 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3362999999999995e-05 2024-09-16T14:49:47.930 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.930 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.930 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.930 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:47.930 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:47.930 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:47.931 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:49:47.932 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:47.930+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:47.970 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:49:48.007 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T14:49:48.007 INFO:tasks.rados.rados.0.smithi040.stdout:379: done (2 left) 2024-09-16T14:49:48.008 INFO:tasks.rados.rados.0.smithi040.stdout:380: done (1 left) 2024-09-16T14:49:48.008 INFO:tasks.rados.rados.0.smithi040.stdout:382: done (0 left) 2024-09-16T14:49:48.008 INFO:tasks.rados.rados.0.smithi040.stdout:383: read oid 50 snap -1 2024-09-16T14:49:48.008 INFO:tasks.rados.rados.0.smithi040.stdout:383: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-16T14:49:48.008 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 14 current snap is 37 2024-09-16T14:49:48.008 INFO:tasks.rados.rados.0.smithi040.stdout:384: seq_num 135 ranges {1146880=720896,1867776=622592,2490368=688128,3178496=245760} 2024-09-16T14:49:48.019 INFO:tasks.rados.rados.0.smithi040.stdout:384: writing smithi04040151-14 from 1146880 to 1867776 tid 1 2024-09-16T14:49:48.023 INFO:tasks.rados.rados.0.smithi040.stdout:384: writing smithi04040151-14 from 1867776 to 2490368 tid 2 2024-09-16T14:49:48.028 INFO:tasks.rados.rados.0.smithi040.stdout:384: writing smithi04040151-14 from 2490368 to 3178496 tid 3 2024-09-16T14:49:48.029 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-09-16T14:49:48.030 INFO:tasks.rados.rados.0.smithi040.stdout:384: writing smithi04040151-14 from 3178496 to 3424256 tid 4 2024-09-16T14:49:48.030 INFO:tasks.rados.rados.0.smithi040.stdout:385: rollback oid 15 current snap is 37 2024-09-16T14:49:48.030 INFO:tasks.rados.rados.0.smithi040.stdout:rollback oid 15 to 36 2024-09-16T14:49:48.030 INFO:tasks.rados.rados.0.smithi040.stdout:383: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-16T14:49:48.032 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:48.031+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:48.052 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:48.052 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:48.052 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:48.052 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:48.058 INFO:tasks.rados.rados.0.smithi040.stdout:384: finishing write tid 1 to smithi04040151-14 2024-09-16T14:49:48.058 INFO:tasks.rados.rados.0.smithi040.stdout:384: finishing write tid 2 to smithi04040151-14 2024-09-16T14:49:48.058 INFO:tasks.rados.rados.0.smithi040.stdout:384: finishing write tid 3 to smithi04040151-14 2024-09-16T14:49:48.058 INFO:tasks.rados.rados.0.smithi040.stdout:385: finishing rollback tid 0 to smithi04040151-15 2024-09-16T14:49:48.058 INFO:tasks.rados.rados.0.smithi040.stdout:384: finishing write tid 4 to smithi04040151-14 2024-09-16T14:49:48.058 INFO:tasks.rados.rados.0.smithi040.stdout:383: done (2 left) 2024-09-16T14:49:48.058 INFO:tasks.rados.rados.0.smithi040.stdout:386: snap_remove snap 34 2024-09-16T14:49:48.061 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:49:48.116 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:48.116 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:48.133 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:48.131+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:48.133 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:48.133 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:48.150 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:48.150 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:48.166 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:48.166 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:48.182 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:48.183 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:48.199 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:48.199 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:48.207 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:48.207 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:48.208 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:48.208 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:48.208 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:48.215 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:48.216 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:49:48.231 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:48.234 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:48.232+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:48.248 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:48.248 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:48.264 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:48.264 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:48.280 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:48.281 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:48.297 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:48.297 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:48.319 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:48.319 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:48.328 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:49:48.335 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:48.334+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:48.436 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:48.436+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:48.456 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:48.457 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:48.466 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:49:48.538 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:48.537+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:48.573 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:48.573 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:48.582 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:49:48.638 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:48.637+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:48.688 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:48.688 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:48.697 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:49:48.740 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:48.739+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:48.804 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:48.804 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:48.812 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:49:48.856 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:48.854+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:48.920 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:48.920 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:48.928 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:49:48.941 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:48.941+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:49.014 INFO:tasks.rados.rados.0.smithi040.stdout:386: done (2 left) 2024-09-16T14:49:49.014 INFO:tasks.rados.rados.0.smithi040.stdout:387: read oid 46 snap -1 2024-09-16T14:49:49.014 INFO:tasks.rados.rados.0.smithi040.stdout:387: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-16T14:49:49.014 INFO:tasks.rados.rados.0.smithi040.stdout:388: snap_create 2024-09-16T14:49:49.014 INFO:tasks.rados.rados.0.smithi040.stdout:384: finishing write tid 5 to smithi04040151-14 2024-09-16T14:49:49.014 INFO:tasks.rados.rados.0.smithi040.stdout:384: finishing write tid 6 to smithi04040151-14 2024-09-16T14:49:49.014 INFO:tasks.rados.rados.0.smithi040.stdout:384: oid 14 updating version 0 to 99 2024-09-16T14:49:49.014 INFO:tasks.rados.rados.0.smithi040.stdout:384: oid 14 updating version 99 to 100 2024-09-16T14:49:49.014 INFO:tasks.rados.rados.0.smithi040.stdout:384: oid 14 updating version 100 to 101 2024-09-16T14:49:49.015 INFO:tasks.rados.rados.0.smithi040.stdout:384: oid 14 updating version 101 to 102 2024-09-16T14:49:49.015 INFO:tasks.rados.rados.0.smithi040.stdout:384: oid 14 updating version 102 to 103 2024-09-16T14:49:49.015 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 14 v 103 (ObjNum 135 snap 37 seq_num 135) dirty exists 2024-09-16T14:49:49.015 INFO:tasks.rados.rados.0.smithi040.stdout:384: left oid 14 (ObjNum 135 snap 37 seq_num 135) 2024-09-16T14:49:49.015 INFO:tasks.rados.rados.0.smithi040.stdout:385: finishing rollback tid 1 to smithi04040151-15 2024-09-16T14:49:49.015 INFO:tasks.rados.rados.0.smithi040.stdout:385: finishing rollback tid 2 to smithi04040151-15 2024-09-16T14:49:49.015 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 15 v 87 (ObjNum 94 snap 17 seq_num 94) dirty exists 2024-09-16T14:49:49.015 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.8 to 1 (10000) 2024-09-16T14:49:49.027 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:49:49.033 INFO:tasks.rados.rados.0.smithi040.stdout:387: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-16T14:49:49.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-09-16T14:49:49.082 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:49.080+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:49.092 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:49.092 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:49.101 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:49:49.143 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:49.142+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:49.244 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:49.243+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:49.252 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:49.252 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:49.261 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:49:49.345 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:49.344+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:49.369 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:49.369 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:49.378 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:49:49.446 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:49.445+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:49.484 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:49.484 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:49.493 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:49:49.547 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:49.546+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:49.600 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:49.600 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:49.609 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:49:49.648 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:49.647+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:49.715 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:49.715 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:49.724 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:49:49.768 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:49.766+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:49.831 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:49.831 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:49.840 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:49:49.858 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is unset 2024-09-16T14:49:49.859 INFO:tasks.rados.rados.0.smithi040.stdout:384: done (3 left) 2024-09-16T14:49:49.859 INFO:tasks.rados.rados.0.smithi040.stdout:385: done (2 left) 2024-09-16T14:49:49.859 INFO:tasks.rados.rados.0.smithi040.stdout:387: done (1 left) 2024-09-16T14:49:49.859 INFO:tasks.rados.rados.0.smithi040.stdout:388: done (0 left) 2024-09-16T14:49:49.859 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.9 to 1 (10000) 2024-09-16T14:49:49.860 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 45 current snap is 38 2024-09-16T14:49:49.860 INFO:tasks.rados.rados.0.smithi040.stdout:389: seq_num 136 ranges {1933312=737280,2670592=770048,3440640=540672,3981312=245760} 2024-09-16T14:49:49.872 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-09-16T14:49:49.873 INFO:tasks.rados.rados.0.smithi040.stdout:389: writing smithi04040151-45 from 1933312 to 2670592 tid 1 2024-09-16T14:49:49.877 INFO:tasks.rados.rados.0.smithi040.stdout:389: writing smithi04040151-45 from 2670592 to 3440640 tid 2 2024-09-16T14:49:49.881 INFO:tasks.rados.rados.0.smithi040.stdout:389: writing smithi04040151-45 from 3440640 to 3981312 tid 3 2024-09-16T14:49:49.883 INFO:tasks.rados.rados.0.smithi040.stdout:389: writing smithi04040151-45 from 3981312 to 4227072 tid 4 2024-09-16T14:49:49.883 INFO:tasks.rados.rados.0.smithi040.stdout:390: snap_create 2024-09-16T14:49:49.885 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:49.883+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:49.890 INFO:tasks.rados.rados.0.smithi040.stdout:389: finishing write tid 1 to smithi04040151-45 2024-09-16T14:49:49.890 INFO:tasks.rados.rados.0.smithi040.stdout:389: finishing write tid 2 to smithi04040151-45 2024-09-16T14:49:49.890 INFO:tasks.rados.rados.0.smithi040.stdout:389: finishing write tid 3 to smithi04040151-45 2024-09-16T14:49:49.890 INFO:tasks.rados.rados.0.smithi040.stdout:389: finishing write tid 4 to smithi04040151-45 2024-09-16T14:49:49.892 INFO:tasks.rados.rados.0.smithi040.stdout:389: finishing write tid 5 to smithi04040151-45 2024-09-16T14:49:49.894 INFO:tasks.rados.rados.0.smithi040.stdout:389: finishing write tid 6 to smithi04040151-45 2024-09-16T14:49:49.894 INFO:tasks.rados.rados.0.smithi040.stdout:389: oid 45 updating version 0 to 80 2024-09-16T14:49:49.894 INFO:tasks.rados.rados.0.smithi040.stdout:389: oid 45 version 80 is already newer than 79 2024-09-16T14:49:49.894 INFO:tasks.rados.rados.0.smithi040.stdout:389: oid 45 version 80 is already newer than 76 2024-09-16T14:49:49.894 INFO:tasks.rados.rados.0.smithi040.stdout:389: oid 45 version 80 is already newer than 78 2024-09-16T14:49:49.894 INFO:tasks.rados.rados.0.smithi040.stdout:389: oid 45 version 80 is already newer than 77 2024-09-16T14:49:49.894 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 45 v 80 (ObjNum 136 snap 38 seq_num 136) dirty exists 2024-09-16T14:49:49.894 INFO:tasks.rados.rados.0.smithi040.stdout:389: left oid 45 (ObjNum 136 snap 38 seq_num 136) 2024-09-16T14:49:49.953 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:49.951+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:49.961 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:49.961 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:49.970 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:49:50.054 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:50.051+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:50.113 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:50.113 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:50.122 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:49:50.154 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:50.153+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:50.225 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:50.225 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:50.233 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:49:50.255 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:50.254+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:50.309 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:49:50.335 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:50.335 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:50.344 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:49:50.356 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:50.354+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:50.432 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:50.433 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:50.447 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:50.447 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:50.449 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:50.449 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:50.457 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:49:50.457 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:50.455+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:50.465 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:50.465 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:50.481 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:50.497 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:50.498 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:50.514 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:50.514 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:50.530 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:50.530 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:50.547 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:50.547 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:50.557 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:50.556+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:50.562 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:50.562 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:50.562 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:50.562 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:50.562 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.562 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:449 3.5s0 3:a7111aa9:::smithi04040151-45:head [stat,delete] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:50.562 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:50.562 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 52.682928525000001, 2024-09-16T14:49:50.562 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013571421, 2024-09-16T14:49:50.562 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.563 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:50.563 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:50.563 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:50.563 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:50.563 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 449 2024-09-16T14:49:50.563 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.563 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.563 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.563 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.563 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:50.563 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.563 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.563 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.564 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.564 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:50.564 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.564 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.564 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.564 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.564 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876385+0000", 2024-09-16T14:49:50.564 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:50.564 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.564 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.564 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.564 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876396+0000", 2024-09-16T14:49:50.564 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1094000000000001e-05 2024-09-16T14:49:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876412+0000", 2024-09-16T14:49:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6563e-05 2024-09-16T14:49:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876421+0000", 2024-09-16T14:49:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5189999999999992e-06 2024-09-16T14:49:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.565 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876477+0000", 2024-09-16T14:49:50.566 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.6679999999999999e-05 2024-09-16T14:49:50.566 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.566 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.566 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:50.566 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876487+0000", 2024-09-16T14:49:50.566 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4509999999999998e-06 2024-09-16T14:49:50.566 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.566 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.566 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.566 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.878486+0000", 2024-09-16T14:49:50.566 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001999635 2024-09-16T14:49:50.566 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.566 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.566 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:50.567 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.878592+0000", 2024-09-16T14:49:50.567 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000105398 2024-09-16T14:49:50.567 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.567 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.567 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.567 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.889958+0000", 2024-09-16T14:49:50.567 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01136595 2024-09-16T14:49:50.567 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.567 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.567 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.567 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.567 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.567 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:467 3.5s0 3:a29e72cc:::smithi04040151-39:head [rollback 14] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:50.567 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:50.568 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 52.563735967, 2024-09-16T14:49:50.568 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012554012, 2024-09-16T14:49:50.568 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.568 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:50.568 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:50.568 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:50.568 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:50.568 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 467 2024-09-16T14:49:50.568 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.568 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.568 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.568 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.568 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.569 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995581+0000", 2024-09-16T14:49:50.570 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.9009999999999998e-06 2024-09-16T14:49:50.570 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.570 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.570 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.570 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995587+0000", 2024-09-16T14:49:50.570 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8850000000000004e-06 2024-09-16T14:49:50.570 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.570 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.570 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.570 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995592+0000", 2024-09-16T14:49:50.570 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9749999999999998e-06 2024-09-16T14:49:50.570 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.570 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.570 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.571 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.996914+0000", 2024-09-16T14:49:50.571 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00132168 2024-09-16T14:49:50.571 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.571 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.571 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:50.571 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997029+0000", 2024-09-16T14:49:50.571 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000115203 2024-09-16T14:49:50.571 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.571 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.571 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:50.571 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997570+0000", 2024-09-16T14:49:50.571 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00054107999999999999 2024-09-16T14:49:50.571 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.571 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.572 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:50.572 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003265+0000", 2024-09-16T14:49:50.572 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0056945679999999997 2024-09-16T14:49:50.572 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.572 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.572 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:50.572 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008067+0000", 2024-09-16T14:49:50.572 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048018039999999998 2024-09-16T14:49:50.572 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.572 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.572 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.572 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008133+0000", 2024-09-16T14:49:50.572 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6365999999999994e-05 2024-09-16T14:49:50.572 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.573 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.573 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.573 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.573 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.573 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:468 3.5s0 3:a29e72cc:::smithi04040151-39:head [append 0~0] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:50.573 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:50.573 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 52.563566907000002, 2024-09-16T14:49:50.573 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012562663999999999, 2024-09-16T14:49:50.573 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.573 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:50.573 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:50.573 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:50.573 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:50.573 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 468 2024-09-16T14:49:50.574 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.574 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.574 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.574 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.574 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:50.574 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.574 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.574 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.574 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.574 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:50.574 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.574 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.574 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.574 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.575 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995747+0000", 2024-09-16T14:49:50.575 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:50.575 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.575 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.575 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.575 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995752+0000", 2024-09-16T14:49:50.575 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.916e-06 2024-09-16T14:49:50.575 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.575 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.575 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.575 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995762+0000", 2024-09-16T14:49:50.575 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4609999999999997e-06 2024-09-16T14:49:50.575 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.575 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.576 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.576 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995768+0000", 2024-09-16T14:49:50.576 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8109999999999997e-06 2024-09-16T14:49:50.576 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.576 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.576 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.576 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.998577+0000", 2024-09-16T14:49:50.576 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0028088369999999998 2024-09-16T14:49:50.576 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.576 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.576 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:50.576 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.998680+0000", 2024-09-16T14:49:50.576 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010318399999999999 2024-09-16T14:49:50.577 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.577 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.577 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:50.577 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.999182+0000", 2024-09-16T14:49:50.577 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00050101099999999999 2024-09-16T14:49:50.577 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.577 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.577 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:50.577 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.005964+0000", 2024-09-16T14:49:50.577 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0067821490000000003 2024-09-16T14:49:50.577 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.577 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.577 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:50.578 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008246+0000", 2024-09-16T14:49:50.578 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0022827670000000002 2024-09-16T14:49:50.578 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.578 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.578 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.578 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008311+0000", 2024-09-16T14:49:50.578 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.4269000000000004e-05 2024-09-16T14:49:50.578 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.578 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.578 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.578 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.578 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.578 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:50.578 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932515+0000", 2024-09-16T14:49:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 42.626800174000003, 2024-09-16T14:49:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010770217, 2024-09-16T14:49:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932515+0000", 2024-09-16T14:49:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932515+0000", 2024-09-16T14:49:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.580 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.580 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.580 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932514+0000", 2024-09-16T14:49:50.580 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:50.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.580 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.580 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.580 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932539+0000", 2024-09-16T14:49:50.580 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4494e-05 2024-09-16T14:49:50.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.580 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.580 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.581 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932549+0000", 2024-09-16T14:49:50.581 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0028e-05 2024-09-16T14:49:50.581 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.581 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.581 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.581 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932556+0000", 2024-09-16T14:49:50.581 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.6920000000000002e-06 2024-09-16T14:49:50.581 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.581 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.581 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.581 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932618+0000", 2024-09-16T14:49:50.581 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.1600000000000007e-05 2024-09-16T14:49:50.581 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.581 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.582 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:50.582 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932626+0000", 2024-09-16T14:49:50.582 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4139999999999994e-06 2024-09-16T14:49:50.582 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.582 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.582 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:50.582 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943268+0000", 2024-09-16T14:49:50.582 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010641695 2024-09-16T14:49:50.582 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.582 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.582 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.582 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943285+0000", 2024-09-16T14:49:50.582 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6929e-05 2024-09-16T14:49:50.582 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.583 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.583 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.583 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.583 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.583 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds1 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:50.583 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:50.583 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 42.626547494, 2024-09-16T14:49:50.583 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010600091000000001, 2024-09-16T14:49:50.583 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.583 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:50.583 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.583 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.583 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.583 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:50.584 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.584 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.584 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.584 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.584 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:50.584 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.584 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.584 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.584 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.584 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:50.584 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:50.584 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.584 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.584 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932785+0000", 2024-09-16T14:49:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7829000000000002e-05 2024-09-16T14:49:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932794+0000", 2024-09-16T14:49:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5179999999999993e-06 2024-09-16T14:49:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932801+0000", 2024-09-16T14:49:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.793e-06 2024-09-16T14:49:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.586 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.586 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933189+0000", 2024-09-16T14:49:50.586 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00038759100000000001 2024-09-16T14:49:50.586 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.586 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.586 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:50.586 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933197+0000", 2024-09-16T14:49:50.586 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5290000000000008e-06 2024-09-16T14:49:50.586 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.586 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.586 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:50.586 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943351+0000", 2024-09-16T14:49:50.586 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010153673 2024-09-16T14:49:50.586 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.587 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.587 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.587 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943368+0000", 2024-09-16T14:49:50.587 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6538999999999999e-05 2024-09-16T14:49:50.587 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.587 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.587 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.587 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.587 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.587 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/27 ECSubWrite(tid=63, reqid=osd.1.0:63, at_version=61'50, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:50.587 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:50.587 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 42.626494759000003, 2024-09-16T14:49:50.587 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010214164, 2024-09-16T14:49:50.587 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.588 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:50.588 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.588 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.588 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.588 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:50.588 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.588 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.588 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.588 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.588 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:50.588 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.588 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.588 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.588 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.589 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:50.589 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:50.589 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.589 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.589 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.589 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932838+0000", 2024-09-16T14:49:50.589 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8576e-05 2024-09-16T14:49:50.589 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.589 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.589 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.589 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932847+0000", 2024-09-16T14:49:50.589 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6519999999999995e-06 2024-09-16T14:49:50.589 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.589 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.590 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.590 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932855+0000", 2024-09-16T14:49:50.590 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7600000000000002e-06 2024-09-16T14:49:50.590 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.590 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.590 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.590 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932929+0000", 2024-09-16T14:49:50.590 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4134999999999994e-05 2024-09-16T14:49:50.590 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.590 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.590 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:50.590 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932937+0000", 2024-09-16T14:49:50.590 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.1519999999999992e-06 2024-09-16T14:49:50.591 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.591 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.591 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:50.591 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941947+0000", 2024-09-16T14:49:50.591 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0090100430000000006 2024-09-16T14:49:50.591 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.591 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.591 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.591 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943034+0000", 2024-09-16T14:49:50.591 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001087111 2024-09-16T14:49:50.591 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.591 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.591 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.591 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.592 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.592 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as2 61/53 ECSubWrite(tid=225, reqid=osd.11.0:225, at_version=61'26, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:50.592 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:50.592 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 42.626400771999997, 2024-09-16T14:49:50.592 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010782797, 2024-09-16T14:49:50.592 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.592 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:50.592 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.592 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.592 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.592 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:50.592 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.592 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.593 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.593 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.593 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:50.593 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.593 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.593 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.593 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.593 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:50.593 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:50.593 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.593 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.593 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.593 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932954+0000", 2024-09-16T14:49:50.594 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0166000000000001e-05 2024-09-16T14:49:50.594 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.594 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.594 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.594 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932980+0000", 2024-09-16T14:49:50.594 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5843e-05 2024-09-16T14:49:50.594 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.594 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.594 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.594 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932987+0000", 2024-09-16T14:49:50.594 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0079999999999996e-06 2024-09-16T14:49:50.595 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.595 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.595 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.595 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933034+0000", 2024-09-16T14:49:50.595 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6934999999999997e-05 2024-09-16T14:49:50.595 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.595 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.595 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:50.595 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933042+0000", 2024-09-16T14:49:50.595 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8429999999999999e-06 2024-09-16T14:49:50.595 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.595 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.595 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:50.596 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943652+0000", 2024-09-16T14:49:50.596 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010609288999999999 2024-09-16T14:49:50.596 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.596 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.596 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.596 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943697+0000", 2024-09-16T14:49:50.596 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5296999999999997e-05 2024-09-16T14:49:50.596 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.596 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.596 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.596 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.596 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.597 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:50.597 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:50.597 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 42.626350686999999, 2024-09-16T14:49:50.597 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010665579999999999, 2024-09-16T14:49:50.597 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.597 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:50.597 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.597 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.597 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.597 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:50.597 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.597 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.597 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932995+0000", 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1084000000000003e-05 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.598 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.599 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933003+0000", 2024-09-16T14:49:50.599 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2199999999999992e-06 2024-09-16T14:49:50.599 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.599 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.599 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.599 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933010+0000", 2024-09-16T14:49:50.599 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.4359999999999999e-06 2024-09-16T14:49:50.599 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.599 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.599 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.599 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935460+0000", 2024-09-16T14:49:50.599 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0024503350000000001 2024-09-16T14:49:50.599 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.600 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.600 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:50.600 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935469+0000", 2024-09-16T14:49:50.600 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.602e-06 2024-09-16T14:49:50.600 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.600 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.600 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:50.600 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943610+0000", 2024-09-16T14:49:50.600 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0081417390000000003 2024-09-16T14:49:50.600 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.600 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.600 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.600 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943630+0000", 2024-09-16T14:49:50.601 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9584000000000001e-05 2024-09-16T14:49:50.601 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.601 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.601 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.601 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.601 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.601 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:579 3.5s0 3:a29e72cc:::smithi04040151-39:head [read 0~671744,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:50.601 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.946788+0000", 2024-09-16T14:49:50.601 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 38.612527190999998, 2024-09-16T14:49:50.601 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.037038522, 2024-09-16T14:49:50.601 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.601 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:50.602 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:50.602 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:50.602 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:50.602 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 579 2024-09-16T14:49:50.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.602 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.602 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.602 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.602 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946788+0000", 2024-09-16T14:49:50.602 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.602 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.602 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.602 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946788+0000", 2024-09-16T14:49:50.603 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.603 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.603 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.603 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.603 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946786+0000", 2024-09-16T14:49:50.603 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:50.603 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.603 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.603 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.603 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946795+0000", 2024-09-16T14:49:50.603 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9549999999999998e-06 2024-09-16T14:49:50.603 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.603 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.603 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.604 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946806+0000", 2024-09-16T14:49:50.604 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1174e-05 2024-09-16T14:49:50.604 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.604 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.604 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.604 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946816+0000", 2024-09-16T14:49:50.604 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1949999999999992e-06 2024-09-16T14:49:50.604 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.604 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.604 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.604 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.947936+0000", 2024-09-16T14:49:50.604 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011208679999999999 2024-09-16T14:49:50.604 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.605 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.605 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:50.605 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.947940+0000", 2024-09-16T14:49:50.605 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.365e-06 2024-09-16T14:49:50.605 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.605 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.605 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.605 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.976737+0000", 2024-09-16T14:49:50.605 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.028796684 2024-09-16T14:49:50.605 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.608 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.608 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:50.608 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.977002+0000", 2024-09-16T14:49:50.608 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00026509800000000002 2024-09-16T14:49:50.608 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.609 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.609 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.609 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.983826+0000", 2024-09-16T14:49:50.609 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0068246660000000001 2024-09-16T14:49:50.609 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.609 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.609 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.609 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.609 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.609 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:580 3.5s0 3:a29e72cc:::smithi04040151-39:head [read 0~671744,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:50.609 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.946829+0000", 2024-09-16T14:49:50.609 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 38.612486222999998, 2024-09-16T14:49:50.610 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0381929240000001, 2024-09-16T14:49:50.610 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.610 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:50.610 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:50.610 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:50.610 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:50.610 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 580 2024-09-16T14:49:50.610 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.610 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.610 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.610 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.610 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946829+0000", 2024-09-16T14:49:50.610 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.611 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.611 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.611 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.611 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946829+0000", 2024-09-16T14:49:50.611 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.611 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.611 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.611 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.611 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946828+0000", 2024-09-16T14:49:50.611 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:50.611 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.611 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.611 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.612 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946832+0000", 2024-09-16T14:49:50.612 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9009999999999996e-06 2024-09-16T14:49:50.612 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.612 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.612 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.612 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946839+0000", 2024-09-16T14:49:50.612 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9619999999999996e-06 2024-09-16T14:49:50.612 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.612 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.612 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.612 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946846+0000", 2024-09-16T14:49:50.612 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7789999999999998e-06 2024-09-16T14:49:50.612 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.612 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.613 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.613 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.948182+0000", 2024-09-16T14:49:50.613 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0013367609999999999 2024-09-16T14:49:50.613 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.613 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.613 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:50.613 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.948185+0000", 2024-09-16T14:49:50.613 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.357e-06 2024-09-16T14:49:50.613 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.613 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.613 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.613 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.979390+0000", 2024-09-16T14:49:50.613 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0312049299999999 2024-09-16T14:49:50.614 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.614 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.614 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:50.614 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.979672+0000", 2024-09-16T14:49:50.614 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00028239299999999998 2024-09-16T14:49:50.614 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.614 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.614 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.614 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.985022+0000", 2024-09-16T14:49:50.614 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053494620000000001 2024-09-16T14:49:50.614 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.614 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.614 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.614 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.615 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.615 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:581 3.5s0 3:a29e72cc:::smithi04040151-39:head [read 0~671744,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:50.615 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.946851+0000", 2024-09-16T14:49:50.615 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 38.612463923999996, 2024-09-16T14:49:50.615 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0393245289999999, 2024-09-16T14:49:50.615 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.615 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:50.615 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:50.615 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:50.615 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:50.615 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 581 2024-09-16T14:49:50.615 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.615 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.615 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.616 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.616 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946851+0000", 2024-09-16T14:49:50.616 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.616 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.616 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.616 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.616 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946851+0000", 2024-09-16T14:49:50.616 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.616 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.616 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.616 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.616 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946851+0000", 2024-09-16T14:49:50.616 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:50.616 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.617 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.617 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.617 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946854+0000", 2024-09-16T14:49:50.617 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.937e-06 2024-09-16T14:49:50.617 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.617 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.617 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.617 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946860+0000", 2024-09-16T14:49:50.617 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.1120000000000001e-06 2024-09-16T14:49:50.617 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.617 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.617 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.617 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946864+0000", 2024-09-16T14:49:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.634e-06 2024-09-16T14:49:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.948218+0000", 2024-09-16T14:49:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0013536209999999999 2024-09-16T14:49:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.948220+0000", 2024-09-16T14:49:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.866e-06 2024-09-16T14:49:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.618 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.619 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980141+0000", 2024-09-16T14:49:50.619 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0319210940000001 2024-09-16T14:49:50.619 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.619 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.619 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:50.619 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980309+0000", 2024-09-16T14:49:50.619 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00016819599999999999 2024-09-16T14:49:50.619 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.619 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.619 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.619 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.986176+0000", 2024-09-16T14:49:50.619 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0058664870000000001 2024-09-16T14:49:50.619 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.619 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.620 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.620 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.620 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.620 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:690 3.cs0 3:3bbed9f6:::smithi04040151-38:head [append 0~720896] snapc 2e=[2e,2b,29,28,25] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T14:49:50.620 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:21.919040+0000", 2024-09-16T14:49:50.620 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 28.640274536, 2024-09-16T14:49:50.620 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012801389999999999, 2024-09-16T14:49:50.620 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.620 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:50.620 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:50.620 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:50.620 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:50.620 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 690 2024-09-16T14:49:50.620 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.621 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.621 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.621 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.621 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.919040+0000", 2024-09-16T14:49:50.621 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.621 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.621 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.621 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.621 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.919040+0000", 2024-09-16T14:49:50.621 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.621 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.621 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.621 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.621 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.919038+0000", 2024-09-16T14:49:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.920351+0000", 2024-09-16T14:49:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001312913 2024-09-16T14:49:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.920366+0000", 2024-09-16T14:49:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4728e-05 2024-09-16T14:49:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.623 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.920389+0000", 2024-09-16T14:49:50.623 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3456999999999999e-05 2024-09-16T14:49:50.623 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.623 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.623 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.623 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.920466+0000", 2024-09-16T14:49:50.623 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7377000000000001e-05 2024-09-16T14:49:50.623 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.623 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.623 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:50.623 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.920545+0000", 2024-09-16T14:49:50.623 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8739000000000006e-05 2024-09-16T14:49:50.623 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.624 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.624 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:50.624 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.921681+0000", 2024-09-16T14:49:50.624 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011363079999999999 2024-09-16T14:49:50.624 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.624 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.624 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:50.624 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.922719+0000", 2024-09-16T14:49:50.624 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0010372249999999999 2024-09-16T14:49:50.624 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.624 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.624 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:50.624 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.931783+0000", 2024-09-16T14:49:50.625 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0090647539999999995 2024-09-16T14:49:50.625 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.625 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.625 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.625 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.931842+0000", 2024-09-16T14:49:50.625 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8369000000000003e-05 2024-09-16T14:49:50.625 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.625 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.625 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.625 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.625 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.625 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:694 3.cs0 3:3bbed9f6:::smithi04040151-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-09-16T14:49:50.625 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:21.927339+0000", 2024-09-16T14:49:50.625 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 28.631975897, 2024-09-16T14:49:50.626 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089489630000000008, 2024-09-16T14:49:50.626 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.626 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:50.626 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:50.626 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:50.626 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:50.626 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 694 2024-09-16T14:49:50.626 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.626 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.626 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.626 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.626 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.927339+0000", 2024-09-16T14:49:50.626 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.627 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.627 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.627 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.627 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.927339+0000", 2024-09-16T14:49:50.627 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.627 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.627 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.627 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.627 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.927339+0000", 2024-09-16T14:49:50.627 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:50.627 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.627 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.627 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.627 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.927341+0000", 2024-09-16T14:49:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.894e-06 2024-09-16T14:49:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.927344+0000", 2024-09-16T14:49:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.7419999999999999e-06 2024-09-16T14:49:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.927353+0000", 2024-09-16T14:49:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3489999999999992e-06 2024-09-16T14:49:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.628 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.629 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.929608+0000", 2024-09-16T14:49:50.629 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0022548749999999999 2024-09-16T14:49:50.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.629 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.629 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:50.629 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.929682+0000", 2024-09-16T14:49:50.629 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4530000000000006e-05 2024-09-16T14:49:50.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.629 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.629 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.629 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.933769+0000", 2024-09-16T14:49:50.629 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0040868349999999996 2024-09-16T14:49:50.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.629 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.630 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:50.630 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.933819+0000", 2024-09-16T14:49:50.630 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0268000000000001e-05 2024-09-16T14:49:50.630 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.630 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.630 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.630 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.936288+0000", 2024-09-16T14:49:50.630 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0024688150000000001 2024-09-16T14:49:50.630 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.630 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.630 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.630 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.630 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.631 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:760 3.5s0 3:a29e72cc:::smithi04040151-39:head [append 0~0] snapc 31=[2e,28] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:49:50.631 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.926013+0000", 2024-09-16T14:49:50.631 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 21.633301973999998, 2024-09-16T14:49:50.631 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.032715963000000001, 2024-09-16T14:49:50.631 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.631 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:50.631 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:50.631 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:50.631 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:50.631 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 760 2024-09-16T14:49:50.631 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.631 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.631 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.631 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.632 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926013+0000", 2024-09-16T14:49:50.632 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.632 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.632 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.632 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.632 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926013+0000", 2024-09-16T14:49:50.632 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.632 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.632 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.632 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.632 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926011+0000", 2024-09-16T14:49:50.632 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:50.632 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.632 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.633 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.633 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926020+0000", 2024-09-16T14:49:50.633 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6729999999999998e-06 2024-09-16T14:49:50.633 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.633 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.633 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.633 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926033+0000", 2024-09-16T14:49:50.633 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3030000000000001e-05 2024-09-16T14:49:50.633 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.633 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.633 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.633 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926041+0000", 2024-09-16T14:49:50.633 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0120000000000005e-06 2024-09-16T14:49:50.633 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.634 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.634 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.634 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928219+0000", 2024-09-16T14:49:50.634 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021786000000000002 2024-09-16T14:49:50.634 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.634 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.634 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:50.634 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928222+0000", 2024-09-16T14:49:50.634 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.943e-06 2024-09-16T14:49:50.634 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.634 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.634 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.634 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.942660+0000", 2024-09-16T14:49:50.634 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014438124 2024-09-16T14:49:50.635 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.635 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.635 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:50.635 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.943382+0000", 2024-09-16T14:49:50.635 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00072144099999999997 2024-09-16T14:49:50.635 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.635 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.635 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.635 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.955102+0000", 2024-09-16T14:49:50.635 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011719985 2024-09-16T14:49:50.635 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.635 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.635 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:50.635 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.955185+0000", 2024-09-16T14:49:50.636 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3676000000000003e-05 2024-09-16T14:49:50.636 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.636 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.636 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:50.636 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.955753+0000", 2024-09-16T14:49:50.636 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00056795200000000004 2024-09-16T14:49:50.636 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.636 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.636 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:50.636 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.958254+0000", 2024-09-16T14:49:50.636 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025006130000000001 2024-09-16T14:49:50.636 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.636 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.637 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:50.637 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.958435+0000", 2024-09-16T14:49:50.637 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00018093800000000001 2024-09-16T14:49:50.637 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.637 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.637 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.637 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.958729+0000", 2024-09-16T14:49:50.637 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00029398599999999999 2024-09-16T14:49:50.637 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.637 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.637 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.637 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.637 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.637 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:761 3.5s0 3:a29e72cc:::smithi04040151-39:head [rollback 2e] snapc 31=[2e,28] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:49:50.638 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.926050+0000", 2024-09-16T14:49:50.638 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 21.633264578999999, 2024-09-16T14:49:50.638 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.037148401999999997, 2024-09-16T14:49:50.638 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.638 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:50.638 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:50.638 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:50.638 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:50.638 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 761 2024-09-16T14:49:50.638 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.638 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.638 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.638 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.638 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926050+0000", 2024-09-16T14:49:50.639 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.639 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.639 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.639 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.639 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926050+0000", 2024-09-16T14:49:50.639 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.639 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.639 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.639 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.639 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926050+0000", 2024-09-16T14:49:50.639 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:50.639 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.639 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.639 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.640 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926053+0000", 2024-09-16T14:49:50.640 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0350000000000002e-06 2024-09-16T14:49:50.640 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.640 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.640 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.640 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926060+0000", 2024-09-16T14:49:50.640 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0790000000000003e-06 2024-09-16T14:49:50.640 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.640 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.640 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.640 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926065+0000", 2024-09-16T14:49:50.640 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3650000000000003e-06 2024-09-16T14:49:50.640 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.640 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.641 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.641 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928321+0000", 2024-09-16T14:49:50.641 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0022560089999999998 2024-09-16T14:49:50.641 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.641 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.641 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:50.641 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928323+0000", 2024-09-16T14:49:50.641 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2580000000000001e-06 2024-09-16T14:49:50.641 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.641 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.641 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.641 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.944730+0000", 2024-09-16T14:49:50.641 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.016407234999999999 2024-09-16T14:49:50.641 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.642 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.642 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:50.642 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.944848+0000", 2024-09-16T14:49:50.642 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000118245 2024-09-16T14:49:50.642 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.642 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.642 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.642 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.956932+0000", 2024-09-16T14:49:50.642 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012083943 2024-09-16T14:49:50.642 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.642 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.642 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:50.642 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.957039+0000", 2024-09-16T14:49:50.642 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000107112 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.957584+0000", 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00054457099999999999 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.960280+0000", 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002696585 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.962848+0000", 2024-09-16T14:49:50.643 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025671560000000001 2024-09-16T14:49:50.644 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.644 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.644 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.644 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.963199+0000", 2024-09-16T14:49:50.644 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00035125400000000001 2024-09-16T14:49:50.644 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.644 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.644 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.644 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.644 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.644 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:762 3.5s0 3:a29e72cc:::smithi04040151-39:head [append 0~0] snapc 31=[2e,28] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:49:50.644 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.926070+0000", 2024-09-16T14:49:50.644 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 21.633244605000002, 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.037652156999999999, 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 762 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926070+0000", 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.645 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.646 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.646 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926070+0000", 2024-09-16T14:49:50.646 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.646 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.646 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.646 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926070+0000", 2024-09-16T14:49:50.646 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:50.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.646 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.646 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.646 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926076+0000", 2024-09-16T14:49:50.646 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9100000000000002e-06 2024-09-16T14:49:50.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.647 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.647 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.647 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926082+0000", 2024-09-16T14:49:50.647 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.489e-06 2024-09-16T14:49:50.647 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.647 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.647 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.647 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926087+0000", 2024-09-16T14:49:50.647 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3900000000000003e-06 2024-09-16T14:49:50.647 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.647 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.647 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.647 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928926+0000", 2024-09-16T14:49:50.647 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0028388900000000002 2024-09-16T14:49:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928928+0000", 2024-09-16T14:49:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.221e-06 2024-09-16T14:49:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.944921+0000", 2024-09-16T14:49:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015993395000000001 2024-09-16T14:49:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:50.648 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.945026+0000", 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000104676 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.958770+0000", 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013743663999999999 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.958841+0000", 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.1088999999999996e-05 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:50.649 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.959195+0000", 2024-09-16T14:49:50.650 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00035468199999999999 2024-09-16T14:49:50.650 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.650 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.650 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:50.650 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.963255+0000", 2024-09-16T14:49:50.650 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0040595680000000004 2024-09-16T14:49:50.650 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.650 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.650 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:50.650 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.963371+0000", 2024-09-16T14:49:50.650 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00011607099999999999 2024-09-16T14:49:50.650 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.650 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.651 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.651 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.963722+0000", 2024-09-16T14:49:50.651 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000351498 2024-09-16T14:49:50.651 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.651 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.651 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.651 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.651 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.651 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:787 3.cs0 3:31a6e1c2:::smithi04040151-23:head [append 0~507904] snapc 39=[39,38,37,28] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T14:49:50.651 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:32.874333+0000", 2024-09-16T14:49:50.651 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 17.684982163000001, 2024-09-16T14:49:50.651 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0086053910000000004, 2024-09-16T14:49:50.651 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.651 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 787 2024-09-16T14:49:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.874333+0000", 2024-09-16T14:49:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.652 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.874333+0000", 2024-09-16T14:49:50.653 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.653 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.653 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.653 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.653 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.874332+0000", 2024-09-16T14:49:50.653 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:50.653 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.653 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.653 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.653 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.874978+0000", 2024-09-16T14:49:50.653 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00064625499999999998 2024-09-16T14:49:50.653 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.653 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.653 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.654 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.874991+0000", 2024-09-16T14:49:50.654 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3125999999999999e-05 2024-09-16T14:49:50.654 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.654 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.654 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.654 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.874997+0000", 2024-09-16T14:49:50.654 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5999999999999997e-06 2024-09-16T14:49:50.654 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.654 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.654 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.654 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.875342+0000", 2024-09-16T14:49:50.654 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00034574399999999999 2024-09-16T14:49:50.654 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.654 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.655 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:50.655 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.875435+0000", 2024-09-16T14:49:50.655 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2326999999999999e-05 2024-09-16T14:49:50.655 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.655 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.655 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:50.655 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.876197+0000", 2024-09-16T14:49:50.655 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00076206500000000003 2024-09-16T14:49:50.655 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.655 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.655 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:50.655 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.876979+0000", 2024-09-16T14:49:50.655 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00078201699999999998 2024-09-16T14:49:50.655 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.656 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.656 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:50.656 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.882865+0000", 2024-09-16T14:49:50.656 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0058867609999999999 2024-09-16T14:49:50.656 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.656 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.656 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.656 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.882938+0000", 2024-09-16T14:49:50.656 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2671999999999998e-05 2024-09-16T14:49:50.656 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.656 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.656 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.656 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.656 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.657 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 89/65 ECSubWriteReply(tid=189, last_complete=90'73, committed=1, applied=1))", 2024-09-16T14:49:50.657 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.946058+0000", 2024-09-16T14:49:50.657 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.6132566, 2024-09-16T14:49:50.657 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011478993999999999, 2024-09-16T14:49:50.657 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.657 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:50.657 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.657 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.657 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.657 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946058+0000", 2024-09-16T14:49:50.657 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.657 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.657 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.657 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.658 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946058+0000", 2024-09-16T14:49:50.658 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.658 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.658 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.658 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.658 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946057+0000", 2024-09-16T14:49:50.658 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:50.658 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.658 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.658 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.658 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946063+0000", 2024-09-16T14:49:50.658 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.0979999999999999e-06 2024-09-16T14:49:50.658 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.658 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946076+0000", 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2495e-05 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946085+0000", 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6500000000000008e-06 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957431+0000", 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011346096999999999 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.659 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.660 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.660 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957537+0000", 2024-09-16T14:49:50.660 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000105917 2024-09-16T14:49:50.660 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.660 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.660 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.660 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.660 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.660 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 89/65 ECSubWriteReply(tid=189, last_complete=90'73, committed=1, applied=1))", 2024-09-16T14:49:50.660 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.946453+0000", 2024-09-16T14:49:50.660 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.612862415, 2024-09-16T14:49:50.660 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011123697, 2024-09-16T14:49:50.660 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.660 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:50.661 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.661 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.661 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.661 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946453+0000", 2024-09-16T14:49:50.661 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.661 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.661 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.661 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.661 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946453+0000", 2024-09-16T14:49:50.661 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.661 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.661 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.661 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.661 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946452+0000", 2024-09-16T14:49:50.662 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:50.662 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.662 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.662 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.662 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946458+0000", 2024-09-16T14:49:50.662 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7009999999999999e-06 2024-09-16T14:49:50.662 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.662 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.662 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.662 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946471+0000", 2024-09-16T14:49:50.662 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2968e-05 2024-09-16T14:49:50.662 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.662 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.662 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946481+0000", 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957563+0000", 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011082949999999999 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957576+0000", 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2758000000000001e-05 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.663 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.664 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.664 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.664 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds1 90/65 ECSubWrite(tid=311, reqid=osd.7.0:311, at_version=90'75, trim_to=0'0, roll_forward_to=89'73))", 2024-09-16T14:49:50.664 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.957718+0000", 2024-09-16T14:49:50.664 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 13.601596706, 2024-09-16T14:49:50.664 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010726197, 2024-09-16T14:49:50.664 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:50.664 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:50.664 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:50.664 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.664 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:50.664 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957718+0000", 2024-09-16T14:49:50.664 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957718+0000", 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957718+0000", 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957737+0000", 2024-09-16T14:49:50.665 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8757999999999999e-05 2024-09-16T14:49:50.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.666 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.666 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:50.666 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957746+0000", 2024-09-16T14:49:50.666 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:49:50.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.666 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.666 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:50.666 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957753+0000", 2024-09-16T14:49:50.666 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9809999999999997e-06 2024-09-16T14:49:50.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.666 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.666 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:50.666 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957967+0000", 2024-09-16T14:49:50.667 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000214026 2024-09-16T14:49:50.667 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.667 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.667 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:50.667 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957976+0000", 2024-09-16T14:49:50.667 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3249999999999997e-06 2024-09-16T14:49:50.667 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.667 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.667 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:50.667 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.968424+0000", 2024-09-16T14:49:50.667 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01044819 2024-09-16T14:49:50.667 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:50.667 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:50.667 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:50.668 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.968444+0000", 2024-09-16T14:49:50.668 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0103999999999999e-05 2024-09-16T14:49:50.668 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.668 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.668 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.668 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:50.668 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:50.668 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:50.669 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:50.669 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:50.670 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:50.670 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:50.671 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:49:50.671 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:50.671 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:50.672 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:50.672 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:50.673 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:50.668+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:50.770 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:50.770+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:50.815 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:50.815 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:50.815 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:50.815 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:50.824 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:49:50.860 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.10 to 1 (10000) 2024-09-16T14:49:50.861 INFO:tasks.rados.rados.0.smithi040.stdout:389: done (1 left) 2024-09-16T14:49:50.861 INFO:tasks.rados.rados.0.smithi040.stdout:390: done (0 left) 2024-09-16T14:49:50.861 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 48 current snap is 39 2024-09-16T14:49:50.861 INFO:tasks.rados.rados.0.smithi040.stdout:391: seq_num 137 ranges {442368=753664,1196032=786432,1982464=655360,2637824=114688} 2024-09-16T14:49:50.866 INFO:tasks.rados.rados.0.smithi040.stdout:391: writing smithi04040151-48 from 442368 to 1196032 tid 1 2024-09-16T14:49:50.871 INFO:tasks.rados.rados.0.smithi040.stdout:391: writing smithi04040151-48 from 1196032 to 1982464 tid 2 2024-09-16T14:49:50.872 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:50.871+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:50.875 INFO:tasks.rados.rados.0.smithi040.stdout:391: writing smithi04040151-48 from 1982464 to 2637824 tid 3 2024-09-16T14:49:50.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-09-16T14:49:50.877 INFO:tasks.rados.rados.0.smithi040.stdout:391: writing smithi04040151-48 from 2637824 to 2752512 tid 4 2024-09-16T14:49:50.877 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 31 current snap is 39 2024-09-16T14:49:50.877 INFO:tasks.rados.rados.0.smithi040.stdout:392: seq_num 138 ranges {4866048=770048,5636096=540672,6176768=753664,6930432=262144} 2024-09-16T14:49:50.908 INFO:tasks.rados.rados.0.smithi040.stdout:392: writing smithi04040151-31 from 4866048 to 5636096 tid 1 2024-09-16T14:49:50.911 INFO:tasks.rados.rados.0.smithi040.stdout:392: writing smithi04040151-31 from 5636096 to 6176768 tid 2 2024-09-16T14:49:50.916 INFO:tasks.rados.rados.0.smithi040.stdout:392: writing smithi04040151-31 from 6176768 to 6930432 tid 3 2024-09-16T14:49:50.918 INFO:tasks.rados.rados.0.smithi040.stdout:392: writing smithi04040151-31 from 6930432 to 7192576 tid 4 2024-09-16T14:49:50.918 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 17 current snap is 39 2024-09-16T14:49:50.918 INFO:tasks.rados.rados.0.smithi040.stdout:391: finishing write tid 1 to smithi04040151-48 2024-09-16T14:49:50.918 INFO:tasks.rados.rados.0.smithi040.stdout:393: seq_num 139 ranges {524288=786432,1310720=688128,1998848=475136,2473984=393216} 2024-09-16T14:49:50.926 INFO:tasks.rados.rados.0.smithi040.stdout:393: writing smithi04040151-17 from 524288 to 1310720 tid 1 2024-09-16T14:49:50.930 INFO:tasks.rados.rados.0.smithi040.stdout:393: writing smithi04040151-17 from 1310720 to 1998848 tid 2 2024-09-16T14:49:50.934 INFO:tasks.rados.rados.0.smithi040.stdout:393: writing smithi04040151-17 from 1998848 to 2473984 tid 3 2024-09-16T14:49:50.937 INFO:tasks.rados.rados.0.smithi040.stdout:393: writing smithi04040151-17 from 2473984 to 2867200 tid 4 2024-09-16T14:49:50.937 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 25 current snap is 39 2024-09-16T14:49:50.937 INFO:tasks.rados.rados.0.smithi040.stdout:391: finishing write tid 2 to smithi04040151-48 2024-09-16T14:49:50.937 INFO:tasks.rados.rados.0.smithi040.stdout:391: finishing write tid 3 to smithi04040151-48 2024-09-16T14:49:50.937 INFO:tasks.rados.rados.0.smithi040.stdout:391: finishing write tid 4 to smithi04040151-48 2024-09-16T14:49:50.937 INFO:tasks.rados.rados.0.smithi040.stdout:391: finishing write tid 5 to smithi04040151-48 2024-09-16T14:49:50.937 INFO:tasks.rados.rados.0.smithi040.stdout:391: finishing write tid 6 to smithi04040151-48 2024-09-16T14:49:50.938 INFO:tasks.rados.rados.0.smithi040.stdout:391: oid 48 updating version 0 to 53 2024-09-16T14:49:50.938 INFO:tasks.rados.rados.0.smithi040.stdout:391: oid 48 version 53 is already newer than 50 2024-09-16T14:49:50.938 INFO:tasks.rados.rados.0.smithi040.stdout:391: oid 48 version 53 is already newer than 52 2024-09-16T14:49:50.938 INFO:tasks.rados.rados.0.smithi040.stdout:391: oid 48 version 53 is already newer than 51 2024-09-16T14:49:50.938 INFO:tasks.rados.rados.0.smithi040.stdout:391: oid 48 updating version 53 to 54 2024-09-16T14:49:50.938 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 48 v 54 (ObjNum 137 snap 39 seq_num 137) dirty exists 2024-09-16T14:49:50.938 INFO:tasks.rados.rados.0.smithi040.stdout:391: left oid 48 (ObjNum 137 snap 39 seq_num 137) 2024-09-16T14:49:50.938 INFO:tasks.rados.rados.0.smithi040.stdout:392: finishing write tid 1 to smithi04040151-31 2024-09-16T14:49:50.938 INFO:tasks.rados.rados.0.smithi040.stdout:394: seq_num 140 ranges {1556480=802816,2359296=425984,2785280=720896,3506176=409600} 2024-09-16T14:49:50.950 INFO:tasks.rados.rados.0.smithi040.stdout:394: writing smithi04040151-25 from 1556480 to 2359296 tid 1 2024-09-16T14:49:50.952 INFO:tasks.rados.rados.0.smithi040.stdout:394: writing smithi04040151-25 from 2359296 to 2785280 tid 2 2024-09-16T14:49:50.956 INFO:tasks.rados.rados.0.smithi040.stdout:394: writing smithi04040151-25 from 2785280 to 3506176 tid 3 2024-09-16T14:49:50.959 INFO:tasks.rados.rados.0.smithi040.stdout:394: writing smithi04040151-25 from 3506176 to 3915776 tid 4 2024-09-16T14:49:50.959 INFO:tasks.rados.rados.0.smithi040.stdout:391: done (3 left) 2024-09-16T14:49:50.960 INFO:tasks.rados.rados.0.smithi040.stdout:395: snap_create 2024-09-16T14:49:50.960 INFO:tasks.rados.rados.0.smithi040.stdout:392: finishing write tid 2 to smithi04040151-31 2024-09-16T14:49:50.960 INFO:tasks.rados.rados.0.smithi040.stdout:392: finishing write tid 3 to smithi04040151-31 2024-09-16T14:49:50.960 INFO:tasks.rados.rados.0.smithi040.stdout:392: finishing write tid 4 to smithi04040151-31 2024-09-16T14:49:50.960 INFO:tasks.rados.rados.0.smithi040.stdout:392: finishing write tid 5 to smithi04040151-31 2024-09-16T14:49:50.960 INFO:tasks.rados.rados.0.smithi040.stdout:392: finishing write tid 6 to smithi04040151-31 2024-09-16T14:49:50.960 INFO:tasks.rados.rados.0.smithi040.stdout:392: oid 31 updating version 0 to 169 2024-09-16T14:49:50.960 INFO:tasks.rados.rados.0.smithi040.stdout:392: oid 31 version 169 is already newer than 168 2024-09-16T14:49:50.960 INFO:tasks.rados.rados.0.smithi040.stdout:392: oid 31 updating version 169 to 172 2024-09-16T14:49:50.960 INFO:tasks.rados.rados.0.smithi040.stdout:392: oid 31 version 172 is already newer than 170 2024-09-16T14:49:50.960 INFO:tasks.rados.rados.0.smithi040.stdout:392: oid 31 version 172 is already newer than 171 2024-09-16T14:49:50.960 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 31 v 172 (ObjNum 138 snap 39 seq_num 138) dirty exists 2024-09-16T14:49:50.960 INFO:tasks.rados.rados.0.smithi040.stdout:392: left oid 31 (ObjNum 138 snap 39 seq_num 138) 2024-09-16T14:49:50.961 INFO:tasks.rados.rados.0.smithi040.stdout:393: finishing write tid 1 to smithi04040151-17 2024-09-16T14:49:50.961 INFO:tasks.rados.rados.0.smithi040.stdout:393: finishing write tid 2 to smithi04040151-17 2024-09-16T14:49:50.961 INFO:tasks.rados.rados.0.smithi040.stdout:393: finishing write tid 3 to smithi04040151-17 2024-09-16T14:49:50.961 INFO:tasks.rados.rados.0.smithi040.stdout:393: finishing write tid 4 to smithi04040151-17 2024-09-16T14:49:50.961 INFO:tasks.rados.rados.0.smithi040.stdout:393: finishing write tid 5 to smithi04040151-17 2024-09-16T14:49:50.961 INFO:tasks.rados.rados.0.smithi040.stdout:393: finishing write tid 6 to smithi04040151-17 2024-09-16T14:49:50.961 INFO:tasks.rados.rados.0.smithi040.stdout:393: oid 17 updating version 0 to 78 2024-09-16T14:49:50.961 INFO:tasks.rados.rados.0.smithi040.stdout:393: oid 17 updating version 78 to 79 2024-09-16T14:49:50.961 INFO:tasks.rados.rados.0.smithi040.stdout:393: oid 17 version 79 is already newer than 76 2024-09-16T14:49:50.961 INFO:tasks.rados.rados.0.smithi040.stdout:393: oid 17 version 79 is already newer than 77 2024-09-16T14:49:50.961 INFO:tasks.rados.rados.0.smithi040.stdout:393: oid 17 updating version 79 to 80 2024-09-16T14:49:50.961 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 17 v 80 (ObjNum 139 snap 39 seq_num 139) dirty exists 2024-09-16T14:49:50.961 INFO:tasks.rados.rados.0.smithi040.stdout:393: left oid 17 (ObjNum 139 snap 39 seq_num 139) 2024-09-16T14:49:50.961 INFO:tasks.rados.rados.0.smithi040.stdout:394: finishing write tid 1 to smithi04040151-25 2024-09-16T14:49:50.962 INFO:tasks.rados.rados.0.smithi040.stdout:394: finishing write tid 2 to smithi04040151-25 2024-09-16T14:49:50.963 INFO:tasks.rados.rados.0.smithi040.stdout:394: finishing write tid 3 to smithi04040151-25 2024-09-16T14:49:50.964 INFO:tasks.rados.rados.0.smithi040.stdout:394: finishing write tid 4 to smithi04040151-25 2024-09-16T14:49:50.965 INFO:tasks.rados.rados.0.smithi040.stdout:394: finishing write tid 5 to smithi04040151-25 2024-09-16T14:49:50.967 INFO:tasks.rados.rados.0.smithi040.stdout:394: finishing write tid 6 to smithi04040151-25 2024-09-16T14:49:50.967 INFO:tasks.rados.rados.0.smithi040.stdout:394: oid 25 updating version 0 to 60 2024-09-16T14:49:50.967 INFO:tasks.rados.rados.0.smithi040.stdout:394: oid 25 updating version 60 to 63 2024-09-16T14:49:50.967 INFO:tasks.rados.rados.0.smithi040.stdout:394: oid 25 version 63 is already newer than 61 2024-09-16T14:49:50.967 INFO:tasks.rados.rados.0.smithi040.stdout:394: oid 25 version 63 is already newer than 62 2024-09-16T14:49:50.967 INFO:tasks.rados.rados.0.smithi040.stdout:394: oid 25 updating version 63 to 64 2024-09-16T14:49:50.967 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 25 v 64 (ObjNum 140 snap 39 seq_num 140) dirty exists 2024-09-16T14:49:50.967 INFO:tasks.rados.rados.0.smithi040.stdout:394: left oid 25 (ObjNum 140 snap 39 seq_num 140) 2024-09-16T14:49:50.973 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:50.971+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:50.978 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:50.978 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:50.978 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:50.978 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:50.987 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:49:51.073 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:51.072+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:51.091 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:51.091 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:51.091 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:51.091 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:51.092 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.092 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:49:51.092 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:51.092 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 88.949852016999998, 2024-09-16T14:49:51.092 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.223181782, 2024-09-16T14:49:51.092 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.092 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:51.092 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.092 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:51.092 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:51.092 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 1 2024-09-16T14:49:51.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.092 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.092 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.093 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.093 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:51.093 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.093 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.093 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.093 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:51.093 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.093 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.093 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.093 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139495+0000", 2024-09-16T14:49:51.093 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:51.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.094 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.094 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.094 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139504+0000", 2024-09-16T14:49:51.094 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6119999999999999e-06 2024-09-16T14:49:51.094 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.094 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.094 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.094 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139514+0000", 2024-09-16T14:49:51.094 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:49:51.094 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.094 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.094 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.094 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139533+0000", 2024-09-16T14:49:51.095 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9573e-05 2024-09-16T14:49:51.095 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.095 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.095 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.095 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139585+0000", 2024-09-16T14:49:51.095 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2170999999999999e-05 2024-09-16T14:49:51.095 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.095 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.095 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:51.095 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139594+0000", 2024-09-16T14:49:51.095 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.884e-06 2024-09-16T14:49:51.095 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.095 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.095 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.096 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359233+0000", 2024-09-16T14:49:51.096 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.21963890899999999 2024-09-16T14:49:51.096 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.096 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.096 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.096 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359437+0000", 2024-09-16T14:49:51.096 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00020374999999999999 2024-09-16T14:49:51.096 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.096 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.096 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:51.096 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359611+0000", 2024-09-16T14:49:51.096 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017445899999999999 2024-09-16T14:49:51.096 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.097 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.097 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:51.097 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.360506+0000", 2024-09-16T14:49:51.097 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00089459299999999995 2024-09-16T14:49:51.097 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.097 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.097 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:51.097 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362594+0000", 2024-09-16T14:49:51.097 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002087776 2024-09-16T14:49:51.097 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.097 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.097 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:51.097 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362636+0000", 2024-09-16T14:49:51.097 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1933e-05 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362679+0000", 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2877999999999998e-05 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:535 3.fs0 3:ff6b916c:::smithi04040151-25:head [append 0~786432] snapc 1e=[1e,1c,1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 43.157986940999997, 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013876462000000001, 2024-09-16T14:49:51.098 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.099 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:51.099 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.099 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.099 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.099 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 535 2024-09-16T14:49:51.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.099 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.099 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.099 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.099 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:51.099 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.099 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.099 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.100 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:51.100 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.100 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.100 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.100 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931360+0000", 2024-09-16T14:49:51.100 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:51.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.100 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.100 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.100 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933014+0000", 2024-09-16T14:49:51.100 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001653715 2024-09-16T14:49:51.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933028+0000", 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4239999999999999e-05 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933036+0000", 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8019999999999992e-06 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933084+0000", 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7663000000000001e-05 2024-09-16T14:49:51.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.102 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.102 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.102 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933184+0000", 2024-09-16T14:49:51.102 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000100549 2024-09-16T14:49:51.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.102 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.102 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.102 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.934324+0000", 2024-09-16T14:49:51.102 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011399229999999999 2024-09-16T14:49:51.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.102 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.102 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.102 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941715+0000", 2024-09-16T14:49:51.102 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.007391023 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.945187+0000", 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034717709999999998 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.945238+0000", 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.1459999999999999e-05 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.103 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.104 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:539 3.fs0 3:ff6b916c:::smithi04040151-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:51.104 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:51.104 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 43.150865408999998, 2024-09-16T14:49:51.104 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01551078, 2024-09-16T14:49:51.104 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.104 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:51.104 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.104 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.104 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.104 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 539 2024-09-16T14:49:51.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.104 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.104 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.104 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.105 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:51.105 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.105 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.105 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.105 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:51.105 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.105 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.105 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.105 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:51.105 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.105 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.106 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.106 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938486+0000", 2024-09-16T14:49:51.106 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7779999999999999e-06 2024-09-16T14:49:51.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.106 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.106 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.106 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938492+0000", 2024-09-16T14:49:51.106 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8120000000000002e-06 2024-09-16T14:49:51.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.106 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.106 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.106 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938495+0000", 2024-09-16T14:49:51.106 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.948e-06 2024-09-16T14:49:51.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.107 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.107 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.107 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942566+0000", 2024-09-16T14:49:51.107 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0040700839999999999 2024-09-16T14:49:51.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.107 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.107 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:51.107 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942659+0000", 2024-09-16T14:49:51.107 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3887000000000004e-05 2024-09-16T14:49:51.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.107 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.107 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.107 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.948875+0000", 2024-09-16T14:49:51.107 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0062154630000000001 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.948968+0000", 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2758999999999997e-05 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.953994+0000", 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050265580000000004 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.108 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.109 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:600 3.ds0 3:bcba132a:::smithi04040151-1:head [append 0~507904] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:51.109 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.993737+0000", 2024-09-16T14:49:51.109 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 39.095612047000003, 2024-09-16T14:49:51.109 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.008095819, 2024-09-16T14:49:51.109 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.109 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:51.109 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.109 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.109 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.109 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 600 2024-09-16T14:49:51.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.109 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.109 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.109 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.993737+0000", 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.993737+0000", 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.993735+0000", 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.110 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.111 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994263+0000", 2024-09-16T14:49:51.111 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00052718199999999995 2024-09-16T14:49:51.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.111 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.111 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.111 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994274+0000", 2024-09-16T14:49:51.111 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1535999999999999e-05 2024-09-16T14:49:51.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.111 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.111 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.111 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994279+0000", 2024-09-16T14:49:51.111 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0839999999999999e-06 2024-09-16T14:49:51.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.111 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.112 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.112 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994330+0000", 2024-09-16T14:49:51.112 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0467999999999999e-05 2024-09-16T14:49:51.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.112 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.112 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:51.112 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994332+0000", 2024-09-16T14:49:51.112 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:49:51.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.112 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.112 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.112 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980708+0000", 2024-09-16T14:49:51.112 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98637637199999995 2024-09-16T14:49:51.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.981437+0000", 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00072860299999999998 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.989211+0000", 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077741490000000002 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.989340+0000", 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00012908900000000001 2024-09-16T14:49:51.113 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.990969+0000", 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001628612 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996293+0000", 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053247440000000002 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.001434+0000", 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0051404349999999996 2024-09-16T14:49:51.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.115 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.115 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.115 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.001833+0000", 2024-09-16T14:49:51.115 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00039878500000000002 2024-09-16T14:49:51.115 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.115 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.115 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.115 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.115 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.115 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:601 3.ds0 3:bcba132a:::smithi04040151-1:head [append 0~606208] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:51.115 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.998248+0000", 2024-09-16T14:49:51.115 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 39.091100728999997, 2024-09-16T14:49:51.115 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0062125260000001, 2024-09-16T14:49:51.115 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.116 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:51.116 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.116 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.116 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.116 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 601 2024-09-16T14:49:51.116 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.116 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.116 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.116 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.116 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998248+0000", 2024-09-16T14:49:51.116 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.116 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.116 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.116 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.117 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998248+0000", 2024-09-16T14:49:51.117 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.117 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.117 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.117 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.117 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998246+0000", 2024-09-16T14:49:51.117 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:51.117 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.117 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.117 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.117 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998747+0000", 2024-09-16T14:49:51.117 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00050040399999999997 2024-09-16T14:49:51.117 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.117 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.118 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.118 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998755+0000", 2024-09-16T14:49:51.118 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3329999999999997e-06 2024-09-16T14:49:51.118 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.118 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.118 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.118 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998759+0000", 2024-09-16T14:49:51.118 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8380000000000004e-06 2024-09-16T14:49:51.118 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.118 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.118 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.118 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998846+0000", 2024-09-16T14:49:51.118 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6456e-05 2024-09-16T14:49:51.118 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.119 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.119 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:51.119 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998847+0000", 2024-09-16T14:49:51.119 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.635e-06 2024-09-16T14:49:51.119 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.119 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.119 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.119 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984175+0000", 2024-09-16T14:49:51.119 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98532833399999997 2024-09-16T14:49:51.119 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.119 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.119 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:51.119 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984375+0000", 2024-09-16T14:49:51.119 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00019923299999999999 2024-09-16T14:49:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992431+0000", 2024-09-16T14:49:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0080562129999999996 2024-09-16T14:49:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992552+0000", 2024-09-16T14:49:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000120986 2024-09-16T14:49:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.120 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.121 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.993913+0000", 2024-09-16T14:49:51.121 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001361206 2024-09-16T14:49:51.121 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.121 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.121 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.121 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996295+0000", 2024-09-16T14:49:51.121 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0023814629999999999 2024-09-16T14:49:51.121 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.121 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.121 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:51.121 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004101+0000", 2024-09-16T14:49:51.121 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0078063500000000001 2024-09-16T14:49:51.121 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.121 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.122 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.122 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004461+0000", 2024-09-16T14:49:51.122 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00035967799999999999 2024-09-16T14:49:51.122 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.122 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.122 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.122 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.122 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.122 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:602 3.ds0 3:bcba132a:::smithi04040151-1:head [append 0~557056] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:51.122 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:51.122 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 39.08728851, 2024-09-16T14:49:51.122 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0032171249999999, 2024-09-16T14:49:51.122 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.122 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:51.123 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.123 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.123 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.123 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 602 2024-09-16T14:49:51.123 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.123 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.123 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.123 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.123 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:51.123 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.123 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.123 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.123 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.123 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002547+0000", 2024-09-16T14:49:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00048700700000000002 2024-09-16T14:49:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002554+0000", 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.435e-06 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002558+0000", 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.501e-06 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002641+0000", 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2874999999999995e-05 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.125 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:51.126 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002642+0000", 2024-09-16T14:49:51.126 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6670000000000001e-06 2024-09-16T14:49:51.126 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.126 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.126 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.126 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984450+0000", 2024-09-16T14:49:51.126 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98180768500000004 2024-09-16T14:49:51.126 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.126 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.126 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:51.126 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984591+0000", 2024-09-16T14:49:51.126 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014090700000000001 2024-09-16T14:49:51.126 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.126 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.127 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.127 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996586+0000", 2024-09-16T14:49:51.127 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011994898 2024-09-16T14:49:51.127 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.127 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.127 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.127 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996695+0000", 2024-09-16T14:49:51.127 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000108962 2024-09-16T14:49:51.127 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.127 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.127 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.127 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.997804+0000", 2024-09-16T14:49:51.127 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011095090000000001 2024-09-16T14:49:51.127 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000509+0000", 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0027042440000000002 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004755+0000", 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0042461269999999997 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005277+0000", 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00052277599999999997 2024-09-16T14:49:51.128 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.129 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.129 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.129 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.129 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.129 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:603 3.ds0 3:bcba132a:::smithi04040151-1:head [setxattr _header (56) in=7b] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:51.129 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:51.129 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 39.086768921000001, 2024-09-16T14:49:51.129 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.003299656, 2024-09-16T14:49:51.129 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.129 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:51.129 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.129 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.129 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.129 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 603 2024-09-16T14:49:51.130 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.130 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.130 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.130 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.130 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:51.130 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.130 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.130 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.130 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.130 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:51.130 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.130 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.130 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.130 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002582+0000", 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.345e-06 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002586+0000", 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4710000000000001e-06 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.131 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.132 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002589+0000", 2024-09-16T14:49:51.132 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.694e-06 2024-09-16T14:49:51.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.132 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.132 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.132 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002673+0000", 2024-09-16T14:49:51.132 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4349000000000002e-05 2024-09-16T14:49:51.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.132 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.132 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:51.132 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002674+0000", 2024-09-16T14:49:51.132 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9300000000000006e-07 2024-09-16T14:49:51.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.133 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.133 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.133 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984634+0000", 2024-09-16T14:49:51.133 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98195955099999999 2024-09-16T14:49:51.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.133 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.133 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:51.133 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984742+0000", 2024-09-16T14:49:51.133 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000108491 2024-09-16T14:49:51.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.133 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.133 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.133 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.998850+0000", 2024-09-16T14:49:51.133 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014108005 2024-09-16T14:49:51.134 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.134 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.134 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.134 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.999006+0000", 2024-09-16T14:49:51.134 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015594799999999999 2024-09-16T14:49:51.134 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.134 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.134 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.134 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.999686+0000", 2024-09-16T14:49:51.134 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00067984999999999996 2024-09-16T14:49:51.134 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.134 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.134 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.134 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000622+0000", 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00093585200000000004 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005500+0000", 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048775049999999999 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005880+0000", 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00037999399999999998 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.135 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.136 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.136 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:604 3.ds0 3:bcba132a:::smithi04040151-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:51.136 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:51.136 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 39.086757378999998, 2024-09-16T14:49:51.136 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.00580336, 2024-09-16T14:49:51.136 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.136 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:51.136 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.136 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.136 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.136 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 604 2024-09-16T14:49:51.136 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.136 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.137 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.137 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.137 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:51.137 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.137 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.137 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.137 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.137 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:51.137 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.137 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.137 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.137 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.137 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:51.137 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.138 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.138 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.138 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.138 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002593+0000", 2024-09-16T14:49:51.138 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0559999999999999e-06 2024-09-16T14:49:51.138 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.138 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.138 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.138 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002597+0000", 2024-09-16T14:49:51.138 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.7349999999999998e-06 2024-09-16T14:49:51.138 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.138 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.138 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.138 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002601+0000", 2024-09-16T14:49:51.139 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6519999999999999e-06 2024-09-16T14:49:51.139 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.139 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.139 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.139 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002698+0000", 2024-09-16T14:49:51.139 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7378000000000002e-05 2024-09-16T14:49:51.139 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.139 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.139 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:51.139 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002699+0000", 2024-09-16T14:49:51.139 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.5099999999999999e-07 2024-09-16T14:49:51.139 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.139 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.139 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.140 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984866+0000", 2024-09-16T14:49:51.140 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98216700499999998 2024-09-16T14:49:51.140 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.140 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.140 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:51.140 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984953+0000", 2024-09-16T14:49:51.140 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6847000000000004e-05 2024-09-16T14:49:51.140 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.140 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.140 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.140 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000753+0000", 2024-09-16T14:49:51.140 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015800455000000001 2024-09-16T14:49:51.140 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.140 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000883+0000", 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00013054000000000001 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006482+0000", 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0055987839999999999 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006630+0000", 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014820600000000001 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.141 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.142 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.142 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.008395+0000", 2024-09-16T14:49:51.142 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0017643139999999999 2024-09-16T14:49:51.142 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.142 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.142 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.142 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.142 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.142 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:751 3.ds0 3:bcba132a:::smithi04040151-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e80)", 2024-09-16T14:49:51.142 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:27.015103+0000", 2024-09-16T14:49:51.142 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.07424563, 2024-09-16T14:49:51.142 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013724121000000001, 2024-09-16T14:49:51.142 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.142 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:51.143 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.143 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.143 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.143 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 751 2024-09-16T14:49:51.143 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.143 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.143 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.143 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.143 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015103+0000", 2024-09-16T14:49:51.143 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.143 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.144 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.144 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.144 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015103+0000", 2024-09-16T14:49:51.144 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.144 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.144 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.144 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.144 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015103+0000", 2024-09-16T14:49:51.144 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.144 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.144 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.144 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.144 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015104+0000", 2024-09-16T14:49:51.144 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.584e-06 2024-09-16T14:49:51.145 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.145 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.145 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.145 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015108+0000", 2024-09-16T14:49:51.145 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9389999999999999e-06 2024-09-16T14:49:51.145 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.145 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.145 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.145 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015111+0000", 2024-09-16T14:49:51.145 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7889999999999999e-06 2024-09-16T14:49:51.145 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.145 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.145 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.145 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.017354+0000", 2024-09-16T14:49:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0022426849999999999 2024-09-16T14:49:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.017423+0000", 2024-09-16T14:49:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9228999999999997e-05 2024-09-16T14:49:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.022608+0000", 2024-09-16T14:49:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0051849920000000002 2024-09-16T14:49:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.146 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.022679+0000", 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0444000000000005e-05 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.028827+0000", 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0061487649999999996 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:756 3.fs0 3:f17be48c:::smithi04040151-41:2e [read 0~3588096,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:27.016084+0000", 2024-09-16T14:49:51.147 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.073264380000001, 2024-09-16T14:49:51.148 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012061232, 2024-09-16T14:49:51.148 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.148 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:51.148 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.148 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.148 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.148 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 756 2024-09-16T14:49:51.148 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.148 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.148 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.148 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.148 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016084+0000", 2024-09-16T14:49:51.148 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.149 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.149 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.149 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.149 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016084+0000", 2024-09-16T14:49:51.149 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.149 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.149 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.149 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.149 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016084+0000", 2024-09-16T14:49:51.149 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:51.149 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.149 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.149 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.149 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016090+0000", 2024-09-16T14:49:51.150 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7279999999999998e-06 2024-09-16T14:49:51.150 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.150 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.150 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.150 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016099+0000", 2024-09-16T14:49:51.150 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9649999999999997e-06 2024-09-16T14:49:51.150 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.150 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.150 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.150 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016106+0000", 2024-09-16T14:49:51.150 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3670000000000002e-06 2024-09-16T14:49:51.150 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.150 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.150 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.151 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.017376+0000", 2024-09-16T14:49:51.151 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0012697279999999999 2024-09-16T14:49:51.151 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.151 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.151 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.151 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.017477+0000", 2024-09-16T14:49:51.151 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010165700000000001 2024-09-16T14:49:51.151 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.151 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.151 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.151 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.028146+0000", 2024-09-16T14:49:51.151 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010668502999999999 2024-09-16T14:49:51.151 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.151 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.152 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.152 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.152 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.152 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:757 3.fs0 3:f17be48c:::smithi04040151-41:2e [read 0~3588096,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T14:49:51.152 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:27.016111+0000", 2024-09-16T14:49:51.152 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 24.073237738, 2024-09-16T14:49:51.152 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013568108000000001, 2024-09-16T14:49:51.152 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.152 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:51.152 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.152 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.152 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.152 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 757 2024-09-16T14:49:51.153 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.153 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.153 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.153 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.153 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016111+0000", 2024-09-16T14:49:51.153 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.153 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.153 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.153 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.153 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016111+0000", 2024-09-16T14:49:51.153 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.153 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.153 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.153 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016111+0000", 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016115+0000", 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016121+0000", 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9909999999999999e-06 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.154 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016125+0000", 2024-09-16T14:49:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6600000000000003e-06 2024-09-16T14:49:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.020236+0000", 2024-09-16T14:49:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0041107019999999999 2024-09-16T14:49:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.020334+0000", 2024-09-16T14:49:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8166999999999997e-05 2024-09-16T14:49:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.029679+0000", 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0093450909999999998 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.7s2 82/27 ECSubWrite(tid=342, reqid=osd.3.0:342, at_version=82'110, trim_to=0'0, roll_forward_to=81'108))", 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.933809+0000", 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 22.155540081000002, 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013532219, 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:51.156 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.157 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.157 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.157 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933809+0000", 2024-09-16T14:49:51.157 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.157 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.157 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.157 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.157 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933809+0000", 2024-09-16T14:49:51.157 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.157 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.157 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.157 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.157 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933808+0000", 2024-09-16T14:49:51.157 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933834+0000", 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5735e-05 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933844+0000", 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0081999999999999e-05 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933851+0000", 2024-09-16T14:49:51.158 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8070000000000002e-06 2024-09-16T14:49:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934134+0000", 2024-09-16T14:49:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00028271999999999998 2024-09-16T14:49:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934143+0000", 2024-09-16T14:49:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4120000000000007e-06 2024-09-16T14:49:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.947319+0000", 2024-09-16T14:49:51.160 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013176155 2024-09-16T14:49:51.160 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.160 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.160 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.160 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.947341+0000", 2024-09-16T14:49:51.160 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1603e-05 2024-09-16T14:49:51.160 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.160 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.160 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.160 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.160 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.160 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs5 82/65 ECSubWrite(tid=345, reqid=osd.3.0:345, at_version=82'69, trim_to=0'0, roll_forward_to=81'65))", 2024-09-16T14:49:51.160 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.933907+0000", 2024-09-16T14:49:51.160 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 22.155442239999999, 2024-09-16T14:49:51.161 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012550997, 2024-09-16T14:49:51.161 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.161 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:51.161 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.161 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.161 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.161 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933907+0000", 2024-09-16T14:49:51.161 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.161 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.161 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.161 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.161 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933907+0000", 2024-09-16T14:49:51.161 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933906+0000", 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933922+0000", 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5930000000000002e-05 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933929+0000", 2024-09-16T14:49:51.162 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6730000000000003e-06 2024-09-16T14:49:51.163 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.163 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.163 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.163 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933934+0000", 2024-09-16T14:49:51.163 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:49:51.163 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.163 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.163 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.163 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.938626+0000", 2024-09-16T14:49:51.163 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0046912070000000002 2024-09-16T14:49:51.163 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.163 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.163 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.938635+0000", 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3589999999999997e-06 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.946337+0000", 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077023760000000004 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.946458+0000", 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000120277 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.164 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.7s2 82/27 ECSubWrite(tid=344, reqid=osd.3.0:344, at_version=82'112, trim_to=0'0, roll_forward_to=81'108))", 2024-09-16T14:49:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.933939+0000", 2024-09-16T14:49:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 22.155409667000001, 2024-09-16T14:49:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014103087, 2024-09-16T14:49:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933939+0000", 2024-09-16T14:49:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.165 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.166 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.166 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.166 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933939+0000", 2024-09-16T14:49:51.166 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.166 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.166 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.166 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.166 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933939+0000", 2024-09-16T14:49:51.166 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:51.166 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.166 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.166 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.166 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933954+0000", 2024-09-16T14:49:51.166 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.491e-05 2024-09-16T14:49:51.167 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.167 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.167 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.167 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933960+0000", 2024-09-16T14:49:51.167 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6109999999999999e-06 2024-09-16T14:49:51.167 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.167 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.167 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.167 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933965+0000", 2024-09-16T14:49:51.167 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9599999999999999e-06 2024-09-16T14:49:51.167 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.167 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.167 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.167 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.941466+0000", 2024-09-16T14:49:51.168 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0075007529999999998 2024-09-16T14:49:51.168 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.168 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.168 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.168 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.941478+0000", 2024-09-16T14:49:51.168 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1501999999999999e-05 2024-09-16T14:49:51.168 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.168 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.168 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.168 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.948023+0000", 2024-09-16T14:49:51.168 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0065449870000000004 2024-09-16T14:49:51.168 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.168 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.948042+0000", 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9545999999999999e-05 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:860 3.ds0 3:bd112605:::smithi04040151-44:3b [read 0~737280,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.862879+0000", 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 9.2264696799999992, 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013219919, 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:51.169 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.170 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.170 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.170 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 860 2024-09-16T14:49:51.170 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.170 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.170 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.170 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.170 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862879+0000", 2024-09-16T14:49:51.170 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.170 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.170 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.170 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.170 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862879+0000", 2024-09-16T14:49:51.170 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862877+0000", 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862886+0000", 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4190000000000002e-06 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862898+0000", 2024-09-16T14:49:51.171 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1994000000000001e-05 2024-09-16T14:49:51.172 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.172 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.172 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.172 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862906+0000", 2024-09-16T14:49:51.172 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.1459999999999996e-06 2024-09-16T14:49:51.172 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.172 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.172 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.172 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862955+0000", 2024-09-16T14:49:51.172 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9147999999999998e-05 2024-09-16T14:49:51.172 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.172 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.172 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:51.172 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862965+0000", 2024-09-16T14:49:51.173 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5519999999999993e-06 2024-09-16T14:49:51.173 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.173 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.173 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.173 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866479+0000", 2024-09-16T14:49:51.173 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0035142139999999999 2024-09-16T14:49:51.173 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.173 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.173 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.173 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866601+0000", 2024-09-16T14:49:51.173 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00012230699999999999 2024-09-16T14:49:51.173 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.173 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.173 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.174 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.876099+0000", 2024-09-16T14:49:51.174 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0094979060000000004 2024-09-16T14:49:51.174 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.174 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.174 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.174 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.174 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.174 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:861 3.ds0 3:bd112605:::smithi04040151-44:3b [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:51.174 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.862920+0000", 2024-09-16T14:49:51.174 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 9.2264292139999995, 2024-09-16T14:49:51.174 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013865529, 2024-09-16T14:49:51.174 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.174 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:51.174 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 861 2024-09-16T14:49:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862920+0000", 2024-09-16T14:49:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862920+0000", 2024-09-16T14:49:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.176 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.176 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.176 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.176 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862919+0000", 2024-09-16T14:49:51.176 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.176 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.176 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.176 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.176 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862922+0000", 2024-09-16T14:49:51.176 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1489999999999998e-06 2024-09-16T14:49:51.176 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.176 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.176 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.176 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862926+0000", 2024-09-16T14:49:51.177 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0969999999999999e-06 2024-09-16T14:49:51.177 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.177 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.177 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.177 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862929+0000", 2024-09-16T14:49:51.177 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.061e-06 2024-09-16T14:49:51.177 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.177 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.177 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.177 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863026+0000", 2024-09-16T14:49:51.177 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6824999999999995e-05 2024-09-16T14:49:51.177 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.177 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.177 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863031+0000", 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1919999999999998e-06 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866928+0000", 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0038978099999999998 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867034+0000", 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010561299999999999 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.178 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.179 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.876785+0000", 2024-09-16T14:49:51.179 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0097512020000000005 2024-09-16T14:49:51.179 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.179 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.179 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.179 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.179 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.179 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:862 3.ds0 3:bd112605:::smithi04040151-44:3b [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:51.179 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.862941+0000", 2024-09-16T14:49:51.179 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 9.2264079549999991, 2024-09-16T14:49:51.179 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014724180999999999, 2024-09-16T14:49:51.179 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.179 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:51.179 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.180 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.180 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.180 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 862 2024-09-16T14:49:51.180 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.180 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.180 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.180 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.180 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862941+0000", 2024-09-16T14:49:51.180 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.180 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.180 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.180 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.180 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862941+0000", 2024-09-16T14:49:51.180 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.181 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.181 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.181 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.181 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862941+0000", 2024-09-16T14:49:51.181 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.181 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.181 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.181 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.181 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862943+0000", 2024-09-16T14:49:51.181 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3180000000000001e-06 2024-09-16T14:49:51.181 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.181 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.181 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.181 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862946+0000", 2024-09-16T14:49:51.182 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6160000000000002e-06 2024-09-16T14:49:51.182 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.182 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.182 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.182 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862949+0000", 2024-09-16T14:49:51.182 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6400000000000001e-06 2024-09-16T14:49:51.182 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.182 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.182 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.182 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863054+0000", 2024-09-16T14:49:51.182 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010529100000000001 2024-09-16T14:49:51.182 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.182 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.182 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863061+0000", 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6590000000000001e-06 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867329+0000", 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0042681100000000003 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867443+0000", 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000113596 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.183 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.184 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.877665+0000", 2024-09-16T14:49:51.184 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01022227 2024-09-16T14:49:51.184 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.184 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.184 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.184 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.184 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.184 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:863 3.13s0 3:c9d06351:::smithi04040151-33:3b [read 0~1654784,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:51.184 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.862960+0000", 2024-09-16T14:49:51.184 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 9.2263888850000004, 2024-09-16T14:49:51.184 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013471337999999999, 2024-09-16T14:49:51.184 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.184 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:51.184 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 863 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862960+0000", 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862960+0000", 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.185 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.186 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.186 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.186 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862960+0000", 2024-09-16T14:49:51.186 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.186 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.186 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.186 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.186 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862962+0000", 2024-09-16T14:49:51.186 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4420000000000001e-06 2024-09-16T14:49:51.186 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.186 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.186 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.186 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862966+0000", 2024-09-16T14:49:51.186 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5810000000000001e-06 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862968+0000", 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7369999999999998e-06 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862999+0000", 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1031000000000002e-05 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863008+0000", 2024-09-16T14:49:51.187 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4200000000000007e-06 2024-09-16T14:49:51.188 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.188 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.188 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.188 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.865719+0000", 2024-09-16T14:49:51.188 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002710877 2024-09-16T14:49:51.188 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.188 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.188 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.188 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866147+0000", 2024-09-16T14:49:51.188 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000428124 2024-09-16T14:49:51.188 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.188 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.188 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.189 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.876431+0000", 2024-09-16T14:49:51.189 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010284422 2024-09-16T14:49:51.189 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.189 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.189 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.189 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.189 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.189 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:864 3.13s0 3:c9d06351:::smithi04040151-33:3b [read 0~1654784,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:51.189 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.863012+0000", 2024-09-16T14:49:51.189 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 9.2263367350000003, 2024-09-16T14:49:51.189 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014700533999999999, 2024-09-16T14:49:51.189 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.189 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:51.190 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.190 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.190 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.190 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 864 2024-09-16T14:49:51.190 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.190 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.190 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.190 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.190 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863012+0000", 2024-09-16T14:49:51.190 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.190 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.190 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.190 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863012+0000", 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863011+0000", 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863018+0000", 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.5679999999999996e-06 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.191 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.192 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863026+0000", 2024-09-16T14:49:51.192 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8980000000000002e-06 2024-09-16T14:49:51.192 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.192 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.192 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.192 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863032+0000", 2024-09-16T14:49:51.192 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:49:51.192 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.192 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.192 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.192 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863063+0000", 2024-09-16T14:49:51.192 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0474e-05 2024-09-16T14:49:51.192 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.192 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.193 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:51.193 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863069+0000", 2024-09-16T14:49:51.193 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:49:51.193 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.193 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.193 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.193 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866510+0000", 2024-09-16T14:49:51.193 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00344123 2024-09-16T14:49:51.193 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.193 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.193 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.193 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866610+0000", 2024-09-16T14:49:51.193 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000100385 2024-09-16T14:49:51.193 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.877713+0000", 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011102386000000001 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:865 3.13s0 3:c9d06351:::smithi04040151-33:3b [read 0~1654784,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.863041+0000", 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 9.2263078010000008, 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015790143999999999, 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:51.195 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:51.195 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:51.195 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.195 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 865 2024-09-16T14:49:51.195 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.195 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:51.195 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.195 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:51.195 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863041+0000", 2024-09-16T14:49:51.195 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.195 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.195 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.195 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:51.195 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863041+0000", 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863041+0000", 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863043+0000", 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0549999999999998e-06 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:51.196 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863047+0000", 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.941e-06 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863050+0000", 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863083+0000", 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3009999999999997e-05 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863085+0000", 2024-09-16T14:49:51.198 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8849999999999999e-06 2024-09-16T14:49:51.198 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.198 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.198 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:51.198 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867224+0000", 2024-09-16T14:49:51.198 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0041382240000000002 2024-09-16T14:49:51.198 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.198 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.198 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:51.198 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867316+0000", 2024-09-16T14:49:51.198 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2559000000000006e-05 2024-09-16T14:49:51.198 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:51.198 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:51.198 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:51.199 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.878831+0000", 2024-09-16T14:49:51.199 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011514912 2024-09-16T14:49:51.199 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.199 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.199 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.199 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:51.199 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:51.199 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:51.200 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:49:51.201 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:51.200+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:51.301 INFO:tasks.ceph.osd.8.smithi114.stderr:2024-09-16T14:49:51.300+0000 7f688cab3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 33112) UID: 0 2024-09-16T14:49:51.313 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:51.313 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:51.313 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:51.313 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:51.321 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:49:51.402 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:51.401+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:51.464 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:51.464 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:51.464 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:51.465 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:51.465 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:51.473 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:49:51.503 INFO:tasks.ceph.osd.4.smithi111.stderr:2024-09-16T14:49:51.502+0000 7f409e7ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32770) UID: 0 2024-09-16T14:49:51.576 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:51.576 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:51.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:51.576 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:51.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.577 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:295 3.9s0 3:96d2b183:::smithi04040151-29:head [copy-from ver 0] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T14:49:51.577 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:51.577 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 67.658287024000003, 2024-09-16T14:49:51.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010216596999999999, 2024-09-16T14:49:51.577 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.577 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:51.577 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:51.577 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:51.577 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.577 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 295 2024-09-16T14:49:51.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.577 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:51.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:51.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915430+0000", 2024-09-16T14:49:51.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:51.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915441+0000", 2024-09-16T14:49:51.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0789e-05 2024-09-16T14:49:51.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915462+0000", 2024-09-16T14:49:51.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0845000000000001e-05 2024-09-16T14:49:51.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915473+0000", 2024-09-16T14:49:51.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0805000000000001e-05 2024-09-16T14:49:51.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915528+0000", 2024-09-16T14:49:51.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4772999999999999e-05 2024-09-16T14:49:51.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:51.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915537+0000", 2024-09-16T14:49:51.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6700000000000006e-06 2024-09-16T14:49:51.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.919813+0000", 2024-09-16T14:49:51.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042757840000000004 2024-09-16T14:49:51.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:51.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.919907+0000", 2024-09-16T14:49:51.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504999999999993e-05 2024-09-16T14:49:51.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.582 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.582 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925649+0000", 2024-09-16T14:49:51.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057427870000000001 2024-09-16T14:49:51.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.582 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.582 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:296 3.9s0 3:96d2b183:::smithi04040151-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-16T14:49:51.582 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:51.582 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 67.658235106999996, 2024-09-16T14:49:51.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010423623, 2024-09-16T14:49:51.583 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.583 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:51.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:51.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:51.583 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.583 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 296 2024-09-16T14:49:51.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.583 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:51.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:51.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915484+0000", 2024-09-16T14:49:51.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915489+0000", 2024-09-16T14:49:51.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6079999999999998e-06 2024-09-16T14:49:51.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915496+0000", 2024-09-16T14:49:51.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7589999999999997e-06 2024-09-16T14:49:51.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915506+0000", 2024-09-16T14:49:51.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0340000000000002e-06 2024-09-16T14:49:51.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915588+0000", 2024-09-16T14:49:51.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2198000000000001e-05 2024-09-16T14:49:51.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:51.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915594+0000", 2024-09-16T14:49:51.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:49:51.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920099+0000", 2024-09-16T14:49:51.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045048700000000002 2024-09-16T14:49:51.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:51.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920147+0000", 2024-09-16T14:49:51.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8239999999999999e-05 2024-09-16T14:49:51.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925726+0000", 2024-09-16T14:49:51.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055787529999999997 2024-09-16T14:49:51.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:51.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925816+0000", 2024-09-16T14:49:51.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0185000000000005e-05 2024-09-16T14:49:51.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.588 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925908+0000", 2024-09-16T14:49:51.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1948999999999997e-05 2024-09-16T14:49:51.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.588 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.588 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as4 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:51.589 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:51.589 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 67.649680455999999, 2024-09-16T14:49:51.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088260140000000001, 2024-09-16T14:49:51.589 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.589 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:51.589 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:51.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:51.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:51.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924062+0000", 2024-09-16T14:49:51.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2892e-05 2024-09-16T14:49:51.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924070+0000", 2024-09-16T14:49:51.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8690000000000004e-06 2024-09-16T14:49:51.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924076+0000", 2024-09-16T14:49:51.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0739999999999998e-06 2024-09-16T14:49:51.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924119+0000", 2024-09-16T14:49:51.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2549999999999997e-05 2024-09-16T14:49:51.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924126+0000", 2024-09-16T14:49:51.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8960000000000003e-06 2024-09-16T14:49:51.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932835+0000", 2024-09-16T14:49:51.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087089200000000002 2024-09-16T14:49:51.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932865+0000", 2024-09-16T14:49:51.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0054000000000001e-05 2024-09-16T14:49:51.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.593 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.593 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es1 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:51.593 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:51.593 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 67.649565867999996, 2024-09-16T14:49:51.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087127249999999993, 2024-09-16T14:49:51.593 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.593 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:51.593 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:51.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:51.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:51.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924168+0000", 2024-09-16T14:49:51.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4800000000000001e-05 2024-09-16T14:49:51.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924176+0000", 2024-09-16T14:49:51.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.25e-06 2024-09-16T14:49:51.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924182+0000", 2024-09-16T14:49:51.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3640000000000004e-06 2024-09-16T14:49:51.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924229+0000", 2024-09-16T14:49:51.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6477999999999998e-05 2024-09-16T14:49:51.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924238+0000", 2024-09-16T14:49:51.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4429999999999992e-06 2024-09-16T14:49:51.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932848+0000", 2024-09-16T14:49:51.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086097910000000003 2024-09-16T14:49:51.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932867+0000", 2024-09-16T14:49:51.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8927999999999999e-05 2024-09-16T14:49:51.597 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.597 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.597 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.598 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:394 3.9s0 3:93498a0b:::smithi04040151-32:head [append 0~491520] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:51.598 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:51.598 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.704147386999999, 2024-09-16T14:49:51.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016461614999999999, 2024-09-16T14:49:51.598 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.598 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:51.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:51.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:51.598 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.598 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 394 2024-09-16T14:49:51.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.598 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:51.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:51.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869570+0000", 2024-09-16T14:49:51.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:51.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872427+0000", 2024-09-16T14:49:51.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028569070000000001 2024-09-16T14:49:51.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872450+0000", 2024-09-16T14:49:51.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2943e-05 2024-09-16T14:49:51.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872461+0000", 2024-09-16T14:49:51.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0882e-05 2024-09-16T14:49:51.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872521+0000", 2024-09-16T14:49:51.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9639000000000003e-05 2024-09-16T14:49:51.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:51.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872530+0000", 2024-09-16T14:49:51.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8810000000000002e-06 2024-09-16T14:49:51.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.878309+0000", 2024-09-16T14:49:51.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057788520000000001 2024-09-16T14:49:51.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:51.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.878434+0000", 2024-09-16T14:49:51.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012550899999999999 2024-09-16T14:49:51.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880126+0000", 2024-09-16T14:49:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016919019999999999 2024-09-16T14:49:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885887+0000", 2024-09-16T14:49:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057608959999999997 2024-09-16T14:49:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885957+0000", 2024-09-16T14:49:51.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0504000000000001e-05 2024-09-16T14:49:51.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886034+0000", 2024-09-16T14:49:51.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6661000000000003e-05 2024-09-16T14:49:51.604 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.604 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.604 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.604 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:395 3.9s0 3:93498a0b:::smithi04040151-32:head [append 0~770048] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:51.604 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:51.604 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.699419972000001, 2024-09-16T14:49:51.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.018911853999999999, 2024-09-16T14:49:51.605 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.605 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:51.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:51.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:51.605 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.605 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 395 2024-09-16T14:49:51.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.605 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:51.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:51.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874298+0000", 2024-09-16T14:49:51.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:51.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875671+0000", 2024-09-16T14:49:51.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0013727069999999999 2024-09-16T14:49:51.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875689+0000", 2024-09-16T14:49:51.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7832000000000001e-05 2024-09-16T14:49:51.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875700+0000", 2024-09-16T14:49:51.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1267000000000001e-05 2024-09-16T14:49:51.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875758+0000", 2024-09-16T14:49:51.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8060999999999999e-05 2024-09-16T14:49:51.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:51.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875766+0000", 2024-09-16T14:49:51.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9580000000000007e-06 2024-09-16T14:49:51.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.608 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881004+0000", 2024-09-16T14:49:51.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052375169999999997 2024-09-16T14:49:51.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:51.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881098+0000", 2024-09-16T14:49:51.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3875999999999994e-05 2024-09-16T14:49:51.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882500+0000", 2024-09-16T14:49:51.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014019600000000001 2024-09-16T14:49:51.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886041+0000", 2024-09-16T14:49:51.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035415210000000002 2024-09-16T14:49:51.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:51.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893136+0000", 2024-09-16T14:49:51.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070949330000000003 2024-09-16T14:49:51.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893212+0000", 2024-09-16T14:49:51.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5739000000000001e-05 2024-09-16T14:49:51.610 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.610 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.610 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.611 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:396 3.9s0 3:93498a0b:::smithi04040151-32:head [setxattr _header (57) in=7b] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:51.611 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:51.611 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.698002817000003, 2024-09-16T14:49:51.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017781597999999999, 2024-09-16T14:49:51.611 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.611 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:51.611 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:51.611 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:51.611 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.611 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 396 2024-09-16T14:49:51.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.611 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875715+0000", 2024-09-16T14:49:51.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875721+0000", 2024-09-16T14:49:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6989999999999997e-06 2024-09-16T14:49:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875728+0000", 2024-09-16T14:49:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9040000000000003e-06 2024-09-16T14:49:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875734+0000", 2024-09-16T14:49:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4519999999999996e-06 2024-09-16T14:49:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875800+0000", 2024-09-16T14:49:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6538000000000006e-05 2024-09-16T14:49:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875804+0000", 2024-09-16T14:49:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.771e-06 2024-09-16T14:49:51.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883232+0000", 2024-09-16T14:49:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074282009999999997 2024-09-16T14:49:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883323+0000", 2024-09-16T14:49:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0740000000000002e-05 2024-09-16T14:49:51.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884057+0000", 2024-09-16T14:49:51.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00073422299999999999 2024-09-16T14:49:51.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893020+0000", 2024-09-16T14:49:51.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089623210000000005 2024-09-16T14:49:51.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:51.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893403+0000", 2024-09-16T14:49:51.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00038375799999999999 2024-09-16T14:49:51.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893499+0000", 2024-09-16T14:49:51.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5284000000000005e-05 2024-09-16T14:49:51.617 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.617 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.617 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.618 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:397 3.9s0 3:93498a0b:::smithi04040151-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:51.618 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:51.618 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.697971692000003, 2024-09-16T14:49:51.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019838090999999999, 2024-09-16T14:49:51.618 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.618 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:51.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:51.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:51.618 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.618 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 397 2024-09-16T14:49:51.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.618 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:51.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:51.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:51.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875750+0000", 2024-09-16T14:49:51.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3939999999999999e-06 2024-09-16T14:49:51.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875755+0000", 2024-09-16T14:49:51.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.31e-06 2024-09-16T14:49:51.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875760+0000", 2024-09-16T14:49:51.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4769999999999997e-06 2024-09-16T14:49:51.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875824+0000", 2024-09-16T14:49:51.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3893000000000006e-05 2024-09-16T14:49:51.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:51.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875829+0000", 2024-09-16T14:49:51.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3489999999999999e-06 2024-09-16T14:49:51.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885652+0000", 2024-09-16T14:49:51.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0098232530000000005 2024-09-16T14:49:51.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:51.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885748+0000", 2024-09-16T14:49:51.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5366999999999997e-05 2024-09-16T14:49:51.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893876+0000", 2024-09-16T14:49:51.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081279479999999994 2024-09-16T14:49:51.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:51.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893967+0000", 2024-09-16T14:49:51.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0777000000000002e-05 2024-09-16T14:49:51.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895586+0000", 2024-09-16T14:49:51.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016197029999999999 2024-09-16T14:49:51.623 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.623 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.623 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.624 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(osd.3.17:4 3.9s0 3:9b721ec3:::smithi04040151-12:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:51.624 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:51.624 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 57.689749398000004, 2024-09-16T14:49:51.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011299052, 2024-09-16T14:49:51.624 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.624 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:51.624 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:51.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:51.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.625 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883969+0000", 2024-09-16T14:49:51.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:51.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.625 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883976+0000", 2024-09-16T14:49:51.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3730000000000004e-06 2024-09-16T14:49:51.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883999+0000", 2024-09-16T14:49:51.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3390000000000001e-05 2024-09-16T14:49:51.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884007+0000", 2024-09-16T14:49:51.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1489999999999997e-06 2024-09-16T14:49:51.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886141+0000", 2024-09-16T14:49:51.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0021347240000000002 2024-09-16T14:49:51.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:51.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886230+0000", 2024-09-16T14:49:51.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8234999999999998e-05 2024-09-16T14:49:51.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895269+0000", 2024-09-16T14:49:51.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090399230000000001 2024-09-16T14:49:51.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.628 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.628 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.628 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:440 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:51.628 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:51.628 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 53.697696077000003, 2024-09-16T14:49:51.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01253772, 2024-09-16T14:49:51.628 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.628 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:51.628 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:51.628 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:51.629 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.629 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 440 2024-09-16T14:49:51.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.629 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:51.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:51.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876022+0000", 2024-09-16T14:49:51.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:51.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876033+0000", 2024-09-16T14:49:51.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1474000000000001e-05 2024-09-16T14:49:51.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876051+0000", 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7628000000000001e-05 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876060+0000", 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339999999999996e-06 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876128+0000", 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7460999999999997e-05 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:51.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876138+0000", 2024-09-16T14:49:51.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8649999999999996e-06 2024-09-16T14:49:51.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882141+0000", 2024-09-16T14:49:51.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060036550000000001 2024-09-16T14:49:51.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:51.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882233+0000", 2024-09-16T14:49:51.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1626000000000007e-05 2024-09-16T14:49:51.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.888561+0000", 2024-09-16T14:49:51.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063284769999999999 2024-09-16T14:49:51.633 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.633 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.633 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.633 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:441 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:51.633 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:51.633 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 53.69764902, 2024-09-16T14:49:51.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.013066019999999999, 2024-09-16T14:49:51.633 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.633 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:51.633 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:51.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:51.634 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.634 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 441 2024-09-16T14:49:51.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.634 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:51.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:51.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876070+0000", 2024-09-16T14:49:51.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876074+0000", 2024-09-16T14:49:51.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1330000000000001e-06 2024-09-16T14:49:51.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876083+0000", 2024-09-16T14:49:51.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2519999999999999e-06 2024-09-16T14:49:51.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876088+0000", 2024-09-16T14:49:51.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5509999999999999e-06 2024-09-16T14:49:51.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876235+0000", 2024-09-16T14:49:51.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014688900000000001 2024-09-16T14:49:51.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:51.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876243+0000", 2024-09-16T14:49:51.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5490000000000001e-06 2024-09-16T14:49:51.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.884127+0000", 2024-09-16T14:49:51.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078848489999999993 2024-09-16T14:49:51.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:51.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.884238+0000", 2024-09-16T14:49:51.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111007 2024-09-16T14:49:51.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889137+0000", 2024-09-16T14:49:51.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048983739999999996 2024-09-16T14:49:51.638 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.638 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.638 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.638 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:442 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:51.638 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:51.638 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 53.697610361999999, 2024-09-16T14:49:51.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014710668, 2024-09-16T14:49:51.638 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.639 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:51.639 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:51.639 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:51.639 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.639 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 442 2024-09-16T14:49:51.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.639 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:51.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:51.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:51.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876113+0000", 2024-09-16T14:49:51.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0759999999999996e-06 2024-09-16T14:49:51.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876119+0000", 2024-09-16T14:49:51.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3339999999999998e-06 2024-09-16T14:49:51.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876126+0000", 2024-09-16T14:49:51.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8410000000000002e-06 2024-09-16T14:49:51.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876273+0000", 2024-09-16T14:49:51.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014714900000000001 2024-09-16T14:49:51.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:51.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876279+0000", 2024-09-16T14:49:51.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7819999999999999e-06 2024-09-16T14:49:51.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.885397+0000", 2024-09-16T14:49:51.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091178149999999996 2024-09-16T14:49:51.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:51.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.885492+0000", 2024-09-16T14:49:51.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5384999999999996e-05 2024-09-16T14:49:51.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.890820+0000", 2024-09-16T14:49:51.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053278559999999997 2024-09-16T14:49:51.643 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.643 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.643 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.643 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs3 61/27 ECSubWrite(tid=238, reqid=osd.3.0:238, at_version=61'42, trim_to=0'0, roll_forward_to=60'38))", 2024-09-16T14:49:51.643 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:51.644 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 43.646851578000003, 2024-09-16T14:49:51.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089143629999999998, 2024-09-16T14:49:51.644 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.644 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:51.644 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:51.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:51.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:51.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926896+0000", 2024-09-16T14:49:51.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7764999999999999e-05 2024-09-16T14:49:51.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926909+0000", 2024-09-16T14:49:51.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3454e-05 2024-09-16T14:49:51.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926918+0000", 2024-09-16T14:49:51.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7099999999999996e-06 2024-09-16T14:49:51.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.646 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.646 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926968+0000", 2024-09-16T14:49:51.646 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0778000000000001e-05 2024-09-16T14:49:51.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.646 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926989+0000", 2024-09-16T14:49:51.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0069999999999999e-05 2024-09-16T14:49:51.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935750+0000", 2024-09-16T14:49:51.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.008761876 2024-09-16T14:49:51.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.647 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.647 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935783+0000", 2024-09-16T14:49:51.647 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2202000000000001e-05 2024-09-16T14:49:51.647 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.648 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.648 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.648 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s6 61/27 ECSubWrite(tid=239, reqid=osd.3.0:239, at_version=61'79, trim_to=0'0, roll_forward_to=60'75))", 2024-09-16T14:49:51.648 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:51.648 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 43.646788780999998, 2024-09-16T14:49:51.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.009451849, 2024-09-16T14:49:51.648 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.648 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:51.648 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.648 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:51.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:51.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.649 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.649 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:51.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.649 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926951+0000", 2024-09-16T14:49:51.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9959999999999999e-05 2024-09-16T14:49:51.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926962+0000", 2024-09-16T14:49:51.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1498000000000001e-05 2024-09-16T14:49:51.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926970+0000", 2024-09-16T14:49:51.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5410000000000003e-06 2024-09-16T14:49:51.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.927857+0000", 2024-09-16T14:49:51.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00088758200000000004 2024-09-16T14:49:51.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.927868+0000", 2024-09-16T14:49:51.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1234e-05 2024-09-16T14:49:51.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936355+0000", 2024-09-16T14:49:51.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0084864520000000002 2024-09-16T14:49:51.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936383+0000", 2024-09-16T14:49:51.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8068e-05 2024-09-16T14:49:51.652 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.652 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.652 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.652 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:709 3.9s0 3:96d2b183:::smithi04040151-29:head [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:51.652 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.900189+0000", 2024-09-16T14:49:51.652 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 27.673531307000001, 2024-09-16T14:49:51.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011070020999999999, 2024-09-16T14:49:51.653 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.653 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:51.653 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:51.653 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:51.653 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.653 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 709 2024-09-16T14:49:51.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.653 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900189+0000", 2024-09-16T14:49:51.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900189+0000", 2024-09-16T14:49:51.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900188+0000", 2024-09-16T14:49:51.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900192+0000", 2024-09-16T14:49:51.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7670000000000001e-06 2024-09-16T14:49:51.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900196+0000", 2024-09-16T14:49:51.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5480000000000003e-06 2024-09-16T14:49:51.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900201+0000", 2024-09-16T14:49:51.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5310000000000003e-06 2024-09-16T14:49:51.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900321+0000", 2024-09-16T14:49:51.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120096 2024-09-16T14:49:51.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:51.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900327+0000", 2024-09-16T14:49:51.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3389999999999998e-06 2024-09-16T14:49:51.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.907572+0000", 2024-09-16T14:49:51.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0072446180000000004 2024-09-16T14:49:51.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.907672+0000", 2024-09-16T14:49:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9617000000000005e-05 2024-09-16T14:49:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.911259+0000", 2024-09-16T14:49:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035870350000000001 2024-09-16T14:49:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs3 81/65 ECSubWrite(tid=337, reqid=osd.3.0:337, at_version=81'65, trim_to=0'0, roll_forward_to=80'61))", 2024-09-16T14:49:51.657 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:27.927767+0000", 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 23.645953292000002, 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088756419999999996, 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927767+0000", 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927767+0000", 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927766+0000", 2024-09-16T14:49:51.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:51.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927792+0000", 2024-09-16T14:49:51.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5914999999999998e-05 2024-09-16T14:49:51.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.659 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.659 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927809+0000", 2024-09-16T14:49:51.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7926000000000001e-05 2024-09-16T14:49:51.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927820+0000", 2024-09-16T14:49:51.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0277e-05 2024-09-16T14:49:51.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930629+0000", 2024-09-16T14:49:51.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00280919 2024-09-16T14:49:51.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930637+0000", 2024-09-16T14:49:51.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3000000000000002e-06 2024-09-16T14:49:51.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.936614+0000", 2024-09-16T14:49:51.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059770400000000003 2024-09-16T14:49:51.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.936642+0000", 2024-09-16T14:49:51.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7911000000000001e-05 2024-09-16T14:49:51.661 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.661 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.661 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:822 3.19s0 3:9b72282b:::smithi04040151-27:head [copy-from ver 52] snapc 3b=[3b,3a,38,37] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.944334+0000", 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.629385364999999, 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.073023423, 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 822 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944334+0000", 2024-09-16T14:49:51.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944334+0000", 2024-09-16T14:49:51.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944332+0000", 2024-09-16T14:49:51.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:51.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944343+0000", 2024-09-16T14:49:51.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0324e-05 2024-09-16T14:49:51.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944361+0000", 2024-09-16T14:49:51.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8354000000000001e-05 2024-09-16T14:49:51.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944372+0000", 2024-09-16T14:49:51.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1123e-05 2024-09-16T14:49:51.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958524+0000", 2024-09-16T14:49:51.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014151403999999999 2024-09-16T14:49:51.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:51.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958527+0000", 2024-09-16T14:49:51.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.101e-06 2024-09-16T14:49:51.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.970154+0000", 2024-09-16T14:49:51.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0116273200000001 2024-09-16T14:49:51.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:51.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.970333+0000", 2024-09-16T14:49:51.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017934799999999999 2024-09-16T14:49:51.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.666 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.666 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.005943+0000", 2024-09-16T14:49:51.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035610044 2024-09-16T14:49:51.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.014246+0000", 2024-09-16T14:49:51.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083026430000000002 2024-09-16T14:49:51.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:51.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017305+0000", 2024-09-16T14:49:51.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003059291 2024-09-16T14:49:51.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017358+0000", 2024-09-16T14:49:51.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2552000000000001e-05 2024-09-16T14:49:51.668 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.668 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.668 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.668 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:823 3.19s0 3:9b72282b:::smithi04040151-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90)", 2024-09-16T14:49:51.668 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:51.668 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.629333368999999, 2024-09-16T14:49:51.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0666982119999999, 2024-09-16T14:49:51.668 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.668 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:51.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:51.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:51.668 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:51.669 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 823 2024-09-16T14:49:51.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.669 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:51.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:51.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944389+0000", 2024-09-16T14:49:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9960000000000002e-06 2024-09-16T14:49:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944399+0000", 2024-09-16T14:49:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8179999999999999e-06 2024-09-16T14:49:51.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944407+0000", 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.456e-06 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958549+0000", 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014142050999999999 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958550+0000", 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1200000000000002e-07 2024-09-16T14:49:51.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971413+0000", 2024-09-16T14:49:51.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.012862798 2024-09-16T14:49:51.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:51.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971466+0000", 2024-09-16T14:49:51.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3282999999999998e-05 2024-09-16T14:49:51.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010860+0000", 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.039393862000000002 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010968+0000", 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108378 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.011085+0000", 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116402 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.674 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es1 90/27 ECSubWrite(tid=301, reqid=osd.11.0:301, at_version=90'62, trim_to=0'0, roll_forward_to=83'60))", 2024-09-16T14:49:51.674 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.956242+0000", 2024-09-16T14:49:51.674 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.617478318, 2024-09-16T14:49:51.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083679859999999991, 2024-09-16T14:49:51.674 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.674 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:51.674 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.674 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956242+0000", 2024-09-16T14:49:51.674 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956242+0000", 2024-09-16T14:49:51.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956240+0000", 2024-09-16T14:49:51.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:51.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956267+0000", 2024-09-16T14:49:51.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6101999999999998e-05 2024-09-16T14:49:51.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956279+0000", 2024-09-16T14:49:51.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2118e-05 2024-09-16T14:49:51.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956290+0000", 2024-09-16T14:49:51.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1311000000000001e-05 2024-09-16T14:49:51.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956492+0000", 2024-09-16T14:49:51.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000201795 2024-09-16T14:49:51.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956501+0000", 2024-09-16T14:49:51.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1060000000000006e-06 2024-09-16T14:49:51.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.964580+0000", 2024-09-16T14:49:51.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0080795469999999994 2024-09-16T14:49:51.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.964609+0000", 2024-09-16T14:49:51.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9043e-05 2024-09-16T14:49:51.678 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.678 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.678 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.678 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.18s4 99/92 ECSubWrite(tid=441, reqid=client.4500.0:905, at_version=99'119, trim_to=0'0, roll_forward_to=98'116))", 2024-09-16T14:49:51.678 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:45.876953+0000", 2024-09-16T14:49:51.678 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 5.6967666599999998, 2024-09-16T14:49:51.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083132399999999995, 2024-09-16T14:49:51.678 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:51.678 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:51.678 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:51.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876953+0000", 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876953+0000", 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876952+0000", 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:51.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876973+0000", 2024-09-16T14:49:51.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0709000000000001e-05 2024-09-16T14:49:51.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:51.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876995+0000", 2024-09-16T14:49:51.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1634e-05 2024-09-16T14:49:51.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:51.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.877003+0000", 2024-09-16T14:49:51.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1270000000000003e-06 2024-09-16T14:49:51.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:51.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.877447+0000", 2024-09-16T14:49:51.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044436600000000002 2024-09-16T14:49:51.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:51.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.877459+0000", 2024-09-16T14:49:51.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1367e-05 2024-09-16T14:49:51.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.681 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:51.681 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.885241+0000", 2024-09-16T14:49:51.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007781924 2024-09-16T14:49:51.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:51.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:51.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:51.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.885266+0000", 2024-09-16T14:49:51.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5795999999999998e-05 2024-09-16T14:49:51.682 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.682 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.682 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.682 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:51.682 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:51.682 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:51.684 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:49:51.684 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:51.682+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:51.785 INFO:tasks.ceph.osd.7.smithi111.stderr:2024-09-16T14:49:51.784+0000 7fbcdc156640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32776) UID: 0 2024-09-16T14:49:51.786 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:51.786 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:51.786 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:51.786 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:51.795 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:49:51.865 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.11 to 1 (10000) 2024-09-16T14:49:51.866 INFO:tasks.rados.rados.0.smithi040.stdout:392: done (3 left) 2024-09-16T14:49:51.866 INFO:tasks.rados.rados.0.smithi040.stdout:393: done (2 left) 2024-09-16T14:49:51.866 INFO:tasks.rados.rados.0.smithi040.stdout:394: done (1 left) 2024-09-16T14:49:51.866 INFO:tasks.rados.rados.0.smithi040.stdout:395: done (0 left) 2024-09-16T14:49:51.866 INFO:tasks.rados.rados.0.smithi040.stdout:396: rmattr oid 48 current snap is 40 2024-09-16T14:49:51.867 INFO:tasks.rados.rados.0.smithi040.stdout:396: done (0 left) 2024-09-16T14:49:51.867 INFO:tasks.rados.rados.0.smithi040.stdout:append oid 27 current snap is 40 2024-09-16T14:49:51.867 INFO:tasks.rados.rados.0.smithi040.stdout:397: seq_num 141 ranges {4866048=425984,5292032=573440,5865472=425984,6291456=524288,6815744=425984} 2024-09-16T14:49:51.874 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:49:51.882 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T14:49:51.895 INFO:tasks.rados.rados.0.smithi040.stdout:397: writing smithi04040151-27 from 4866048 to 5292032 tid 1 2024-09-16T14:49:51.895 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:51.894+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:51.898 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:51.898 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:51.898 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:51.898 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:51.898 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:51.899 INFO:tasks.rados.rados.0.smithi040.stdout:397: writing smithi04040151-27 from 5292032 to 5865472 tid 2 2024-09-16T14:49:51.902 INFO:tasks.rados.rados.0.smithi040.stdout:397: writing smithi04040151-27 from 5865472 to 6291456 tid 3 2024-09-16T14:49:51.904 INFO:tasks.rados.rados.0.smithi040.stdout:397: writing smithi04040151-27 from 6291456 to 6815744 tid 4 2024-09-16T14:49:51.907 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:49:51.909 INFO:tasks.rados.rados.0.smithi040.stdout:397: writing smithi04040151-27 from 6815744 to 7241728 tid 5 2024-09-16T14:49:51.909 INFO:tasks.rados.rados.0.smithi040.stdout:398: read oid 7 snap 37 2024-09-16T14:49:51.909 INFO:tasks.rados.rados.0.smithi040.stdout:398: expect (ObjNum 109 snap 24 seq_num 109) 2024-09-16T14:49:51.909 INFO:tasks.rados.rados.0.smithi040.stdout:399: snap_remove snap 39 2024-09-16T14:49:51.986 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:51.985+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:52.016 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:52.016 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:52.016 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:52.016 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:52.017 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.017 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:49:52.017 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:52.017 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 71.103816519000006, 2024-09-16T14:49:52.018 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007542137, 2024-09-16T14:49:52.018 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.018 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.018 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.018 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.018 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.018 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:52.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.019 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.019 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.019 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:52.019 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.020 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:52.020 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.020 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.020 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909917+0000", 2024-09-16T14:49:52.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6513000000000001e-05 2024-09-16T14:49:52.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.021 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:49:52.021 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:49:52.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.021 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.021 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909941+0000", 2024-09-16T14:49:52.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1859999999999992e-06 2024-09-16T14:49:52.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910004+0000", 2024-09-16T14:49:52.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2706999999999994e-05 2024-09-16T14:49:52.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.022 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.022 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.022 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910012+0000", 2024-09-16T14:49:52.022 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.157e-06 2024-09-16T14:49:52.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917395+0000", 2024-09-16T14:49:52.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073827659999999998 2024-09-16T14:49:52.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.023 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.023 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.023 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917423+0000", 2024-09-16T14:49:52.023 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8568000000000002e-05 2024-09-16T14:49:52.023 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.023 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.023 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.024 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=37'32, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:52.024 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:52.024 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.092706175000004, 2024-09-16T14:49:52.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070991120000000003, 2024-09-16T14:49:52.024 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.024 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.024 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.024 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.024 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.024 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:52.024 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:52.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.025 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.025 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.025 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920991+0000", 2024-09-16T14:49:52.025 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921010+0000", 2024-09-16T14:49:52.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8529000000000002e-05 2024-09-16T14:49:52.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.026 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.026 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.026 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921018+0000", 2024-09-16T14:49:52.026 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4670000000000004e-06 2024-09-16T14:49:52.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921025+0000", 2024-09-16T14:49:52.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9330000000000004e-06 2024-09-16T14:49:52.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.027 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921076+0000", 2024-09-16T14:49:52.027 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0662999999999999e-05 2024-09-16T14:49:52.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.027 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.027 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921084+0000", 2024-09-16T14:49:52.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4780000000000008e-06 2024-09-16T14:49:52.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928073+0000", 2024-09-16T14:49:52.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069883940000000002 2024-09-16T14:49:52.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.028 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.028 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.028 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928091+0000", 2024-09-16T14:49:52.028 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7992999999999999e-05 2024-09-16T14:49:52.029 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.029 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.029 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.029 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:52.029 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:52.029 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.088497805000003, 2024-09-16T14:49:52.029 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079457759999999999, 2024-09-16T14:49:52.030 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.030 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.030 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:52.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.030 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.030 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.030 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:52.030 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:52.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.031 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.031 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.031 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925222+0000", 2024-09-16T14:49:52.031 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1871000000000001e-05 2024-09-16T14:49:52.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925238+0000", 2024-09-16T14:49:52.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6719999999999999e-05 2024-09-16T14:49:52.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925244+0000", 2024-09-16T14:49:52.032 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3630000000000002e-06 2024-09-16T14:49:52.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.032 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.032 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.032 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925291+0000", 2024-09-16T14:49:52.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6944000000000003e-05 2024-09-16T14:49:52.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925299+0000", 2024-09-16T14:49:52.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:49:52.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.033 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.033 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933129+0000", 2024-09-16T14:49:52.033 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078297330000000002 2024-09-16T14:49:52.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.033 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.034 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.034 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933146+0000", 2024-09-16T14:49:52.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.664e-05 2024-09-16T14:49:52.034 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.034 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.034 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.034 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.034 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es5 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:52.034 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:52.034 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.088446169999997, 2024-09-16T14:49:52.034 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078358790000000005, 2024-09-16T14:49:52.034 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.035 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.035 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:52.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.035 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.035 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.035 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:52.035 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925251+0000", 2024-09-16T14:49:52.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.036 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925265+0000", 2024-09-16T14:49:52.036 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4078e-05 2024-09-16T14:49:52.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.036 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.036 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925273+0000", 2024-09-16T14:49:52.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1069999999999999e-06 2024-09-16T14:49:52.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925279+0000", 2024-09-16T14:49:52.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4409999999999998e-06 2024-09-16T14:49:52.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.037 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.037 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.037 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925330+0000", 2024-09-16T14:49:52.037 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1208999999999997e-05 2024-09-16T14:49:52.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925339+0000", 2024-09-16T14:49:52.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8109999999999992e-06 2024-09-16T14:49:52.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.038 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933064+0000", 2024-09-16T14:49:52.038 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077252149999999997 2024-09-16T14:49:52.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.038 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.039 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933088+0000", 2024-09-16T14:49:52.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3317999999999999e-05 2024-09-16T14:49:52.039 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.039 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.039 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.039 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.039 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:52.039 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933425+0000", 2024-09-16T14:49:52.039 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.080272991999998, 2024-09-16T14:49:52.039 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090001820000000007, 2024-09-16T14:49:52.039 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933425+0000", 2024-09-16T14:49:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933425+0000", 2024-09-16T14:49:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933424+0000", 2024-09-16T14:49:52.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933478+0000", 2024-09-16T14:49:52.041 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3735e-05 2024-09-16T14:49:52.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.041 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.041 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.041 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933493+0000", 2024-09-16T14:49:52.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4978e-05 2024-09-16T14:49:52.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933502+0000", 2024-09-16T14:49:52.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6509999999999996e-06 2024-09-16T14:49:52.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.042 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933761+0000", 2024-09-16T14:49:52.042 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025924500000000002 2024-09-16T14:49:52.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.042 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.042 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933772+0000", 2024-09-16T14:49:52.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0694e-05 2024-09-16T14:49:52.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942409+0000", 2024-09-16T14:49:52.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086371539999999993 2024-09-16T14:49:52.043 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.043 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.043 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.043 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942425+0000", 2024-09-16T14:49:52.043 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5781e-05 2024-09-16T14:49:52.043 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.044 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.044 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.044 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.cs4 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:52.044 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:52.044 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.079911385999999, 2024-09-16T14:49:52.044 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086187320000000005, 2024-09-16T14:49:52.044 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.044 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.044 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.044 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.044 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.044 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:52.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:52.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.045 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.045 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:52.045 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.045 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933807+0000", 2024-09-16T14:49:52.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1107e-05 2024-09-16T14:49:52.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933817+0000", 2024-09-16T14:49:52.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8870000000000004e-06 2024-09-16T14:49:52.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.046 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.046 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.046 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933826+0000", 2024-09-16T14:49:52.046 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5319999999999995e-06 2024-09-16T14:49:52.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933899+0000", 2024-09-16T14:49:52.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2294999999999998e-05 2024-09-16T14:49:52.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933908+0000", 2024-09-16T14:49:52.047 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4110000000000002e-06 2024-09-16T14:49:52.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.047 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.047 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.047 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942383+0000", 2024-09-16T14:49:52.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0084747099999999999 2024-09-16T14:49:52.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.048 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.048 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.048 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942405+0000", 2024-09-16T14:49:52.048 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2399e-05 2024-09-16T14:49:52.048 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.048 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.048 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.049 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:52.049 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.934497+0000", 2024-09-16T14:49:52.049 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.079201232999999, 2024-09-16T14:49:52.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078697109999999997, 2024-09-16T14:49:52.049 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.049 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.049 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.049 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.049 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.049 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934497+0000", 2024-09-16T14:49:52.049 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934497+0000", 2024-09-16T14:49:52.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.050 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.050 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.050 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934496+0000", 2024-09-16T14:49:52.050 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.051 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935463+0000", 2024-09-16T14:49:52.051 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00096685900000000003 2024-09-16T14:49:52.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.051 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.051 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935498+0000", 2024-09-16T14:49:52.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5179000000000003e-05 2024-09-16T14:49:52.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.052 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.052 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935509+0000", 2024-09-16T14:49:52.052 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1223000000000001e-05 2024-09-16T14:49:52.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.052 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935573+0000", 2024-09-16T14:49:52.053 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4114000000000003e-05 2024-09-16T14:49:52.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.053 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.053 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.053 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935583+0000", 2024-09-16T14:49:52.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.008e-05 2024-09-16T14:49:52.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.054 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.054 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942344+0000", 2024-09-16T14:49:52.054 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067610760000000004 2024-09-16T14:49:52.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.055 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.055 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.055 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942366+0000", 2024-09-16T14:49:52.055 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2059e-05 2024-09-16T14:49:52.055 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.055 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.056 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.056 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.056 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 61/27 ECSubWrite(tid=242, reqid=osd.3.0:242, at_version=61'79, trim_to=0'0, roll_forward_to=60'77))", 2024-09-16T14:49:52.056 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.944331+0000", 2024-09-16T14:49:52.056 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.069367102999998, 2024-09-16T14:49:52.057 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083214770000000007, 2024-09-16T14:49:52.057 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.057 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.057 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.058 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.058 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944331+0000", 2024-09-16T14:49:52.058 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.058 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.058 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.059 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944331+0000", 2024-09-16T14:49:52.059 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.059 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.059 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.060 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944330+0000", 2024-09-16T14:49:52.060 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.060 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.060 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.061 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944354+0000", 2024-09-16T14:49:52.061 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3702000000000001e-05 2024-09-16T14:49:52.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.062 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.062 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944367+0000", 2024-09-16T14:49:52.062 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3485999999999999e-05 2024-09-16T14:49:52.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.062 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.063 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.063 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944375+0000", 2024-09-16T14:49:52.063 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3219999999999993e-06 2024-09-16T14:49:52.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.064 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.064 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.064 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944425+0000", 2024-09-16T14:49:52.064 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9289000000000002e-05 2024-09-16T14:49:52.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.065 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.065 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.065 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944436+0000", 2024-09-16T14:49:52.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1381e-05 2024-09-16T14:49:52.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.066 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.066 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.066 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.952629+0000", 2024-09-16T14:49:52.066 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081932840000000003 2024-09-16T14:49:52.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.067 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.067 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.952652+0000", 2024-09-16T14:49:52.067 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.296e-05 2024-09-16T14:49:52.067 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.068 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.068 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.068 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.068 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 77/27 ECSubWrite(tid=311, reqid=osd.3.0:311, at_version=77'100, trim_to=0'0, roll_forward_to=76'98))", 2024-09-16T14:49:52.069 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.915451+0000", 2024-09-16T14:49:52.069 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.098247139000001, 2024-09-16T14:49:52.069 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079661820000000005, 2024-09-16T14:49:52.069 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.069 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.070 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.070 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.070 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.070 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915451+0000", 2024-09-16T14:49:52.070 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.070 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.071 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.071 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915451+0000", 2024-09-16T14:49:52.071 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.071 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.072 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.072 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915450+0000", 2024-09-16T14:49:52.072 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.072 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.073 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915534+0000", 2024-09-16T14:49:52.073 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4160999999999996e-05 2024-09-16T14:49:52.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.073 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.073 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.074 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915547+0000", 2024-09-16T14:49:52.074 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2764e-05 2024-09-16T14:49:52.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.074 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.074 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915555+0000", 2024-09-16T14:49:52.075 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2090000000000005e-06 2024-09-16T14:49:52.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.075 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.075 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.075 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915659+0000", 2024-09-16T14:49:52.076 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103879 2024-09-16T14:49:52.076 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.076 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.076 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915685+0000", 2024-09-16T14:49:52.077 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5959999999999999e-05 2024-09-16T14:49:52.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.077 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.077 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.077 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923392+0000", 2024-09-16T14:49:52.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077072379999999999 2024-09-16T14:49:52.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.078 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.078 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.078 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923417+0000", 2024-09-16T14:49:52.078 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5000000000000001e-05 2024-09-16T14:49:52.079 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.079 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.079 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.079 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.080 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.0s1 77/65 ECSubWrite(tid=279, reqid=osd.11.0:279, at_version=77'35, trim_to=0'0, roll_forward_to=74'31))", 2024-09-16T14:49:52.080 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.916327+0000", 2024-09-16T14:49:52.080 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.097371076999998, 2024-09-16T14:49:52.080 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070483610000000004, 2024-09-16T14:49:52.080 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.080 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.081 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.081 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.081 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.081 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916327+0000", 2024-09-16T14:49:52.081 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.082 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.082 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.082 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916327+0000", 2024-09-16T14:49:52.082 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916326+0000", 2024-09-16T14:49:52.083 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.083 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.083 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.083 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916348+0000", 2024-09-16T14:49:52.084 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2327000000000002e-05 2024-09-16T14:49:52.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.084 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.084 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.084 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916363+0000", 2024-09-16T14:49:52.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4295e-05 2024-09-16T14:49:52.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.085 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.085 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.085 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916371+0000", 2024-09-16T14:49:52.085 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9519999999999994e-06 2024-09-16T14:49:52.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916417+0000", 2024-09-16T14:49:52.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6279000000000002e-05 2024-09-16T14:49:52.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916427+0000", 2024-09-16T14:49:52.086 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0164e-05 2024-09-16T14:49:52.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.086 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.086 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.086 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923355+0000", 2024-09-16T14:49:52.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069278569999999999 2024-09-16T14:49:52.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.087 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.087 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923375+0000", 2024-09-16T14:49:52.087 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0341999999999999e-05 2024-09-16T14:49:52.087 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.087 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.087 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.087 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.087 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 77/72 ECSubWrite(tid=14, reqid=client.4500.0:702, at_version=77'33, trim_to=0'0, roll_forward_to=77'32))", 2024-09-16T14:49:52.087 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:52.088 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.096007376999999, 2024-09-16T14:49:52.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070951110000000003, 2024-09-16T14:49:52.088 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.088 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.088 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.088 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:52.088 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.088 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:52.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:52.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.089 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.089 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.089 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918213+0000", 2024-09-16T14:49:52.089 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000523721 2024-09-16T14:49:52.089 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918232+0000", 2024-09-16T14:49:52.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8743999999999999e-05 2024-09-16T14:49:52.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.090 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.090 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.090 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918241+0000", 2024-09-16T14:49:52.090 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9539999999999993e-06 2024-09-16T14:49:52.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918356+0000", 2024-09-16T14:49:52.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011514999999999999 2024-09-16T14:49:52.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918367+0000", 2024-09-16T14:49:52.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0933e-05 2024-09-16T14:49:52.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.091 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.091 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.091 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.924758+0000", 2024-09-16T14:49:52.091 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063913499999999996 2024-09-16T14:49:52.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.092 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.092 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.924786+0000", 2024-09-16T14:49:52.092 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7129000000000001e-05 2024-09-16T14:49:52.092 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.092 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.092 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.092 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.092 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 81/27 ECSubWrite(tid=339, reqid=osd.3.0:339, at_version=81'108, trim_to=0'0, roll_forward_to=80'104))", 2024-09-16T14:49:52.092 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:27.930359+0000", 2024-09-16T14:49:52.092 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.08333906, 2024-09-16T14:49:52.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0085286319999999995, 2024-09-16T14:49:52.093 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.093 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.093 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930359+0000", 2024-09-16T14:49:52.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.093 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.093 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.093 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930359+0000", 2024-09-16T14:49:52.093 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930358+0000", 2024-09-16T14:49:52.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.094 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.094 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930382+0000", 2024-09-16T14:49:52.094 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4034999999999999e-05 2024-09-16T14:49:52.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930393+0000", 2024-09-16T14:49:52.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1062e-05 2024-09-16T14:49:52.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.095 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.095 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.095 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930403+0000", 2024-09-16T14:49:52.095 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8779999999999993e-06 2024-09-16T14:49:52.095 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930675+0000", 2024-09-16T14:49:52.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027152900000000001 2024-09-16T14:49:52.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930681+0000", 2024-09-16T14:49:52.096 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.229e-06 2024-09-16T14:49:52.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.096 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.096 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.096 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.938865+0000", 2024-09-16T14:49:52.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081845249999999998 2024-09-16T14:49:52.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.097 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.097 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.938887+0000", 2024-09-16T14:49:52.097 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1946e-05 2024-09-16T14:49:52.097 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.097 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.097 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.097 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.097 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 81/65 ECSubWrite(tid=83, reqid=osd.1.0:83, at_version=81'66, trim_to=0'0, roll_forward_to=80'64))", 2024-09-16T14:49:52.097 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:27.931996+0000", 2024-09-16T14:49:52.097 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 24.081702304, 2024-09-16T14:49:52.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074671399999999997, 2024-09-16T14:49:52.098 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.098 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.098 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.931996+0000", 2024-09-16T14:49:52.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.098 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.098 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.098 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.098 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.931996+0000", 2024-09-16T14:49:52.098 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.931995+0000", 2024-09-16T14:49:52.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.099 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.932025+0000", 2024-09-16T14:49:52.099 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9884999999999999e-05 2024-09-16T14:49:52.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.099 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.099 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.932035+0000", 2024-09-16T14:49:52.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0182e-05 2024-09-16T14:49:52.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.932043+0000", 2024-09-16T14:49:52.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0900000000000005e-06 2024-09-16T14:49:52.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.100 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.100 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.100 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.933159+0000", 2024-09-16T14:49:52.100 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011161980000000001 2024-09-16T14:49:52.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.933170+0000", 2024-09-16T14:49:52.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0925e-05 2024-09-16T14:49:52.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.939441+0000", 2024-09-16T14:49:52.101 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062703339999999998 2024-09-16T14:49:52.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.101 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.101 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.939463+0000", 2024-09-16T14:49:52.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2005e-05 2024-09-16T14:49:52.102 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.102 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.102 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.102 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 90/27 ECSubWrite(tid=310, reqid=osd.7.0:310, at_version=90'48, trim_to=0'0, roll_forward_to=89'46))", 2024-09-16T14:49:52.102 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:52.102 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.059464857, 2024-09-16T14:49:52.102 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011963438999999999, 2024-09-16T14:49:52.102 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.102 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.102 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.102 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:52.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:52.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.103 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.103 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.103 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:52.103 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954255+0000", 2024-09-16T14:49:52.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2704000000000001e-05 2024-09-16T14:49:52.104 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.104 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.104 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.104 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954273+0000", 2024-09-16T14:49:52.104 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7609999999999999e-05 2024-09-16T14:49:52.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.105 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.105 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.105 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954283+0000", 2024-09-16T14:49:52.105 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9539999999999999e-06 2024-09-16T14:49:52.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.106 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.106 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.106 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954340+0000", 2024-09-16T14:49:52.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6968000000000001e-05 2024-09-16T14:49:52.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.107 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.107 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954352+0000", 2024-09-16T14:49:52.107 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1548e-05 2024-09-16T14:49:52.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.966177+0000", 2024-09-16T14:49:52.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011825293000000001 2024-09-16T14:49:52.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.108 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.108 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.108 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.966196+0000", 2024-09-16T14:49:52.108 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9636999999999999e-05 2024-09-16T14:49:52.108 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.108 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.108 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.109 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.109 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 91/90 ECSubWrite(tid=383, reqid=client.4500.0:825, at_version=91'144, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:52.109 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:52.109 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.037743508, 2024-09-16T14:49:52.109 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079391419999999997, 2024-09-16T14:49:52.109 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.109 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.109 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.109 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.109 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.109 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:52.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:52.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.110 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.110 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.110 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:52.110 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976430+0000", 2024-09-16T14:49:52.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000476222 2024-09-16T14:49:52.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.111 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.111 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.111 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976441+0000", 2024-09-16T14:49:52.111 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1457000000000001e-05 2024-09-16T14:49:52.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976449+0000", 2024-09-16T14:49:52.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9719999999999992e-06 2024-09-16T14:49:52.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976957+0000", 2024-09-16T14:49:52.112 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050779500000000001 2024-09-16T14:49:52.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.112 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.112 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.112 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976967+0000", 2024-09-16T14:49:52.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0424e-05 2024-09-16T14:49:52.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.983864+0000", 2024-09-16T14:49:52.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068963369999999998 2024-09-16T14:49:52.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.113 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.113 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.113 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.983894+0000", 2024-09-16T14:49:52.113 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9784000000000001e-05 2024-09-16T14:49:52.113 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.113 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.113 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.114 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.114 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.17s5 91/90 ECSubWrite(tid=394, reqid=osd.3.0:391, at_version=0'0, trim_to=0'0, roll_forward_to=91'144))", 2024-09-16T14:49:52.114 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.006074+0000", 2024-09-16T14:49:52.114 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.007623815000001, 2024-09-16T14:49:52.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010644525, 2024-09-16T14:49:52.114 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.114 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.114 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.114 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.114 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.114 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006074+0000", 2024-09-16T14:49:52.114 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006074+0000", 2024-09-16T14:49:52.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.115 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006073+0000", 2024-09-16T14:49:52.115 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.115 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.115 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006086+0000", 2024-09-16T14:49:52.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2862e-05 2024-09-16T14:49:52.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.116 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.116 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006102+0000", 2024-09-16T14:49:52.116 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6331e-05 2024-09-16T14:49:52.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006113+0000", 2024-09-16T14:49:52.117 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0518e-05 2024-09-16T14:49:52.117 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.117 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.117 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.117 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006171+0000", 2024-09-16T14:49:52.118 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8326999999999999e-05 2024-09-16T14:49:52.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.118 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.118 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.118 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006182+0000", 2024-09-16T14:49:52.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1134999999999999e-05 2024-09-16T14:49:52.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.119 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.119 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016684+0000", 2024-09-16T14:49:52.119 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010501664000000001 2024-09-16T14:49:52.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.119 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.120 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.120 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016719+0000", 2024-09-16T14:49:52.120 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4480000000000002e-05 2024-09-16T14:49:52.120 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.120 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.120 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.121 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.121 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.19s3 91/90 ECSubWrite(tid=102, reqid=client.4500.0:822, at_version=91'77, trim_to=0'0, roll_forward_to=91'75))", 2024-09-16T14:49:52.121 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.006461+0000", 2024-09-16T14:49:52.121 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.007236636, 2024-09-16T14:49:52.121 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010302113, 2024-09-16T14:49:52.121 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.121 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.121 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006461+0000", 2024-09-16T14:49:52.122 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.122 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.122 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006461+0000", 2024-09-16T14:49:52.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.123 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.123 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006460+0000", 2024-09-16T14:49:52.123 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.123 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.123 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007931+0000", 2024-09-16T14:49:52.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014713079999999999 2024-09-16T14:49:52.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.124 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.124 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.124 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007954+0000", 2024-09-16T14:49:52.124 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2308999999999999e-05 2024-09-16T14:49:52.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.125 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007970+0000", 2024-09-16T14:49:52.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5880999999999999e-05 2024-09-16T14:49:52.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.125 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008058+0000", 2024-09-16T14:49:52.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8506999999999998e-05 2024-09-16T14:49:52.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008071+0000", 2024-09-16T14:49:52.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2434e-05 2024-09-16T14:49:52.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016742+0000", 2024-09-16T14:49:52.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086718799999999999 2024-09-16T14:49:52.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016763+0000", 2024-09-16T14:49:52.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0890000000000002e-05 2024-09-16T14:49:52.128 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.128 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.128 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.129 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.17s5 91/90 ECSubWrite(tid=395, reqid=osd.3.0:395, at_version=91'146, trim_to=0'0, roll_forward_to=91'144))", 2024-09-16T14:49:52.129 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.008031+0000", 2024-09-16T14:49:52.129 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.005666446999999, 2024-09-16T14:49:52.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087059660000000007, 2024-09-16T14:49:52.129 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.129 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.129 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008031+0000", 2024-09-16T14:49:52.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008031+0000", 2024-09-16T14:49:52.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008030+0000", 2024-09-16T14:49:52.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:52.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008049+0000", 2024-09-16T14:49:52.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9230999999999999e-05 2024-09-16T14:49:52.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008059+0000", 2024-09-16T14:49:52.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7729999999999994e-06 2024-09-16T14:49:52.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008066+0000", 2024-09-16T14:49:52.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4849999999999997e-06 2024-09-16T14:49:52.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.132 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.132 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008113+0000", 2024-09-16T14:49:52.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7154999999999998e-05 2024-09-16T14:49:52.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008124+0000", 2024-09-16T14:49:52.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0933e-05 2024-09-16T14:49:52.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016723+0000", 2024-09-16T14:49:52.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0085989640000000006 2024-09-16T14:49:52.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016737+0000", 2024-09-16T14:49:52.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4635e-05 2024-09-16T14:49:52.134 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.134 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.134 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.134 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.14s2 91/90 ECSubWrite(tid=103, reqid=osd.1.0:102, at_version=0'0, trim_to=0'0, roll_forward_to=91'76))", 2024-09-16T14:49:52.134 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.009355+0000", 2024-09-16T14:49:52.134 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.00434237, 2024-09-16T14:49:52.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074028089999999998, 2024-09-16T14:49:52.134 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.134 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009355+0000", 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009355+0000", 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009354+0000", 2024-09-16T14:49:52.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:52.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009370+0000", 2024-09-16T14:49:52.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6163000000000001e-05 2024-09-16T14:49:52.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009384+0000", 2024-09-16T14:49:52.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3655000000000001e-05 2024-09-16T14:49:52.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009393+0000", 2024-09-16T14:49:52.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1919999999999994e-06 2024-09-16T14:49:52.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009453+0000", 2024-09-16T14:49:52.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9626000000000001e-05 2024-09-16T14:49:52.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009467+0000", 2024-09-16T14:49:52.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4123e-05 2024-09-16T14:49:52.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016732+0000", 2024-09-16T14:49:52.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007265395 2024-09-16T14:49:52.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016758+0000", 2024-09-16T14:49:52.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6157e-05 2024-09-16T14:49:52.138 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.138 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.138 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.139 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.ds2 95/65 ECSubWrite(tid=335, reqid=osd.7.0:333, at_version=0'0, trim_to=0'0, roll_forward_to=95'81))", 2024-09-16T14:49:52.139 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:41.883865+0000", 2024-09-16T14:49:52.139 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.129832596, 2024-09-16T14:49:52.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0072428129999999999, 2024-09-16T14:49:52.139 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.139 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.139 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883865+0000", 2024-09-16T14:49:52.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883865+0000", 2024-09-16T14:49:52.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883865+0000", 2024-09-16T14:49:52.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883874+0000", 2024-09-16T14:49:52.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7290000000000006e-06 2024-09-16T14:49:52.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883882+0000", 2024-09-16T14:49:52.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4859999999999997e-06 2024-09-16T14:49:52.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883889+0000", 2024-09-16T14:49:52.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3309999999999998e-06 2024-09-16T14:49:52.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883986+0000", 2024-09-16T14:49:52.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6576999999999994e-05 2024-09-16T14:49:52.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883994+0000", 2024-09-16T14:49:52.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0139999999999998e-06 2024-09-16T14:49:52.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.891085+0000", 2024-09-16T14:49:52.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070908899999999999 2024-09-16T14:49:52.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.891108+0000", 2024-09-16T14:49:52.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3181999999999999e-05 2024-09-16T14:49:52.143 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.143 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.143 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.143 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.143 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.143 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:52.144 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:49:52.145 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:52.143+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:52.246 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:52.244+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:52.248 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:52.248 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:52.248 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:52.248 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:52.258 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:49:52.346 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:52.345+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:52.360 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:52.360 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:52.360 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:52.360 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:52.360 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:52.369 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:49:52.447 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:52.446+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:52.472 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:52.535 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:52.535 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:52.535 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:52.535 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.535 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:49:52.535 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:52.536 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 71.561034054000004, 2024-09-16T14:49:52.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079855010000000008, 2024-09-16T14:49:52.536 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.536 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.536 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.536 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:52.536 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.536 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.536 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:52.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909483+0000", 2024-09-16T14:49:52.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.537 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.537 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909521+0000", 2024-09-16T14:49:52.537 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7185999999999999e-05 2024-09-16T14:49:52.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909543+0000", 2024-09-16T14:49:52.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2802999999999998e-05 2024-09-16T14:49:52.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909551+0000", 2024-09-16T14:49:52.538 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5039999999999996e-06 2024-09-16T14:49:52.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.538 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.538 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909927+0000", 2024-09-16T14:49:52.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037637400000000001 2024-09-16T14:49:52.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:49:52.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2040000000000002e-06 2024-09-16T14:49:52.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.539 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.539 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917442+0000", 2024-09-16T14:49:52.539 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007508659 2024-09-16T14:49:52.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.539 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.540 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.540 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917469+0000", 2024-09-16T14:49:52.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7197999999999999e-05 2024-09-16T14:49:52.540 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.540 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.540 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.540 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.540 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:49:52.540 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:52.540 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 71.556961708000003, 2024-09-16T14:49:52.540 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067701159999999996, 2024-09-16T14:49:52.541 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.541 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.541 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:52.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.541 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.541 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.541 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:52.541 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:52.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913579+0000", 2024-09-16T14:49:52.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2829000000000001e-05 2024-09-16T14:49:52.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.542 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.542 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.542 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913591+0000", 2024-09-16T14:49:52.542 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2264e-05 2024-09-16T14:49:52.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913600+0000", 2024-09-16T14:49:52.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9959999999999999e-06 2024-09-16T14:49:52.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913648+0000", 2024-09-16T14:49:52.543 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7759000000000001e-05 2024-09-16T14:49:52.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.543 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.543 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.543 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913657+0000", 2024-09-16T14:49:52.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6709999999999994e-06 2024-09-16T14:49:52.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920296+0000", 2024-09-16T14:49:52.544 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066385120000000001 2024-09-16T14:49:52.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.544 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.544 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.544 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920326+0000", 2024-09-16T14:49:52.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0405000000000001e-05 2024-09-16T14:49:52.545 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.545 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.545 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.545 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.545 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 34/27 ECSubWrite(tid=74, reqid=client.4500.0:229, at_version=34'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:49:52.545 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:52.545 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 71.556145749999999, 2024-09-16T14:49:52.545 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060056090000000003, 2024-09-16T14:49:52.545 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.545 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.546 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:52.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.546 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:52.546 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.546 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914468+0000", 2024-09-16T14:49:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6398999999999996e-05 2024-09-16T14:49:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914483+0000", 2024-09-16T14:49:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5238e-05 2024-09-16T14:49:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.547 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914494+0000", 2024-09-16T14:49:52.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0192e-05 2024-09-16T14:49:52.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914622+0000", 2024-09-16T14:49:52.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012855599999999999 2024-09-16T14:49:52.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.548 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.548 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.548 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914635+0000", 2024-09-16T14:49:52.548 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2510000000000001e-05 2024-09-16T14:49:52.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920339+0000", 2024-09-16T14:49:52.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057041549999999998 2024-09-16T14:49:52.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.549 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.549 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.549 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920378+0000", 2024-09-16T14:49:52.549 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8918999999999998e-05 2024-09-16T14:49:52.549 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.549 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.549 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.550 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.550 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:49:52.550 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:52.550 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 71.554132973999998, 2024-09-16T14:49:52.550 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069011319999999999, 2024-09-16T14:49:52.550 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.550 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.551 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.551 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:52.551 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.551 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:52.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:52.552 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:49:52.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.552 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.552 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.552 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916410+0000", 2024-09-16T14:49:52.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5343000000000001e-05 2024-09-16T14:49:52.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916418+0000", 2024-09-16T14:49:52.553 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2360000000000004e-06 2024-09-16T14:49:52.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.553 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.553 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.553 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916428+0000", 2024-09-16T14:49:52.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6779999999999995e-06 2024-09-16T14:49:52.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.554 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.554 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916492+0000", 2024-09-16T14:49:52.554 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4219000000000003e-05 2024-09-16T14:49:52.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.554 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916502+0000", 2024-09-16T14:49:52.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6690000000000001e-06 2024-09-16T14:49:52.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.555 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.555 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.555 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923254+0000", 2024-09-16T14:49:52.555 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067518070000000003 2024-09-16T14:49:52.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.556 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.556 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.556 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923286+0000", 2024-09-16T14:49:52.556 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2367999999999999e-05 2024-09-16T14:49:52.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.556 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.556 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.557 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.557 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s7 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:52.557 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:52.557 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.548828589999999, 2024-09-16T14:49:52.557 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006972692, 2024-09-16T14:49:52.557 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.557 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.558 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.558 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:52.558 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.558 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:52.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.559 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.559 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.559 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:52.559 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921709+0000", 2024-09-16T14:49:52.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0411000000000002e-05 2024-09-16T14:49:52.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.560 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.560 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921719+0000", 2024-09-16T14:49:52.560 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2650000000000002e-06 2024-09-16T14:49:52.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.561 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921726+0000", 2024-09-16T14:49:52.561 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8770000000000004e-06 2024-09-16T14:49:52.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.561 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.561 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921773+0000", 2024-09-16T14:49:52.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7435999999999998e-05 2024-09-16T14:49:52.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.562 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.562 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.562 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921782+0000", 2024-09-16T14:49:52.562 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6039999999999993e-06 2024-09-16T14:49:52.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928635+0000", 2024-09-16T14:49:52.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068533129999999998 2024-09-16T14:49:52.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.563 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.563 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.563 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928662+0000", 2024-09-16T14:49:52.563 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7089000000000001e-05 2024-09-16T14:49:52.564 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.564 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.564 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.564 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.564 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:52.564 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:52.564 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.548332102000003, 2024-09-16T14:49:52.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064783530000000001, 2024-09-16T14:49:52.565 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.565 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.565 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.565 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.565 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:52.565 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.566 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:52.566 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.566 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.566 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922185+0000", 2024-09-16T14:49:52.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.567 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.567 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922205+0000", 2024-09-16T14:49:52.567 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9871e-05 2024-09-16T14:49:52.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.567 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922219+0000", 2024-09-16T14:49:52.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4266e-05 2024-09-16T14:49:52.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.568 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.568 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.568 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922229+0000", 2024-09-16T14:49:52.568 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5619999999999993e-06 2024-09-16T14:49:52.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.569 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.569 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922270+0000", 2024-09-16T14:49:52.569 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0917000000000001e-05 2024-09-16T14:49:52.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.569 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922278+0000", 2024-09-16T14:49:52.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3890000000000005e-06 2024-09-16T14:49:52.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928630+0000", 2024-09-16T14:49:52.570 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063524439999999996 2024-09-16T14:49:52.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.570 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.570 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.570 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928664+0000", 2024-09-16T14:49:52.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3651e-05 2024-09-16T14:49:52.571 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.571 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.571 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.571 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s5 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:49:52.571 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:52.571 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.546883698000002, 2024-09-16T14:49:52.571 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069615850000000002, 2024-09-16T14:49:52.571 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.571 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.572 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:52.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.572 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:52.572 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.572 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.572 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923633+0000", 2024-09-16T14:49:52.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923644+0000", 2024-09-16T14:49:52.573 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.08e-05 2024-09-16T14:49:52.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.573 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.573 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.573 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923657+0000", 2024-09-16T14:49:52.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2836999999999999e-05 2024-09-16T14:49:52.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923664+0000", 2024-09-16T14:49:52.574 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1470000000000004e-06 2024-09-16T14:49:52.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.574 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.574 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.574 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923885+0000", 2024-09-16T14:49:52.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022155800000000001 2024-09-16T14:49:52.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923894+0000", 2024-09-16T14:49:52.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9260000000000006e-06 2024-09-16T14:49:52.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.575 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.575 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.575 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930571+0000", 2024-09-16T14:49:52.575 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066770420000000002 2024-09-16T14:49:52.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.576 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.576 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930596+0000", 2024-09-16T14:49:52.576 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4383e-05 2024-09-16T14:49:52.576 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.576 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.576 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.576 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.576 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as3 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:52.576 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:52.577 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.546137462000004, 2024-09-16T14:49:52.577 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062471519999999997, 2024-09-16T14:49:52.577 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.577 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.577 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:52.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:52.578 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.578 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.578 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.578 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.578 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:52.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924405+0000", 2024-09-16T14:49:52.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5147000000000002e-05 2024-09-16T14:49:52.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.579 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.579 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924414+0000", 2024-09-16T14:49:52.579 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4810000000000006e-06 2024-09-16T14:49:52.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924421+0000", 2024-09-16T14:49:52.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0899999999999999e-06 2024-09-16T14:49:52.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.580 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.580 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.580 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924468+0000", 2024-09-16T14:49:52.580 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.727e-05 2024-09-16T14:49:52.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924477+0000", 2024-09-16T14:49:52.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7360000000000007e-06 2024-09-16T14:49:52.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930601+0000", 2024-09-16T14:49:52.581 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061242529999999996 2024-09-16T14:49:52.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.581 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.581 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.581 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930628+0000", 2024-09-16T14:49:52.582 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6517e-05 2024-09-16T14:49:52.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.582 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.582 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.582 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:52.582 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:52.582 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 68.545887257999993, 2024-09-16T14:49:52.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006019328, 2024-09-16T14:49:52.583 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.583 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.583 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.583 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.583 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.583 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:52.583 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:52.584 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.584 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.584 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.584 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924630+0000", 2024-09-16T14:49:52.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924650+0000", 2024-09-16T14:49:52.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0024e-05 2024-09-16T14:49:52.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.585 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.585 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924660+0000", 2024-09-16T14:49:52.585 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9219999999999992e-06 2024-09-16T14:49:52.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924668+0000", 2024-09-16T14:49:52.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6820000000000003e-06 2024-09-16T14:49:52.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924711+0000", 2024-09-16T14:49:52.586 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3009000000000001e-05 2024-09-16T14:49:52.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.586 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.586 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924720+0000", 2024-09-16T14:49:52.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9290000000000004e-06 2024-09-16T14:49:52.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930634+0000", 2024-09-16T14:49:52.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059144810000000001 2024-09-16T14:49:52.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.587 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.587 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.587 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930650+0000", 2024-09-16T14:49:52.587 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5688999999999998e-05 2024-09-16T14:49:52.587 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.587 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.588 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:52.588 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:52.588 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.537518138999999, 2024-09-16T14:49:52.588 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062137989999999999, 2024-09-16T14:49:52.588 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.588 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.588 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.588 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.588 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:52.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:52.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.589 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.589 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.589 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.932999+0000", 2024-09-16T14:49:52.589 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.590 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.590 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933022+0000", 2024-09-16T14:49:52.590 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3518000000000001e-05 2024-09-16T14:49:52.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.590 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933035+0000", 2024-09-16T14:49:52.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2565000000000001e-05 2024-09-16T14:49:52.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.591 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.591 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.591 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933044+0000", 2024-09-16T14:49:52.591 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3009999999999995e-06 2024-09-16T14:49:52.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934016+0000", 2024-09-16T14:49:52.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00097159700000000004 2024-09-16T14:49:52.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934022+0000", 2024-09-16T14:49:52.592 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4869999999999999e-06 2024-09-16T14:49:52.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.592 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.592 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939202+0000", 2024-09-16T14:49:52.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051795820000000003 2024-09-16T14:49:52.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.593 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.593 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939214+0000", 2024-09-16T14:49:52.593 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1663e-05 2024-09-16T14:49:52.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.593 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.593 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.593 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.593 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.cs3 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 44.537051603999998, 2024-09-16T14:49:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057306709999999997, 2024-09-16T14:49:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.594 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:52.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933465+0000", 2024-09-16T14:49:52.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.595 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.595 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933489+0000", 2024-09-16T14:49:52.595 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3940000000000001e-05 2024-09-16T14:49:52.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.595 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933501+0000", 2024-09-16T14:49:52.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1932e-05 2024-09-16T14:49:52.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.596 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.596 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.596 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933510+0000", 2024-09-16T14:49:52.596 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5820000000000002e-06 2024-09-16T14:49:52.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936680+0000", 2024-09-16T14:49:52.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003169893 2024-09-16T14:49:52.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936689+0000", 2024-09-16T14:49:52.597 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5810000000000003e-06 2024-09-16T14:49:52.597 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.597 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.597 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.597 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939176+0000", 2024-09-16T14:49:52.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002486238 2024-09-16T14:49:52.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.598 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.598 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939197+0000", 2024-09-16T14:49:52.598 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1325e-05 2024-09-16T14:49:52.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.598 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.598 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.598 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.598 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 62/27 ECSubWrite(tid=244, reqid=osd.7.0:244, at_version=62'27, trim_to=0'0, roll_forward_to=61'25))", 2024-09-16T14:49:52.598 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:52.598 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 43.527688597000001, 2024-09-16T14:49:52.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007111081, 2024-09-16T14:49:52.599 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.599 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.599 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:52.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.599 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.599 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.599 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:52.599 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:52.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.600 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942855+0000", 2024-09-16T14:49:52.600 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5999999999999998e-05 2024-09-16T14:49:52.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.600 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942867+0000", 2024-09-16T14:49:52.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2481e-05 2024-09-16T14:49:52.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942879+0000", 2024-09-16T14:49:52.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1838000000000001e-05 2024-09-16T14:49:52.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.601 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.601 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942938+0000", 2024-09-16T14:49:52.601 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8282000000000002e-05 2024-09-16T14:49:52.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942948+0000", 2024-09-16T14:49:52.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0497e-05 2024-09-16T14:49:52.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949905+0000", 2024-09-16T14:49:52.602 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069572940000000002 2024-09-16T14:49:52.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.602 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.602 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.602 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949940+0000", 2024-09-16T14:49:52.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4965999999999997e-05 2024-09-16T14:49:52.603 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.603 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.603 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.603 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 62/27 ECSubWrite(tid=234, reqid=osd.11.0:234, at_version=62'39, trim_to=0'0, roll_forward_to=61'37))", 2024-09-16T14:49:52.603 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:52.603 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 43.527255113000002, 2024-09-16T14:49:52.603 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066952440000000004, 2024-09-16T14:49:52.603 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.603 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.603 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.603 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:52.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:52.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.604 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.604 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.604 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943262+0000", 2024-09-16T14:49:52.604 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943285+0000", 2024-09-16T14:49:52.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2650000000000002e-05 2024-09-16T14:49:52.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.605 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943301+0000", 2024-09-16T14:49:52.605 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5398000000000001e-05 2024-09-16T14:49:52.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.605 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.605 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943309+0000", 2024-09-16T14:49:52.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9269999999999994e-06 2024-09-16T14:49:52.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943373+0000", 2024-09-16T14:49:52.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3194999999999994e-05 2024-09-16T14:49:52.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.606 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.606 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.606 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943383+0000", 2024-09-16T14:49:52.606 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0778000000000001e-05 2024-09-16T14:49:52.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949934+0000", 2024-09-16T14:49:52.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065509790000000002 2024-09-16T14:49:52.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.607 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.607 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949958+0000", 2024-09-16T14:49:52.607 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3598999999999999e-05 2024-09-16T14:49:52.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.607 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.607 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.607 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.608 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 76/27 ECSubWrite(tid=303, reqid=client.4500.0:699, at_version=76'98, trim_to=0'0, roll_forward_to=76'95))", 2024-09-16T14:49:52.608 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:52.608 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 29.560484568, 2024-09-16T14:49:52.608 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006028032, 2024-09-16T14:49:52.608 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.608 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.608 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.608 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.608 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:52.609 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.609 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.609 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:52.610 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.610 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.610 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.610 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910032+0000", 2024-09-16T14:49:52.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910053+0000", 2024-09-16T14:49:52.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0665e-05 2024-09-16T14:49:52.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.611 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.611 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.611 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910067+0000", 2024-09-16T14:49:52.611 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4416000000000001e-05 2024-09-16T14:49:52.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910077+0000", 2024-09-16T14:49:52.612 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4509999999999998e-06 2024-09-16T14:49:52.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.612 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.612 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910367+0000", 2024-09-16T14:49:52.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00029006799999999999 2024-09-16T14:49:52.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910378+0000", 2024-09-16T14:49:52.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0628000000000001e-05 2024-09-16T14:49:52.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.613 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.613 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.613 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.916036+0000", 2024-09-16T14:49:52.613 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0056580270000000004 2024-09-16T14:49:52.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.614 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.614 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.916061+0000", 2024-09-16T14:49:52.614 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5658000000000001e-05 2024-09-16T14:49:52.614 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.614 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.614 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.614 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.615 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 77/27 ECSubWrite(tid=277, reqid=osd.11.0:277, at_version=77'54, trim_to=0'0, roll_forward_to=75'52))", 2024-09-16T14:49:52.615 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:52.615 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.559997200000002, 2024-09-16T14:49:52.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064553950000000001, 2024-09-16T14:49:52.615 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.615 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.615 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.615 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.615 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:52.615 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.615 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:52.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910520+0000", 2024-09-16T14:49:52.616 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:49:52.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.616 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.616 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.616 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910554+0000", 2024-09-16T14:49:52.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3111999999999997e-05 2024-09-16T14:49:52.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910578+0000", 2024-09-16T14:49:52.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4757999999999999e-05 2024-09-16T14:49:52.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.617 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.617 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.617 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910590+0000", 2024-09-16T14:49:52.617 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2156e-05 2024-09-16T14:49:52.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910646+0000", 2024-09-16T14:49:52.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5979000000000001e-05 2024-09-16T14:49:52.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.618 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910656+0000", 2024-09-16T14:49:52.618 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9410000000000002e-06 2024-09-16T14:49:52.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.618 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.618 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916943+0000", 2024-09-16T14:49:52.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062870260000000002 2024-09-16T14:49:52.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.619 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.619 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916976+0000", 2024-09-16T14:49:52.619 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2641999999999997e-05 2024-09-16T14:49:52.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.619 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.620 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 91/27 ECSubWrite(tid=316, reqid=osd.7.0:315, at_version=0'0, trim_to=0'0, roll_forward_to=91'50))", 2024-09-16T14:49:52.620 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.963325+0000", 2024-09-16T14:49:52.620 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.507193108999999, 2024-09-16T14:49:52.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060303650000000002, 2024-09-16T14:49:52.620 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.620 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.620 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.620 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.620 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963325+0000", 2024-09-16T14:49:52.620 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.620 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963325+0000", 2024-09-16T14:49:52.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.621 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.621 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.621 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963324+0000", 2024-09-16T14:49:52.621 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963338+0000", 2024-09-16T14:49:52.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3813e-05 2024-09-16T14:49:52.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.622 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963350+0000", 2024-09-16T14:49:52.622 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2512e-05 2024-09-16T14:49:52.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.622 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.622 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963360+0000", 2024-09-16T14:49:52.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0141e-05 2024-09-16T14:49:52.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963425+0000", 2024-09-16T14:49:52.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4579999999999995e-05 2024-09-16T14:49:52.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.623 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.623 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963437+0000", 2024-09-16T14:49:52.623 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1780000000000001e-05 2024-09-16T14:49:52.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.623 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.969335+0000", 2024-09-16T14:49:52.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0058980500000000002 2024-09-16T14:49:52.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.624 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.624 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.624 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.969355+0000", 2024-09-16T14:49:52.624 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0489e-05 2024-09-16T14:49:52.624 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.625 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.625 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.625 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 91/90 ECSubWrite(tid=383, reqid=client.4500.0:825, at_version=91'144, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:52.625 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.976070+0000", 2024-09-16T14:49:52.625 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.494447901000001, 2024-09-16T14:49:52.625 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066451510000000002, 2024-09-16T14:49:52.625 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.625 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.625 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976070+0000", 2024-09-16T14:49:52.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976070+0000", 2024-09-16T14:49:52.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.626 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.626 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.626 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976069+0000", 2024-09-16T14:49:52.626 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:52.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976953+0000", 2024-09-16T14:49:52.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00088349499999999998 2024-09-16T14:49:52.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.627 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.627 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.627 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977007+0000", 2024-09-16T14:49:52.627 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4676000000000003e-05 2024-09-16T14:49:52.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977083+0000", 2024-09-16T14:49:52.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5831e-05 2024-09-16T14:49:52.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.628 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.628 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977120+0000", 2024-09-16T14:49:52.628 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6887000000000001e-05 2024-09-16T14:49:52.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.628 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.629 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977129+0000", 2024-09-16T14:49:52.629 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9120000000000004e-06 2024-09-16T14:49:52.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.629 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.629 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.982689+0000", 2024-09-16T14:49:52.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055597590000000001 2024-09-16T14:49:52.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.630 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.630 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.982715+0000", 2024-09-16T14:49:52.630 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6478999999999998e-05 2024-09-16T14:49:52.630 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.630 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.630 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.630 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.630 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.19s7 91/90 ECSubWrite(tid=102, reqid=client.4500.0:822, at_version=91'77, trim_to=0'0, roll_forward_to=91'75))", 2024-09-16T14:49:52.630 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.007185+0000", 2024-09-16T14:49:52.631 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.463333167, 2024-09-16T14:49:52.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068803600000000003, 2024-09-16T14:49:52.631 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.631 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.631 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007185+0000", 2024-09-16T14:49:52.631 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.631 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.631 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007185+0000", 2024-09-16T14:49:52.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007183+0000", 2024-09-16T14:49:52.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:52.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.632 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.632 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008457+0000", 2024-09-16T14:49:52.632 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001274025 2024-09-16T14:49:52.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.632 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008473+0000", 2024-09-16T14:49:52.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5605999999999999e-05 2024-09-16T14:49:52.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008483+0000", 2024-09-16T14:49:52.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8430000000000005e-06 2024-09-16T14:49:52.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.633 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.633 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.633 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008541+0000", 2024-09-16T14:49:52.633 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7911999999999997e-05 2024-09-16T14:49:52.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008551+0000", 2024-09-16T14:49:52.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0353999999999999e-05 2024-09-16T14:49:52.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.634 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.634 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.634 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.014044+0000", 2024-09-16T14:49:52.634 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054927439999999999 2024-09-16T14:49:52.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.635 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.635 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.014065+0000", 2024-09-16T14:49:52.635 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1124999999999998e-05 2024-09-16T14:49:52.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.635 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.635 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.635 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.17s4 91/90 ECSubWrite(tid=395, reqid=osd.3.0:395, at_version=91'146, trim_to=0'0, roll_forward_to=91'144))", 2024-09-16T14:49:52.635 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:52.636 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 14.461063549, 2024-09-16T14:49:52.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0076634049999999999, 2024-09-16T14:49:52.636 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.636 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.636 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:52.636 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.636 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.636 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:52.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:52.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.637 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.637 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009478+0000", 2024-09-16T14:49:52.637 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4235000000000001e-05 2024-09-16T14:49:52.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.637 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009486+0000", 2024-09-16T14:49:52.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7940000000000003e-06 2024-09-16T14:49:52.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.638 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009492+0000", 2024-09-16T14:49:52.638 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7980000000000002e-06 2024-09-16T14:49:52.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.638 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.638 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010420+0000", 2024-09-16T14:49:52.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00092807100000000004 2024-09-16T14:49:52.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010429+0000", 2024-09-16T14:49:52.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8389999999999996e-06 2024-09-16T14:49:52.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.639 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.639 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.639 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017094+0000", 2024-09-16T14:49:52.639 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066650879999999996 2024-09-16T14:49:52.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.640 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.640 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017118+0000", 2024-09-16T14:49:52.640 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3941e-05 2024-09-16T14:49:52.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.640 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.640 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s5 104/90 ECSubWrite(tid=128, reqid=client.4500.0:971, at_version=104'50, trim_to=0'0, roll_forward_to=91'48))", 2024-09-16T14:49:52.640 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:50.871724+0000", 2024-09-16T14:49:52.640 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 1.5987935639999999, 2024-09-16T14:49:52.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074684510000000001, 2024-09-16T14:49:52.641 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:52.641 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:52.641 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:52.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:52.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.871724+0000", 2024-09-16T14:49:52.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.641 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.641 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:52.641 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.871724+0000", 2024-09-16T14:49:52.641 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:52.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:52.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.871724+0000", 2024-09-16T14:49:52.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:52.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.642 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:52.642 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.872414+0000", 2024-09-16T14:49:52.642 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00068969599999999995 2024-09-16T14:49:52.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.642 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:52.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.872429+0000", 2024-09-16T14:49:52.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5865000000000001e-05 2024-09-16T14:49:52.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:52.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.872440+0000", 2024-09-16T14:49:52.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0771000000000001e-05 2024-09-16T14:49:52.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.643 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.643 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:52.643 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.872501+0000", 2024-09-16T14:49:52.643 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1230999999999997e-05 2024-09-16T14:49:52.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:52.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.872515+0000", 2024-09-16T14:49:52.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3448000000000001e-05 2024-09-16T14:49:52.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.644 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:52.644 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.879160+0000", 2024-09-16T14:49:52.644 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006645349 2024-09-16T14:49:52.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:52.644 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:52.645 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:52.645 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.879193+0000", 2024-09-16T14:49:52.645 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2490000000000002e-05 2024-09-16T14:49:52.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.645 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:52.645 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:52.645 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:52.647 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:49:52.648 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:52.646+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:52.673 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T14:49:52.747 INFO:tasks.ceph.osd.10.smithi114.stderr:2024-09-16T14:49:52.746+0000 7f95e46b3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 33116) UID: 0 2024-09-16T14:49:52.790 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:52.791 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:52.791 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:52.791 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:52.800 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:49:52.800 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:52.801 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:52.817 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:52.817 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:52.833 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:52.833 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:52.848 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:52.848 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:52.864 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is unset 2024-09-16T14:49:52.865 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T14:49:52.865 INFO:tasks.rados.rados.0.smithi040.stdout:399: done (2 left) 2024-09-16T14:49:52.865 INFO:tasks.rados.rados.0.smithi040.stdout: waiting on 2 2024-09-16T14:49:52.866 INFO:tasks.rados.rados.0.smithi040.stdout:397: finishing write tid 1 to smithi04040151-27 2024-09-16T14:49:52.866 INFO:tasks.rados.rados.0.smithi040.stdout:397: finishing write tid 2 to smithi04040151-27 2024-09-16T14:49:52.866 INFO:tasks.rados.rados.0.smithi040.stdout:397: finishing write tid 3 to smithi04040151-27 2024-09-16T14:49:52.866 INFO:tasks.rados.rados.0.smithi040.stdout:397: finishing write tid 4 to smithi04040151-27 2024-09-16T14:49:52.866 INFO:tasks.rados.rados.0.smithi040.stdout:397: finishing write tid 5 to smithi04040151-27 2024-09-16T14:49:52.866 INFO:tasks.rados.rados.0.smithi040.stdout:397: finishing write tid 6 to smithi04040151-27 2024-09-16T14:49:52.866 INFO:tasks.rados.rados.0.smithi040.stdout:397: finishing write tid 7 to smithi04040151-27 2024-09-16T14:49:52.866 INFO:tasks.rados.rados.0.smithi040.stdout:397: oid 27 updating version 0 to 88 2024-09-16T14:49:52.866 INFO:tasks.rados.rados.0.smithi040.stdout:397: oid 27 version 88 is already newer than 86 2024-09-16T14:49:52.866 INFO:tasks.rados.rados.0.smithi040.stdout:397: oid 27 version 88 is already newer than 85 2024-09-16T14:49:52.866 INFO:tasks.rados.rados.0.smithi040.stdout:397: oid 27 updating version 88 to 90 2024-09-16T14:49:52.866 INFO:tasks.rados.rados.0.smithi040.stdout:397: oid 27 version 90 is already newer than 87 2024-09-16T14:49:52.867 INFO:tasks.rados.rados.0.smithi040.stdout:397: oid 27 version 90 is already newer than 89 2024-09-16T14:49:52.867 INFO:tasks.rados.rados.0.smithi040.stdout:update_object_version oid 27 v 90 (ObjNum 141 snap 40 seq_num 141) dirty exists 2024-09-16T14:49:52.867 INFO:tasks.rados.rados.0.smithi040.stdout:397: left oid 27 (ObjNum 141 snap 40 seq_num 141) 2024-09-16T14:49:52.867 INFO:tasks.rados.rados.0.smithi040.stdout:398: expect (ObjNum 109 snap 24 seq_num 109) 2024-09-16T14:49:52.867 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:52.867 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:52.880 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T14:49:52.880 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:49:52.883 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:52.882+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:52.885 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:52.904 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:52.904 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:52.932 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:52.932 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:52.941 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:52.941 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:52.942 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:52.942 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:52.942 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:52.949 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:52.948+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:52.950 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:49:52.952 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:52.953 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:52.969 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:52.969 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:52.983 INFO:tasks.rados.rados.0.smithi040.stdout:397: done (1 left) 2024-09-16T14:49:52.983 INFO:tasks.rados.rados.0.smithi040.stdout:398: done (0 left) 2024-09-16T14:49:52.985 INFO:tasks.rados.rados.0.smithi040.stderr:0 errors. 2024-09-16T14:49:52.985 INFO:tasks.rados.rados.0.smithi040.stderr: 2024-09-16T14:49:52.986 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:52.987 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:52.988 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:49:53.003 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:53.004 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:53.050 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:53.048+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:53.095 INFO:teuthology.orchestra.run.smithi114.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:53.096 INFO:teuthology.orchestra.run.smithi114.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:53.104 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:49:53.150 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:53.149+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:53.187 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:49:53.188 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:49:53.200 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-16T14:49:52.862669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110535219,"num_objects":56,"num_object_clones":11,"num_object_copies":424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":366,"num_read_kb":86927,"num_write":739,"num_write_kb":93628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":49225728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2124,"ondisk_log_size":2124,"up":226,"acting":226,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":226,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":801552,"kb_used_data":225616,"kb_used_omap":19,"kb_used_meta":575724,"kb_avail":1131660528,"statfs":{"total":1159641169920,"available":1158820380672,"internally_reserved":0,"allocated":231030784,"data_stored":245224270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19502,"internal_metadata":589542354},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18292736,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":-11,"num_read_kb":4382,"num_write":-50,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-19,"num_bytes_recovered":-6012928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.852439"},"pg_stats":[{"pgid":"3.19","version":"105'90","reported_seq":129,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T14:49:51.915643+0000","last_change":"2024-09-16T14:49:49.018972+0000","last_active":"2024-09-16T14:49:51.915643+0000","last_peered":"2024-09-16T14:49:51.915643+0000","last_clean":"2024-09-16T14:49:51.915643+0000","last_became_active":"2024-09-16T14:49:39.892805+0000","last_became_peered":"2024-09-16T14:49:39.892805+0000","last_unstale":"2024-09-16T14:49:51.915643+0000","last_undegraded":"2024-09-16T14:49:51.915643+0000","last_fullsized":"2024-09-16T14:49:51.915643+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"95'81","last_scrub_stamp":"2024-09-16T14:49:42.521147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.521147+0000","objects_scrubbed":3,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:10.802992+0000","scrub_duration":0.003238852,"objects_trimmed":0,"snaptrim_duration":0.0042894789999999997,"stat_sum":{"num_bytes":7438336,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":2320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,10,5,7,11,2],"acting":[1,3,4,10,5,7,11,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.18","version":"102'122","reported_seq":171,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.716793+0000","last_change":"2024-09-16T14:49:50.716793+0000","last_active":"2024-09-16T14:49:50.716793+0000","last_peered":"2024-09-16T14:49:50.716793+0000","last_clean":"2024-09-16T14:49:50.716793+0000","last_became_active":"2024-09-16T14:49:39.893130+0000","last_became_peered":"2024-09-16T14:49:39.893130+0000","last_unstale":"2024-09-16T14:49:50.716793+0000","last_undegraded":"2024-09-16T14:49:50.716793+0000","last_fullsized":"2024-09-16T14:49:50.716793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"102'122","last_scrub_stamp":"2024-09-16T14:49:50.716587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:50.716587+0000","objects_scrubbed":2,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:17.481776+0000","scrub_duration":0.0042360469999999997,"objects_trimmed":1,"snaptrim_duration":0.021640853000000002,"stat_sum":{"num_bytes":6111232,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,1,2,8,7,4,5],"acting":[3,9,1,2,8,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.17","version":"104'172","reported_seq":224,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.926382+0000","last_change":"2024-09-16T14:49:49.034483+0000","last_active":"2024-09-16T14:49:50.926382+0000","last_peered":"2024-09-16T14:49:50.926382+0000","last_clean":"2024-09-16T14:49:50.926382+0000","last_became_active":"2024-09-16T14:49:42.869291+0000","last_became_peered":"2024-09-16T14:49:42.869291+0000","last_unstale":"2024-09-16T14:49:50.926382+0000","last_undegraded":"2024-09-16T14:49:50.926382+0000","last_fullsized":"2024-09-16T14:49:50.926382+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015625354000000001,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":18847,"num_write":90,"num_write_kb":17184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":20987904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,9,3,11,6,2,1],"acting":[0,5,9,3,11,6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.16","version":"83'46","reported_seq":52,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.877236+0000","last_change":"2024-09-16T14:49:49.020712+0000","last_active":"2024-09-16T14:49:49.877236+0000","last_peered":"2024-09-16T14:49:49.877236+0000","last_clean":"2024-09-16T14:49:49.877236+0000","last_became_active":"2024-09-16T14:49:39.885128+0000","last_became_peered":"2024-09-16T14:49:39.885128+0000","last_unstale":"2024-09-16T14:49:49.877236+0000","last_undegraded":"2024-09-16T14:49:49.877236+0000","last_fullsized":"2024-09-16T14:49:49.877236+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078294200000000001,"stat_sum":{"num_bytes":1900544,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":436,"num_write":31,"num_write_kb":2696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3,8,2,10,6],"acting":[5,7,1,3,8,2,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.15","version":"83'61","reported_seq":64,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.976286+0000","last_change":"2024-09-16T14:49:49.016770+0000","last_active":"2024-09-16T14:49:49.976286+0000","last_peered":"2024-09-16T14:49:49.976286+0000","last_clean":"2024-09-16T14:49:49.976286+0000","last_became_active":"2024-09-16T14:49:39.883704+0000","last_became_peered":"2024-09-16T14:49:39.883704+0000","last_unstale":"2024-09-16T14:49:49.976286+0000","last_undegraded":"2024-09-16T14:49:49.976286+0000","last_fullsized":"2024-09-16T14:49:49.976286+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"83'61","last_scrub_stamp":"2024-09-16T14:49:42.159081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.159081+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:49.030695+0000","scrub_duration":0.0019877559999999998,"objects_trimmed":0,"snaptrim_duration":0.0015822239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,4,2,6,0,9],"acting":[8,11,7,4,2,6,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.14","version":"95'82","reported_seq":128,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.859102+0000","last_change":"2024-09-16T14:49:49.022706+0000","last_active":"2024-09-16T14:49:49.859102+0000","last_peered":"2024-09-16T14:49:49.859102+0000","last_clean":"2024-09-16T14:49:49.859102+0000","last_became_active":"2024-09-16T14:49:39.886232+0000","last_became_peered":"2024-09-16T14:49:39.886232+0000","last_unstale":"2024-09-16T14:49:49.859102+0000","last_undegraded":"2024-09-16T14:49:49.859102+0000","last_fullsized":"2024-09-16T14:49:49.859102+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"95'82","last_scrub_stamp":"2024-09-16T14:49:42.377752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.377752+0000","objects_scrubbed":2,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:46.676662+0000","scrub_duration":0.002944898,"objects_trimmed":0,"snaptrim_duration":0.0059355980000000003,"stat_sum":{"num_bytes":2473984,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2116,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,3,11,7,6,2,5],"acting":[4,8,3,11,7,6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T14:49:45.874490+0000","last_change":"2024-09-16T14:49:12.961994+0000","last_active":"2024-09-16T14:49:45.874490+0000","last_peered":"2024-09-16T14:49:45.874490+0000","last_clean":"2024-09-16T14:49:45.874490+0000","last_became_active":"2024-09-16T14:49:12.961697+0000","last_became_peered":"2024-09-16T14:49:12.961697+0000","last_unstale":"2024-09-16T14:49:45.874490+0000","last_undegraded":"2024-09-16T14:49:45.874490+0000","last_fullsized":"2024-09-16T14:49:45.874490+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"104'54","reported_seq":337,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.884052+0000","last_change":"2024-09-16T14:49:49.018845+0000","last_active":"2024-09-16T14:49:50.884052+0000","last_peered":"2024-09-16T14:49:50.884052+0000","last_clean":"2024-09-16T14:49:50.884052+0000","last_became_active":"2024-09-16T14:49:37.968269+0000","last_became_peered":"2024-09-16T14:49:37.968269+0000","last_unstale":"2024-09-16T14:49:50.884052+0000","last_undegraded":"2024-09-16T14:49:50.884052+0000","last_fullsized":"2024-09-16T14:49:50.884052+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'48","last_scrub_stamp":"2024-09-16T14:49:43.049407+0000","last_deep_scrub":"91'48","last_deep_scrub_stamp":"2024-09-16T14:49:43.049407+0000","last_clean_scrub_stamp":"2024-09-16T14:49:43.049407+0000","objects_scrubbed":3,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:55.469156+0000","scrub_duration":0.004290266,"objects_trimmed":0,"snaptrim_duration":0.0010204050000000001,"stat_sum":{"num_bytes":2981888,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,5],"acting":[0,1,4,7,8,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.859648+0000","last_change":"2024-09-16T14:48:26.161840+0000","last_active":"2024-09-16T14:49:50.859648+0000","last_peered":"2024-09-16T14:49:50.859648+0000","last_clean":"2024-09-16T14:49:50.859648+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:49:50.859648+0000","last_undegraded":"2024-09-16T14:49:50.859648+0000","last_fullsized":"2024-09-16T14:49:50.859648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"99'155","reported_seq":434,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883243+0000","last_change":"2024-09-16T14:49:49.012411+0000","last_active":"2024-09-16T14:49:49.883243+0000","last_peered":"2024-09-16T14:49:49.883243+0000","last_clean":"2024-09-16T14:49:49.883243+0000","last_became_active":"2024-09-16T14:49:37.961523+0000","last_became_peered":"2024-09-16T14:49:37.961523+0000","last_unstale":"2024-09-16T14:49:49.883243+0000","last_undegraded":"2024-09-16T14:49:49.883243+0000","last_fullsized":"2024-09-16T14:49:49.883243+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'150","last_scrub_stamp":"2024-09-16T14:49:40.650365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.650365+0000","objects_scrubbed":2,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:01.008992+0000","scrub_duration":0.0038572889999999999,"objects_trimmed":0,"snaptrim_duration":0.00085950499999999999,"stat_sum":{"num_bytes":4767744,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":108,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.976066+0000","last_change":"2024-09-16T14:49:43.127050+0000","last_active":"2024-09-16T14:49:49.976066+0000","last_peered":"2024-09-16T14:49:49.976066+0000","last_clean":"2024-09-16T14:49:49.976066+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:49:49.976066+0000","last_undegraded":"2024-09-16T14:49:49.976066+0000","last_fullsized":"2024-09-16T14:49:49.976066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_clean_scrub_stamp":"2024-09-16T14:49:43.126944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:52.930143+0000","scrub_duration":0.0012658890000000001,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"104'80","reported_seq":394,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.941580+0000","last_change":"2024-09-16T14:49:49.016947+0000","last_active":"2024-09-16T14:49:50.941580+0000","last_peered":"2024-09-16T14:49:50.941580+0000","last_clean":"2024-09-16T14:49:50.941580+0000","last_became_active":"2024-09-16T14:49:37.959809+0000","last_became_peered":"2024-09-16T14:49:37.959809+0000","last_unstale":"2024-09-16T14:49:50.941580+0000","last_undegraded":"2024-09-16T14:49:50.941580+0000","last_fullsized":"2024-09-16T14:49:50.941580+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'74","last_scrub_stamp":"2024-09-16T14:49:40.525004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.525004+0000","objects_scrubbed":2,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:04.420225+0000","scrub_duration":0.0030813329999999999,"objects_trimmed":0,"snaptrim_duration":0.0023863780000000002,"stat_sum":{"num_bytes":3194880,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":1538,"num_write":5,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,9,5,10,2,6,0],"acting":[1,11,9,5,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.859573+0000","last_change":"2024-09-16T14:48:26.161822+0000","last_active":"2024-09-16T14:49:50.859573+0000","last_peered":"2024-09-16T14:49:50.859573+0000","last_clean":"2024-09-16T14:49:50.859573+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:49:50.859573+0000","last_undegraded":"2024-09-16T14:49:50.859573+0000","last_fullsized":"2024-09-16T14:49:50.859573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"103'80","reported_seq":464,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.450013+0000","last_change":"2024-09-16T14:49:50.450013+0000","last_active":"2024-09-16T14:49:50.450013+0000","last_peered":"2024-09-16T14:49:50.450013+0000","last_clean":"2024-09-16T14:49:50.450013+0000","last_became_active":"2024-09-16T14:49:37.958130+0000","last_became_peered":"2024-09-16T14:49:37.958130+0000","last_unstale":"2024-09-16T14:49:50.450013+0000","last_undegraded":"2024-09-16T14:49:50.450013+0000","last_fullsized":"2024-09-16T14:49:50.450013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'80","last_scrub_stamp":"2024-09-16T14:49:50.449861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:50.449861+0000","objects_scrubbed":3,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:17.237609+0000","scrub_duration":0.0035009910000000002,"objects_trimmed":0,"snaptrim_duration":0.00080948000000000001,"stat_sum":{"num_bytes":8781824,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,1,11,10],"acting":[5,3,2,6,7,1,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.3","version":"83'37","reported_seq":344,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.859337+0000","last_change":"2024-09-16T14:49:49.023007+0000","last_active":"2024-09-16T14:49:49.859337+0000","last_peered":"2024-09-16T14:49:49.859337+0000","last_clean":"2024-09-16T14:49:49.859337+0000","last_became_active":"2024-09-16T14:49:37.960053+0000","last_became_peered":"2024-09-16T14:49:37.960053+0000","last_unstale":"2024-09-16T14:49:49.859337+0000","last_undegraded":"2024-09-16T14:49:49.859337+0000","last_fullsized":"2024-09-16T14:49:49.859337+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'37","last_scrub_stamp":"2024-09-16T14:49:38.437020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:38.437020+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:44.378876+0000","scrub_duration":0.0029472029999999998,"objects_trimmed":0,"snaptrim_duration":0.0055016789999999998,"stat_sum":{"num_bytes":81920,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":110,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.876420+0000","last_change":"2024-09-16T14:49:28.351058+0000","last_active":"2024-09-16T14:49:49.876420+0000","last_peered":"2024-09-16T14:49:49.876420+0000","last_clean":"2024-09-16T14:49:49.876420+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:49:49.876420+0000","last_undegraded":"2024-09-16T14:49:49.876420+0000","last_fullsized":"2024-09-16T14:49:49.876420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-16T14:49:28.350938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:28.350938+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:42.292353+0000","scrub_duration":0.0018950309999999999,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"91'47","reported_seq":345,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T14:49:51.917258+0000","last_change":"2024-09-16T14:49:49.021090+0000","last_active":"2024-09-16T14:49:51.917258+0000","last_peered":"2024-09-16T14:49:51.917258+0000","last_clean":"2024-09-16T14:49:51.917258+0000","last_became_active":"2024-09-16T14:49:37.961051+0000","last_became_peered":"2024-09-16T14:49:37.961051+0000","last_unstale":"2024-09-16T14:49:51.917258+0000","last_undegraded":"2024-09-16T14:49:51.917258+0000","last_fullsized":"2024-09-16T14:49:51.917258+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'47","last_scrub_stamp":"2024-09-16T14:49:41.126704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:41.126704+0000","objects_scrubbed":2,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:44.106417+0000","scrub_duration":0.0027967320000000001,"objects_trimmed":0,"snaptrim_duration":0.0026831110000000002,"stat_sum":{"num_bytes":5767168,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":9169,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883009+0000","last_change":"2024-09-16T14:49:27.155463+0000","last_active":"2024-09-16T14:49:49.883009+0000","last_peered":"2024-09-16T14:49:49.883009+0000","last_clean":"2024-09-16T14:49:49.883009+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:49:49.883009+0000","last_undegraded":"2024-09-16T14:49:49.883009+0000","last_fullsized":"2024-09-16T14:49:49.883009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:27.155307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:27.155307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:32.986356+0000","scrub_duration":0.001601157,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":279,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.868799+0000","last_change":"2024-09-16T14:49:49.019384+0000","last_active":"2024-09-16T14:49:50.868799+0000","last_peered":"2024-09-16T14:49:50.868799+0000","last_clean":"2024-09-16T14:49:50.868799+0000","last_became_active":"2024-09-16T14:49:37.959116+0000","last_became_peered":"2024-09-16T14:49:37.959116+0000","last_unstale":"2024-09-16T14:49:50.868799+0000","last_undegraded":"2024-09-16T14:49:50.868799+0000","last_fullsized":"2024-09-16T14:49:50.868799+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:42.094901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.094901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:00.419660+0000","scrub_duration":0.0018558680000000001,"objects_trimmed":0,"snaptrim_duration":0.0010354800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882809+0000","last_change":"2024-09-16T14:48:26.568303+0000","last_active":"2024-09-16T14:49:49.882809+0000","last_peered":"2024-09-16T14:49:49.882809+0000","last_clean":"2024-09-16T14:49:49.882809+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:49:49.882809+0000","last_undegraded":"2024-09-16T14:49:49.882809+0000","last_fullsized":"2024-09-16T14:49:49.882809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"98'110","reported_seq":422,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882601+0000","last_change":"2024-09-16T14:49:49.013432+0000","last_active":"2024-09-16T14:49:49.882601+0000","last_peered":"2024-09-16T14:49:49.882601+0000","last_clean":"2024-09-16T14:49:49.882601+0000","last_became_active":"2024-09-16T14:49:37.958372+0000","last_became_peered":"2024-09-16T14:49:37.958372+0000","last_unstale":"2024-09-16T14:49:49.882601+0000","last_undegraded":"2024-09-16T14:49:49.882601+0000","last_fullsized":"2024-09-16T14:49:49.882601+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'102","last_scrub_stamp":"2024-09-16T14:49:38.639259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:38.639259+0000","objects_scrubbed":1,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:50.153167+0000","scrub_duration":0.0030648189999999999,"objects_trimmed":0,"snaptrim_duration":0.001860719,"stat_sum":{"num_bytes":4112384,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":109,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.876474+0000","last_change":"2024-09-16T14:49:40.527258+0000","last_active":"2024-09-16T14:49:49.876474+0000","last_peered":"2024-09-16T14:49:49.876474+0000","last_clean":"2024-09-16T14:49:49.876474+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:49:49.876474+0000","last_undegraded":"2024-09-16T14:49:49.876474+0000","last_fullsized":"2024-09-16T14:49:49.876474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2024-09-16T14:49:40.527164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.527164+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:46.029615+0000","scrub_duration":0.0012857120000000001,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":267,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883100+0000","last_change":"2024-09-16T14:48:22.359960+0000","last_active":"2024-09-16T14:49:49.883100+0000","last_peered":"2024-09-16T14:49:49.883100+0000","last_clean":"2024-09-16T14:49:49.883100+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:49:49.883100+0000","last_undegraded":"2024-09-16T14:49:49.883100+0000","last_fullsized":"2024-09-16T14:49:49.883100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:48:21.118926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"102'105","reported_seq":382,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.911099+0000","last_change":"2024-09-16T14:49:49.034250+0000","last_active":"2024-09-16T14:49:50.911099+0000","last_peered":"2024-09-16T14:49:50.911099+0000","last_clean":"2024-09-16T14:49:50.911099+0000","last_became_active":"2024-09-16T14:49:37.960827+0000","last_became_peered":"2024-09-16T14:49:37.960827+0000","last_unstale":"2024-09-16T14:49:50.911099+0000","last_undegraded":"2024-09-16T14:49:50.911099+0000","last_fullsized":"2024-09-16T14:49:50.911099+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'97","last_scrub_stamp":"2024-09-16T14:49:40.167318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.167318+0000","objects_scrubbed":2,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:55.504556+0000","scrub_duration":0.002425441,"objects_trimmed":0,"snaptrim_duration":0.016083350999999999,"stat_sum":{"num_bytes":4407296,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2,4,3,5,10,8],"acting":[11,1,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.9","version":"102'83","reported_seq":366,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.976342+0000","last_change":"2024-09-16T14:49:49.031507+0000","last_active":"2024-09-16T14:49:49.976342+0000","last_peered":"2024-09-16T14:49:49.976342+0000","last_clean":"2024-09-16T14:49:49.976342+0000","last_became_active":"2024-09-16T14:49:37.960644+0000","last_became_peered":"2024-09-16T14:49:37.960644+0000","last_unstale":"2024-09-16T14:49:49.976342+0000","last_undegraded":"2024-09-16T14:49:49.976342+0000","last_fullsized":"2024-09-16T14:49:49.976342+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'77","last_scrub_stamp":"2024-09-16T14:49:40.131033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.131033+0000","objects_scrubbed":3,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:51.352479+0000","scrub_duration":0.0026976370000000001,"objects_trimmed":1,"snaptrim_duration":0.016770514,"stat_sum":{"num_bytes":5701632,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.a","version":"99'68","reported_seq":194,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T14:49:45.952464+0000","last_change":"2024-09-16T14:49:44.889238+0000","last_active":"2024-09-16T14:49:45.952464+0000","last_peered":"2024-09-16T14:49:45.952464+0000","last_clean":"2024-09-16T14:49:45.952464+0000","last_became_active":"2024-09-16T14:49:19.868841+0000","last_became_peered":"2024-09-16T14:49:19.868841+0000","last_unstale":"2024-09-16T14:49:45.952464+0000","last_undegraded":"2024-09-16T14:49:45.952464+0000","last_fullsized":"2024-09-16T14:49:45.952464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01518982,"stat_sum":{"num_bytes":9125888,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":1063,"num_write":48,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":9224192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,10,8,9,3,7],"acting":[6,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"2"},{"start":"2e","length":"2"},{"start":"31","length":"1"},{"start":"37","length":"5"},{"start":"40","length":"1"},{"start":"43","length":"1"}]},{"pgid":"3.b","version":"90'77","reported_seq":343,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882753+0000","last_change":"2024-09-16T14:49:49.011747+0000","last_active":"2024-09-16T14:49:49.882753+0000","last_peered":"2024-09-16T14:49:49.882753+0000","last_clean":"2024-09-16T14:49:49.882753+0000","last_became_active":"2024-09-16T14:49:12.972366+0000","last_became_peered":"2024-09-16T14:49:12.972366+0000","last_unstale":"2024-09-16T14:49:49.882753+0000","last_undegraded":"2024-09-16T14:49:49.882753+0000","last_fullsized":"2024-09-16T14:49:49.882753+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:03.456665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043729700000000001,"stat_sum":{"num_bytes":1130496,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":7388,"num_write":44,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1785856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,1,6],"acting":[3,0,4,8,11,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.c","version":"102'89","reported_seq":377,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.876733+0000","last_change":"2024-09-16T14:49:49.029918+0000","last_active":"2024-09-16T14:49:49.876733+0000","last_peered":"2024-09-16T14:49:49.876733+0000","last_clean":"2024-09-16T14:49:49.876733+0000","last_became_active":"2024-09-16T14:49:12.979797+0000","last_became_peered":"2024-09-16T14:49:12.979797+0000","last_unstale":"2024-09-16T14:49:49.876733+0000","last_undegraded":"2024-09-16T14:49:49.876733+0000","last_fullsized":"2024-09-16T14:49:49.876733+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.201481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098966060000000005,"stat_sum":{"num_bytes":7389184,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":12010,"num_write":33,"num_write_kb":9920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3440640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.d","version":"98'82","reported_seq":387,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882964+0000","last_change":"2024-09-16T14:49:49.034992+0000","last_active":"2024-09-16T14:49:49.882964+0000","last_peered":"2024-09-16T14:49:49.882964+0000","last_clean":"2024-09-16T14:49:49.882964+0000","last_became_active":"2024-09-16T14:49:12.979812+0000","last_became_peered":"2024-09-16T14:49:12.979812+0000","last_unstale":"2024-09-16T14:49:49.882964+0000","last_undegraded":"2024-09-16T14:49:49.882964+0000","last_fullsized":"2024-09-16T14:49:49.882964+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010903285,"stat_sum":{"num_bytes":5210112,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":10527,"num_write":78,"num_write_kb":7952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4489216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.e","version":"102'86","reported_seq":334,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.911320+0000","last_change":"2024-09-16T14:49:49.035327+0000","last_active":"2024-09-16T14:49:50.911320+0000","last_peered":"2024-09-16T14:49:50.911320+0000","last_clean":"2024-09-16T14:49:50.911320+0000","last_became_active":"2024-09-16T14:48:35.075626+0000","last_became_peered":"2024-09-16T14:48:35.075626+0000","last_unstale":"2024-09-16T14:49:50.911320+0000","last_undegraded":"2024-09-16T14:49:50.911320+0000","last_fullsized":"2024-09-16T14:49:50.911320+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.908348+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.016608049,"stat_sum":{"num_bytes":13107200,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":4441,"num_write":75,"num_write_kb":13360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.f","version":"104'64","reported_seq":307,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.965499+0000","last_change":"2024-09-16T14:49:49.034518+0000","last_active":"2024-09-16T14:49:50.965499+0000","last_peered":"2024-09-16T14:49:50.965499+0000","last_clean":"2024-09-16T14:49:50.965499+0000","last_became_active":"2024-09-16T14:48:35.075111+0000","last_became_peered":"2024-09-16T14:48:35.075111+0000","last_unstale":"2024-09-16T14:49:50.965499+0000","last_undegraded":"2024-09-16T14:49:50.965499+0000","last_fullsized":"2024-09-16T14:49:50.965499+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:58.979883+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01172438,"stat_sum":{"num_bytes":3915776,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":10587,"num_write":62,"num_write_kb":7728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.10","version":"82'43","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T14:49:45.873772+0000","last_change":"2024-09-16T14:49:44.876682+0000","last_active":"2024-09-16T14:49:45.873772+0000","last_peered":"2024-09-16T14:49:45.873772+0000","last_clean":"2024-09-16T14:49:45.873772+0000","last_became_active":"2024-09-16T14:49:39.890294+0000","last_became_peered":"2024-09-16T14:49:39.890294+0000","last_unstale":"2024-09-16T14:49:45.873772+0000","last_undegraded":"2024-09-16T14:49:45.873772+0000","last_fullsized":"2024-09-16T14:49:45.873772+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"82'43","last_scrub_stamp":"2024-09-16T14:49:42.551604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.551604+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:59.224582+0000","scrub_duration":0.0021534729999999999,"objects_trimmed":0,"snaptrim_duration":0.0032657860000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,5,3,0,2,1,7],"acting":[6,9,5,3,0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"2"},{"start":"2e","length":"2"},{"start":"31","length":"1"},{"start":"37","length":"5"},{"start":"40","length":"1"},{"start":"43","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":93,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.873262+0000","last_change":"2024-09-16T14:49:49.015897+0000","last_active":"2024-09-16T14:49:50.873262+0000","last_peered":"2024-09-16T14:49:50.873262+0000","last_clean":"2024-09-16T14:49:50.873262+0000","last_became_active":"2024-09-16T14:49:39.875227+0000","last_became_peered":"2024-09-16T14:49:39.875227+0000","last_unstale":"2024-09-16T14:49:50.873262+0000","last_undegraded":"2024-09-16T14:49:50.873262+0000","last_fullsized":"2024-09-16T14:49:50.873262+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:40.245626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.245626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:41.418675+0000","scrub_duration":0.0016751209999999999,"objects_trimmed":0,"snaptrim_duration":0.00079420900000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6,0,3,9,1,11],"acting":[10,4,6,0,3,9,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.12","version":"96'106","reported_seq":165,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.868833+0000","last_change":"2024-09-16T14:49:49.018091+0000","last_active":"2024-09-16T14:49:50.868833+0000","last_peered":"2024-09-16T14:49:50.868833+0000","last_clean":"2024-09-16T14:49:50.868833+0000","last_became_active":"2024-09-16T14:49:42.870631+0000","last_became_peered":"2024-09-16T14:49:42.870631+0000","last_unstale":"2024-09-16T14:49:50.868833+0000","last_undegraded":"2024-09-16T14:49:50.868833+0000","last_fullsized":"2024-09-16T14:49:50.868833+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060451400000000001,"stat_sum":{"num_bytes":3088384,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":3413,"num_write":30,"num_write_kb":4640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":7602176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1,2,5,6,11],"acting":[0,7,3,1,2,5,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.13","version":"102'45","reported_seq":95,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883456+0000","last_change":"2024-09-16T14:49:49.034362+0000","last_active":"2024-09-16T14:49:49.883456+0000","last_peered":"2024-09-16T14:49:49.883456+0000","last_clean":"2024-09-16T14:49:49.883456+0000","last_became_active":"2024-09-16T14:49:39.885593+0000","last_became_peered":"2024-09-16T14:49:39.885593+0000","last_unstale":"2024-09-16T14:49:49.883456+0000","last_undegraded":"2024-09-16T14:49:49.883456+0000","last_fullsized":"2024-09-16T14:49:49.883456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.011674103,"stat_sum":{"num_bytes":868352,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":5278,"num_write":19,"num_write_kb":1112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1695744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2,6,1,9,8],"acting":[7,5,4,2,6,1,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"46","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":109944832,"num_objects":52,"num_object_clones":11,"num_object_copies":416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":240,"num_read_kb":86818,"num_write":556,"num_write_kb":90688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":49225728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224763904,"data_stored":252866560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1984,"ondisk_log_size":1984,"up":208,"acting":208,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411348,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77140,"kb_used_data":23880,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94294700,"statfs":{"total":96636764160,"available":96557772800,"internally_reserved":0,"allocated":24453120,"data_stored":28269658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52700000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.38300000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.46700000000000003}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.437},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.311}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.57099999999999995}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.48699999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.503},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51800000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59799999999999998}]},{"osd":9,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.622}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.64800000000000002}]}]},{"osd":10,"up_from":18,"seq":77309411350,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68084,"kb_used_data":18920,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94303756,"statfs":{"total":96636764160,"available":96567046144,"internally_reserved":0,"allocated":19374080,"data_stored":20768301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.505},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.48799999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.60399999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57299999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.46100000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.432}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72299999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.39700000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.37},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.38}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.40799999999999997}]}]},{"osd":6,"up_from":41,"seq":176093659150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46620,"kb_used_data":15632,"kb_used_omap":1,"kb_used_meta":30974,"kb_avail":94325220,"statfs":{"total":96636764160,"available":96589025280,"internally_reserved":0,"allocated":16007168,"data_stored":17012453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1965,"internal_metadata":31717459},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":1,"up_from":18,"seq":77309411350,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91184,"kb_used_data":23972,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94280656,"statfs":{"total":96636764160,"available":96543391744,"internally_reserved":0,"allocated":24547328,"data_stored":26425852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.312}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.0590000000000002,"5min":4.0590000000000002,"15min":4.0590000000000002},"last":0.42999999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.503},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":3.9620000000000002,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.55600000000000005}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.79300000000000004}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.67500000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.68700000000000006}]},{"osd":11,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":4.6589999999999998,"5min":4.6589999999999998,"15min":4.6589999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.66500000000000004}]}]},{"osd":0,"up_from":18,"seq":77309411348,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66224,"kb_used_data":17056,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94305616,"statfs":{"total":96636764160,"available":96568950784,"internally_reserved":0,"allocated":17465344,"data_stored":19409466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.41399999999999998}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.5}]},{"osd":8,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.629},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.77800000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57299999999999995}]},{"osd":10,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.745}]},{"osd":11,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.84299999999999997}]}]},{"osd":5,"up_from":18,"seq":77309411349,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68260,"kb_used_data":19088,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94303580,"statfs":{"total":96636764160,"available":96566865920,"internally_reserved":0,"allocated":19546112,"data_stored":20416939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.46000000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.38}]},{"osd":2,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.436}]},{"osd":3,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.30199999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.34000000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.503}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63800000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.73299999999999998}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.624}]}]},{"osd":9,"up_from":17,"seq":73014444053,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69128,"kb_used_data":19956,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94302712,"statfs":{"total":96636764160,"available":96565977088,"internally_reserved":0,"allocated":20434944,"data_stored":20539800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.47499999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.46200000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.40400000000000003}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.38},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.44400000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.627},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.57499999999999996}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.67700000000000005}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.435}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.53900000000000003}]},{"osd":11,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.58999999999999997}]}]},{"osd":8,"up_from":17,"seq":73014444053,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63980,"kb_used_data":14808,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94307860,"statfs":{"total":96636764160,"available":96571248640,"internally_reserved":0,"allocated":15163392,"data_stored":14997912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.59299999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.79700000000000004}]},{"osd":5,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.69699999999999995}]},{"osd":6,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68200000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.74099999999999999}]},{"osd":9,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.77000000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.81100000000000005}]},{"osd":11,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.75800000000000001}]}]},{"osd":7,"up_from":17,"seq":73014444053,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66072,"kb_used_data":16904,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94305768,"statfs":{"total":96636764160,"available":96569106432,"internally_reserved":0,"allocated":17309696,"data_stored":17337272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":50330060},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5620000000000003,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.58299999999999996}]},{"osd":3,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.65800000000000003}]},{"osd":4,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52000000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.313},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.40600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.442}]},{"osd":9,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57199999999999995}]},{"osd":10,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.50700000000000001}]},{"osd":11,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.49199999999999999}]}]},{"osd":4,"up_from":17,"seq":73014444052,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66264,"kb_used_data":17092,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94305576,"statfs":{"total":96636764160,"available":96568909824,"internally_reserved":0,"allocated":17502208,"data_stored":17336728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64200000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.65100000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.435},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.38300000000000001}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.504}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.58199999999999996}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.621},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.59699999999999998}]},{"osd":11,"last update":"Mon Sep 16 14:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.68200000000000005}]}]},{"osd":3,"up_from":17,"seq":73014444054,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71548,"kb_used_data":22376,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94300292,"statfs":{"total":96636764160,"available":96563499008,"internally_reserved":0,"allocated":22913024,"data_stored":23787928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.253},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.32700000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.28399999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.34999999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.46100000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.436},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.51400000000000001}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58199999999999996}]},{"osd":9,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.55200000000000005}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.63800000000000001}]},{"osd":11,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.62},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.60699999999999998}]}]},{"osd":2,"up_from":17,"seq":73014444053,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47048,"kb_used_data":15932,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94324792,"statfs":{"total":96636764160,"available":96588587008,"internally_reserved":0,"allocated":16314368,"data_stored":18921961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.28499999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.38200000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.32300000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46600000000000003}]},{"osd":8,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.51700000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.69799999999999995}]},{"osd":10,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.68600000000000005}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16232448,"data_stored":18636800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23916544,"data_stored":27471872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15093760,"data_stored":18157568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22286336,"data_stored":24829952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17461248,"data_stored":18247680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20078592,"data_stored":22863872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":14835712,"data_stored":16281600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16736256,"data_stored":17719296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":15122432,"data_stored":15908864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":21561344,"data_stored":23965696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":18751488,"data_stored":21815296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":22687744,"data_stored":26968064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:49:53.204 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:49:53.216 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:53.216 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:53.225 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:49:53.284 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:53.283+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:53.304 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:49:53.305 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":106,"fsid":"aaaa1c0f-de8b-48c8-9412-815c8b70b7b8","created":"2024-09-16T14:48:10.063465+0000","modified":"2024-09-16T14:49:52.859707+0000","last_up_change":"2024-09-16T14:48:47.933230+0000","last_in_change":"2024-09-16T14:49:11.939833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:48:20.279244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T14:48:22.580854+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":0.75,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2024-09-16T14:48:32.884410+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":8,"min_size":5,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":26,"pg_placement_num":26,"pg_placement_num_target":26,"pg_num_target":26,"pg_num_pending":26,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"90","auid":0,"snap_mode":"selfmanaged","snap_seq":77,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"lrcprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":16384,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"ffcfdeb7-eb75-4b3a-bc2a-f1834271654c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":725262},{"type":"v1","addr":"172.21.15.40:6801","nonce":725262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":725262},{"type":"v1","addr":"172.21.15.40:6803","nonce":725262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":725262},{"type":"v1","addr":"172.21.15.40:6807","nonce":725262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":725262},{"type":"v1","addr":"172.21.15.40:6805","nonce":725262}]},"public_addr":"172.21.15.40:6801/725262","cluster_addr":"172.21.15.40:6803/725262","heartbeat_back_addr":"172.21.15.40:6807/725262","heartbeat_front_addr":"172.21.15.40:6805/725262","state":["exists","up"]},{"osd":1,"uuid":"6fb8c93f-5e3c-492f-bd0c-6d97764467f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6821","nonce":501335559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6825","nonce":501335559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6831","nonce":501335559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6829","nonce":501335559}]},"public_addr":"172.21.15.40:6821/501335559","cluster_addr":"172.21.15.40:6825/501335559","heartbeat_back_addr":"172.21.15.40:6831/501335559","heartbeat_front_addr":"172.21.15.40:6829/501335559","state":["exists","up"]},{"osd":2,"uuid":"500457a9-0d59-451c-b2ff-28aa363056bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6809","nonce":1490162929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6811","nonce":1490162929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6815","nonce":1490162929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6813","nonce":1490162929}]},"public_addr":"172.21.15.40:6809/1490162929","cluster_addr":"172.21.15.40:6811/1490162929","heartbeat_back_addr":"172.21.15.40:6815/1490162929","heartbeat_front_addr":"172.21.15.40:6813/1490162929","state":["exists","up"]},{"osd":3,"uuid":"f6e48d98-5d87-4cd5-8bd8-6e4c9dc56a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6817","nonce":682463486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6820","nonce":682463486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6828","nonce":682463486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6824","nonce":682463486}]},"public_addr":"172.21.15.40:6817/682463486","cluster_addr":"172.21.15.40:6820/682463486","heartbeat_back_addr":"172.21.15.40:6828/682463486","heartbeat_front_addr":"172.21.15.40:6824/682463486","state":["exists","up"]},{"osd":4,"uuid":"474dd40c-2b73-44ea-b6a9-48605b78125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6818","nonce":1550824417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6819","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6821","nonce":1550824417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6828","nonce":1550824417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6824","nonce":1550824417}]},"public_addr":"172.21.15.111:6818/1550824417","cluster_addr":"172.21.15.111:6821/1550824417","heartbeat_back_addr":"172.21.15.111:6828/1550824417","heartbeat_front_addr":"172.21.15.111:6824/1550824417","state":["exists","up"]},{"osd":5,"uuid":"479bfe9b-00f6-4763-9740-9c9fd11e2587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6801","nonce":3785556374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6803","nonce":3785556374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6807","nonce":3785556374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6805","nonce":3785556374}]},"public_addr":"172.21.15.111:6801/3785556374","cluster_addr":"172.21.15.111:6803/3785556374","heartbeat_back_addr":"172.21.15.111:6807/3785556374","heartbeat_front_addr":"172.21.15.111:6805/3785556374","state":["exists","up"]},{"osd":6,"uuid":"6ac36041-045e-4813-ad35-ca4e224811fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":23,"up_from":41,"up_thru":92,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6820","nonce":722937064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6825","nonce":722937064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6831","nonce":722937064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6827","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6829","nonce":722937064}]},"public_addr":"172.21.15.111:6820/722937064","cluster_addr":"172.21.15.111:6825/722937064","heartbeat_back_addr":"172.21.15.111:6831/722937064","heartbeat_front_addr":"172.21.15.111:6829/722937064","state":["exists","up"]},{"osd":7,"uuid":"129006cf-b82b-4e11-a007-63deb355d007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6809","nonce":3591930331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6811","nonce":3591930331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6815","nonce":3591930331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6813","nonce":3591930331}]},"public_addr":"172.21.15.111:6809/3591930331","cluster_addr":"172.21.15.111:6811/3591930331","heartbeat_back_addr":"172.21.15.111:6815/3591930331","heartbeat_front_addr":"172.21.15.111:6813/3591930331","state":["exists","up"]},{"osd":8,"uuid":"27e746ee-f67a-4a73-8976-86d012469657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6817","nonce":2473986485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6819","nonce":2473986485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6824","nonce":2473986485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6821","nonce":2473986485}]},"public_addr":"172.21.15.114:6817/2473986485","cluster_addr":"172.21.15.114:6819/2473986485","heartbeat_back_addr":"172.21.15.114:6824/2473986485","heartbeat_front_addr":"172.21.15.114:6821/2473986485","state":["exists","up"]},{"osd":9,"uuid":"90975d4f-24b3-4c84-916e-3138bc52a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6801","nonce":2290892207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6803","nonce":2290892207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6807","nonce":2290892207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6805","nonce":2290892207}]},"public_addr":"172.21.15.114:6801/2290892207","cluster_addr":"172.21.15.114:6803/2290892207","heartbeat_back_addr":"172.21.15.114:6807/2290892207","heartbeat_front_addr":"172.21.15.114:6805/2290892207","state":["exists","up"]},{"osd":10,"uuid":"89c8157b-52ce-4c1c-9df7-e214df30a827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6809","nonce":1795470140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6811","nonce":1795470140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6815","nonce":1795470140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6813","nonce":1795470140}]},"public_addr":"172.21.15.114:6809/1795470140","cluster_addr":"172.21.15.114:6811/1795470140","heartbeat_back_addr":"172.21.15.114:6815/1795470140","heartbeat_front_addr":"172.21.15.114:6813/1795470140","state":["exists","up"]},{"osd":11,"uuid":"0dfbd657-c9f0-42f9-9b43-323c45d89c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6825","nonce":1823608150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6827","nonce":1823608150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6831","nonce":1823608150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6829","nonce":1823608150}]},"public_addr":"172.21.15.114:6825/1823608150","cluster_addr":"172.21.15.114:6827/1823608150","heartbeat_back_addr":"172.21.15.114:6831/1823608150","heartbeat_front_addr":"172.21.15.114:6829/1823608150","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:21.058050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:29.713159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:46.059619+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:45.409424+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:19.879455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:39.305292+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:32.379251+0000","dead_epoch":25},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:25.580810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:32.215102+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:19.285409+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:36.675606+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:32.320290+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"lrcprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","l":"3","m":"2","name":"lrcprofile","plugin":"lrc"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":70,"length":1},{"begin":76,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":76,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:49:53.318 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:49:53.328 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:53.329 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:53.338 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:49:53.382 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:53.381+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:53.440 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:53.441 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:53.449 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T14:49:53.453 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:53.452+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:53.533 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:49:53.534 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:49:53.545 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-16T14:49:52.862669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110535219,"num_objects":56,"num_object_clones":11,"num_object_copies":424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":366,"num_read_kb":86927,"num_write":739,"num_write_kb":93628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":49225728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2124,"ondisk_log_size":2124,"up":226,"acting":226,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":226,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":801552,"kb_used_data":225616,"kb_used_omap":19,"kb_used_meta":575724,"kb_avail":1131660528,"statfs":{"total":1159641169920,"available":1158820380672,"internally_reserved":0,"allocated":231030784,"data_stored":245224270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19502,"internal_metadata":589542354},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18292736,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":-11,"num_read_kb":4382,"num_write":-50,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-19,"num_bytes_recovered":-6012928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.852439"},"pg_stats":[{"pgid":"3.19","version":"105'90","reported_seq":129,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T14:49:51.915643+0000","last_change":"2024-09-16T14:49:49.018972+0000","last_active":"2024-09-16T14:49:51.915643+0000","last_peered":"2024-09-16T14:49:51.915643+0000","last_clean":"2024-09-16T14:49:51.915643+0000","last_became_active":"2024-09-16T14:49:39.892805+0000","last_became_peered":"2024-09-16T14:49:39.892805+0000","last_unstale":"2024-09-16T14:49:51.915643+0000","last_undegraded":"2024-09-16T14:49:51.915643+0000","last_fullsized":"2024-09-16T14:49:51.915643+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"95'81","last_scrub_stamp":"2024-09-16T14:49:42.521147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.521147+0000","objects_scrubbed":3,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:10.802992+0000","scrub_duration":0.003238852,"objects_trimmed":0,"snaptrim_duration":0.0042894789999999997,"stat_sum":{"num_bytes":7438336,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":2320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,10,5,7,11,2],"acting":[1,3,4,10,5,7,11,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.18","version":"102'122","reported_seq":171,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.716793+0000","last_change":"2024-09-16T14:49:50.716793+0000","last_active":"2024-09-16T14:49:50.716793+0000","last_peered":"2024-09-16T14:49:50.716793+0000","last_clean":"2024-09-16T14:49:50.716793+0000","last_became_active":"2024-09-16T14:49:39.893130+0000","last_became_peered":"2024-09-16T14:49:39.893130+0000","last_unstale":"2024-09-16T14:49:50.716793+0000","last_undegraded":"2024-09-16T14:49:50.716793+0000","last_fullsized":"2024-09-16T14:49:50.716793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"102'122","last_scrub_stamp":"2024-09-16T14:49:50.716587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:50.716587+0000","objects_scrubbed":2,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:17.481776+0000","scrub_duration":0.0042360469999999997,"objects_trimmed":1,"snaptrim_duration":0.021640853000000002,"stat_sum":{"num_bytes":6111232,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,1,2,8,7,4,5],"acting":[3,9,1,2,8,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.17","version":"104'172","reported_seq":224,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.926382+0000","last_change":"2024-09-16T14:49:49.034483+0000","last_active":"2024-09-16T14:49:50.926382+0000","last_peered":"2024-09-16T14:49:50.926382+0000","last_clean":"2024-09-16T14:49:50.926382+0000","last_became_active":"2024-09-16T14:49:42.869291+0000","last_became_peered":"2024-09-16T14:49:42.869291+0000","last_unstale":"2024-09-16T14:49:50.926382+0000","last_undegraded":"2024-09-16T14:49:50.926382+0000","last_fullsized":"2024-09-16T14:49:50.926382+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015625354000000001,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":18847,"num_write":90,"num_write_kb":17184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":20987904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,9,3,11,6,2,1],"acting":[0,5,9,3,11,6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.16","version":"83'46","reported_seq":52,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.877236+0000","last_change":"2024-09-16T14:49:49.020712+0000","last_active":"2024-09-16T14:49:49.877236+0000","last_peered":"2024-09-16T14:49:49.877236+0000","last_clean":"2024-09-16T14:49:49.877236+0000","last_became_active":"2024-09-16T14:49:39.885128+0000","last_became_peered":"2024-09-16T14:49:39.885128+0000","last_unstale":"2024-09-16T14:49:49.877236+0000","last_undegraded":"2024-09-16T14:49:49.877236+0000","last_fullsized":"2024-09-16T14:49:49.877236+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078294200000000001,"stat_sum":{"num_bytes":1900544,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":436,"num_write":31,"num_write_kb":2696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3,8,2,10,6],"acting":[5,7,1,3,8,2,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.15","version":"83'61","reported_seq":64,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.976286+0000","last_change":"2024-09-16T14:49:49.016770+0000","last_active":"2024-09-16T14:49:49.976286+0000","last_peered":"2024-09-16T14:49:49.976286+0000","last_clean":"2024-09-16T14:49:49.976286+0000","last_became_active":"2024-09-16T14:49:39.883704+0000","last_became_peered":"2024-09-16T14:49:39.883704+0000","last_unstale":"2024-09-16T14:49:49.976286+0000","last_undegraded":"2024-09-16T14:49:49.976286+0000","last_fullsized":"2024-09-16T14:49:49.976286+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"83'61","last_scrub_stamp":"2024-09-16T14:49:42.159081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.159081+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:49.030695+0000","scrub_duration":0.0019877559999999998,"objects_trimmed":0,"snaptrim_duration":0.0015822239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,4,2,6,0,9],"acting":[8,11,7,4,2,6,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.14","version":"95'82","reported_seq":128,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.859102+0000","last_change":"2024-09-16T14:49:49.022706+0000","last_active":"2024-09-16T14:49:49.859102+0000","last_peered":"2024-09-16T14:49:49.859102+0000","last_clean":"2024-09-16T14:49:49.859102+0000","last_became_active":"2024-09-16T14:49:39.886232+0000","last_became_peered":"2024-09-16T14:49:39.886232+0000","last_unstale":"2024-09-16T14:49:49.859102+0000","last_undegraded":"2024-09-16T14:49:49.859102+0000","last_fullsized":"2024-09-16T14:49:49.859102+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"95'82","last_scrub_stamp":"2024-09-16T14:49:42.377752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.377752+0000","objects_scrubbed":2,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:46.676662+0000","scrub_duration":0.002944898,"objects_trimmed":0,"snaptrim_duration":0.0059355980000000003,"stat_sum":{"num_bytes":2473984,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2116,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,3,11,7,6,2,5],"acting":[4,8,3,11,7,6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T14:49:45.874490+0000","last_change":"2024-09-16T14:49:12.961994+0000","last_active":"2024-09-16T14:49:45.874490+0000","last_peered":"2024-09-16T14:49:45.874490+0000","last_clean":"2024-09-16T14:49:45.874490+0000","last_became_active":"2024-09-16T14:49:12.961697+0000","last_became_peered":"2024-09-16T14:49:12.961697+0000","last_unstale":"2024-09-16T14:49:45.874490+0000","last_undegraded":"2024-09-16T14:49:45.874490+0000","last_fullsized":"2024-09-16T14:49:45.874490+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"104'54","reported_seq":337,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.884052+0000","last_change":"2024-09-16T14:49:49.018845+0000","last_active":"2024-09-16T14:49:50.884052+0000","last_peered":"2024-09-16T14:49:50.884052+0000","last_clean":"2024-09-16T14:49:50.884052+0000","last_became_active":"2024-09-16T14:49:37.968269+0000","last_became_peered":"2024-09-16T14:49:37.968269+0000","last_unstale":"2024-09-16T14:49:50.884052+0000","last_undegraded":"2024-09-16T14:49:50.884052+0000","last_fullsized":"2024-09-16T14:49:50.884052+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'48","last_scrub_stamp":"2024-09-16T14:49:43.049407+0000","last_deep_scrub":"91'48","last_deep_scrub_stamp":"2024-09-16T14:49:43.049407+0000","last_clean_scrub_stamp":"2024-09-16T14:49:43.049407+0000","objects_scrubbed":3,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:55.469156+0000","scrub_duration":0.004290266,"objects_trimmed":0,"snaptrim_duration":0.0010204050000000001,"stat_sum":{"num_bytes":2981888,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,5],"acting":[0,1,4,7,8,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.859648+0000","last_change":"2024-09-16T14:48:26.161840+0000","last_active":"2024-09-16T14:49:50.859648+0000","last_peered":"2024-09-16T14:49:50.859648+0000","last_clean":"2024-09-16T14:49:50.859648+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:49:50.859648+0000","last_undegraded":"2024-09-16T14:49:50.859648+0000","last_fullsized":"2024-09-16T14:49:50.859648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"99'155","reported_seq":434,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883243+0000","last_change":"2024-09-16T14:49:49.012411+0000","last_active":"2024-09-16T14:49:49.883243+0000","last_peered":"2024-09-16T14:49:49.883243+0000","last_clean":"2024-09-16T14:49:49.883243+0000","last_became_active":"2024-09-16T14:49:37.961523+0000","last_became_peered":"2024-09-16T14:49:37.961523+0000","last_unstale":"2024-09-16T14:49:49.883243+0000","last_undegraded":"2024-09-16T14:49:49.883243+0000","last_fullsized":"2024-09-16T14:49:49.883243+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'150","last_scrub_stamp":"2024-09-16T14:49:40.650365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.650365+0000","objects_scrubbed":2,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:01.008992+0000","scrub_duration":0.0038572889999999999,"objects_trimmed":0,"snaptrim_duration":0.00085950499999999999,"stat_sum":{"num_bytes":4767744,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":108,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.976066+0000","last_change":"2024-09-16T14:49:43.127050+0000","last_active":"2024-09-16T14:49:49.976066+0000","last_peered":"2024-09-16T14:49:49.976066+0000","last_clean":"2024-09-16T14:49:49.976066+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:49:49.976066+0000","last_undegraded":"2024-09-16T14:49:49.976066+0000","last_fullsized":"2024-09-16T14:49:49.976066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_clean_scrub_stamp":"2024-09-16T14:49:43.126944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:52.930143+0000","scrub_duration":0.0012658890000000001,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"104'80","reported_seq":394,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.941580+0000","last_change":"2024-09-16T14:49:49.016947+0000","last_active":"2024-09-16T14:49:50.941580+0000","last_peered":"2024-09-16T14:49:50.941580+0000","last_clean":"2024-09-16T14:49:50.941580+0000","last_became_active":"2024-09-16T14:49:37.959809+0000","last_became_peered":"2024-09-16T14:49:37.959809+0000","last_unstale":"2024-09-16T14:49:50.941580+0000","last_undegraded":"2024-09-16T14:49:50.941580+0000","last_fullsized":"2024-09-16T14:49:50.941580+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'74","last_scrub_stamp":"2024-09-16T14:49:40.525004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.525004+0000","objects_scrubbed":2,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:04.420225+0000","scrub_duration":0.0030813329999999999,"objects_trimmed":0,"snaptrim_duration":0.0023863780000000002,"stat_sum":{"num_bytes":3194880,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":1538,"num_write":5,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,9,5,10,2,6,0],"acting":[1,11,9,5,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.859573+0000","last_change":"2024-09-16T14:48:26.161822+0000","last_active":"2024-09-16T14:49:50.859573+0000","last_peered":"2024-09-16T14:49:50.859573+0000","last_clean":"2024-09-16T14:49:50.859573+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:49:50.859573+0000","last_undegraded":"2024-09-16T14:49:50.859573+0000","last_fullsized":"2024-09-16T14:49:50.859573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"103'80","reported_seq":464,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.450013+0000","last_change":"2024-09-16T14:49:50.450013+0000","last_active":"2024-09-16T14:49:50.450013+0000","last_peered":"2024-09-16T14:49:50.450013+0000","last_clean":"2024-09-16T14:49:50.450013+0000","last_became_active":"2024-09-16T14:49:37.958130+0000","last_became_peered":"2024-09-16T14:49:37.958130+0000","last_unstale":"2024-09-16T14:49:50.450013+0000","last_undegraded":"2024-09-16T14:49:50.450013+0000","last_fullsized":"2024-09-16T14:49:50.450013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'80","last_scrub_stamp":"2024-09-16T14:49:50.449861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:50.449861+0000","objects_scrubbed":3,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:17.237609+0000","scrub_duration":0.0035009910000000002,"objects_trimmed":0,"snaptrim_duration":0.00080948000000000001,"stat_sum":{"num_bytes":8781824,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,1,11,10],"acting":[5,3,2,6,7,1,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.3","version":"83'37","reported_seq":344,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.859337+0000","last_change":"2024-09-16T14:49:49.023007+0000","last_active":"2024-09-16T14:49:49.859337+0000","last_peered":"2024-09-16T14:49:49.859337+0000","last_clean":"2024-09-16T14:49:49.859337+0000","last_became_active":"2024-09-16T14:49:37.960053+0000","last_became_peered":"2024-09-16T14:49:37.960053+0000","last_unstale":"2024-09-16T14:49:49.859337+0000","last_undegraded":"2024-09-16T14:49:49.859337+0000","last_fullsized":"2024-09-16T14:49:49.859337+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'37","last_scrub_stamp":"2024-09-16T14:49:38.437020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:38.437020+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:44.378876+0000","scrub_duration":0.0029472029999999998,"objects_trimmed":0,"snaptrim_duration":0.0055016789999999998,"stat_sum":{"num_bytes":81920,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":110,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.876420+0000","last_change":"2024-09-16T14:49:28.351058+0000","last_active":"2024-09-16T14:49:49.876420+0000","last_peered":"2024-09-16T14:49:49.876420+0000","last_clean":"2024-09-16T14:49:49.876420+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:49:49.876420+0000","last_undegraded":"2024-09-16T14:49:49.876420+0000","last_fullsized":"2024-09-16T14:49:49.876420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-16T14:49:28.350938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:28.350938+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:42.292353+0000","scrub_duration":0.0018950309999999999,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"91'47","reported_seq":345,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T14:49:51.917258+0000","last_change":"2024-09-16T14:49:49.021090+0000","last_active":"2024-09-16T14:49:51.917258+0000","last_peered":"2024-09-16T14:49:51.917258+0000","last_clean":"2024-09-16T14:49:51.917258+0000","last_became_active":"2024-09-16T14:49:37.961051+0000","last_became_peered":"2024-09-16T14:49:37.961051+0000","last_unstale":"2024-09-16T14:49:51.917258+0000","last_undegraded":"2024-09-16T14:49:51.917258+0000","last_fullsized":"2024-09-16T14:49:51.917258+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'47","last_scrub_stamp":"2024-09-16T14:49:41.126704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:41.126704+0000","objects_scrubbed":2,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:44.106417+0000","scrub_duration":0.0027967320000000001,"objects_trimmed":0,"snaptrim_duration":0.0026831110000000002,"stat_sum":{"num_bytes":5767168,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":9169,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883009+0000","last_change":"2024-09-16T14:49:27.155463+0000","last_active":"2024-09-16T14:49:49.883009+0000","last_peered":"2024-09-16T14:49:49.883009+0000","last_clean":"2024-09-16T14:49:49.883009+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:49:49.883009+0000","last_undegraded":"2024-09-16T14:49:49.883009+0000","last_fullsized":"2024-09-16T14:49:49.883009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:27.155307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:27.155307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:32.986356+0000","scrub_duration":0.001601157,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":279,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.868799+0000","last_change":"2024-09-16T14:49:49.019384+0000","last_active":"2024-09-16T14:49:50.868799+0000","last_peered":"2024-09-16T14:49:50.868799+0000","last_clean":"2024-09-16T14:49:50.868799+0000","last_became_active":"2024-09-16T14:49:37.959116+0000","last_became_peered":"2024-09-16T14:49:37.959116+0000","last_unstale":"2024-09-16T14:49:50.868799+0000","last_undegraded":"2024-09-16T14:49:50.868799+0000","last_fullsized":"2024-09-16T14:49:50.868799+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:42.094901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.094901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:00.419660+0000","scrub_duration":0.0018558680000000001,"objects_trimmed":0,"snaptrim_duration":0.0010354800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882809+0000","last_change":"2024-09-16T14:48:26.568303+0000","last_active":"2024-09-16T14:49:49.882809+0000","last_peered":"2024-09-16T14:49:49.882809+0000","last_clean":"2024-09-16T14:49:49.882809+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:49:49.882809+0000","last_undegraded":"2024-09-16T14:49:49.882809+0000","last_fullsized":"2024-09-16T14:49:49.882809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"98'110","reported_seq":422,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882601+0000","last_change":"2024-09-16T14:49:49.013432+0000","last_active":"2024-09-16T14:49:49.882601+0000","last_peered":"2024-09-16T14:49:49.882601+0000","last_clean":"2024-09-16T14:49:49.882601+0000","last_became_active":"2024-09-16T14:49:37.958372+0000","last_became_peered":"2024-09-16T14:49:37.958372+0000","last_unstale":"2024-09-16T14:49:49.882601+0000","last_undegraded":"2024-09-16T14:49:49.882601+0000","last_fullsized":"2024-09-16T14:49:49.882601+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'102","last_scrub_stamp":"2024-09-16T14:49:38.639259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:38.639259+0000","objects_scrubbed":1,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:50.153167+0000","scrub_duration":0.0030648189999999999,"objects_trimmed":0,"snaptrim_duration":0.001860719,"stat_sum":{"num_bytes":4112384,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":109,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.876474+0000","last_change":"2024-09-16T14:49:40.527258+0000","last_active":"2024-09-16T14:49:49.876474+0000","last_peered":"2024-09-16T14:49:49.876474+0000","last_clean":"2024-09-16T14:49:49.876474+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:49:49.876474+0000","last_undegraded":"2024-09-16T14:49:49.876474+0000","last_fullsized":"2024-09-16T14:49:49.876474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2024-09-16T14:49:40.527164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.527164+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:46.029615+0000","scrub_duration":0.0012857120000000001,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":267,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883100+0000","last_change":"2024-09-16T14:48:22.359960+0000","last_active":"2024-09-16T14:49:49.883100+0000","last_peered":"2024-09-16T14:49:49.883100+0000","last_clean":"2024-09-16T14:49:49.883100+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:49:49.883100+0000","last_undegraded":"2024-09-16T14:49:49.883100+0000","last_fullsized":"2024-09-16T14:49:49.883100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:48:21.118926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"102'105","reported_seq":382,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.911099+0000","last_change":"2024-09-16T14:49:49.034250+0000","last_active":"2024-09-16T14:49:50.911099+0000","last_peered":"2024-09-16T14:49:50.911099+0000","last_clean":"2024-09-16T14:49:50.911099+0000","last_became_active":"2024-09-16T14:49:37.960827+0000","last_became_peered":"2024-09-16T14:49:37.960827+0000","last_unstale":"2024-09-16T14:49:50.911099+0000","last_undegraded":"2024-09-16T14:49:50.911099+0000","last_fullsized":"2024-09-16T14:49:50.911099+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'97","last_scrub_stamp":"2024-09-16T14:49:40.167318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.167318+0000","objects_scrubbed":2,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:55.504556+0000","scrub_duration":0.002425441,"objects_trimmed":0,"snaptrim_duration":0.016083350999999999,"stat_sum":{"num_bytes":4407296,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2,4,3,5,10,8],"acting":[11,1,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.9","version":"102'83","reported_seq":366,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.976342+0000","last_change":"2024-09-16T14:49:49.031507+0000","last_active":"2024-09-16T14:49:49.976342+0000","last_peered":"2024-09-16T14:49:49.976342+0000","last_clean":"2024-09-16T14:49:49.976342+0000","last_became_active":"2024-09-16T14:49:37.960644+0000","last_became_peered":"2024-09-16T14:49:37.960644+0000","last_unstale":"2024-09-16T14:49:49.976342+0000","last_undegraded":"2024-09-16T14:49:49.976342+0000","last_fullsized":"2024-09-16T14:49:49.976342+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'77","last_scrub_stamp":"2024-09-16T14:49:40.131033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.131033+0000","objects_scrubbed":3,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:51.352479+0000","scrub_duration":0.0026976370000000001,"objects_trimmed":1,"snaptrim_duration":0.016770514,"stat_sum":{"num_bytes":5701632,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.a","version":"99'68","reported_seq":194,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T14:49:45.952464+0000","last_change":"2024-09-16T14:49:44.889238+0000","last_active":"2024-09-16T14:49:45.952464+0000","last_peered":"2024-09-16T14:49:45.952464+0000","last_clean":"2024-09-16T14:49:45.952464+0000","last_became_active":"2024-09-16T14:49:19.868841+0000","last_became_peered":"2024-09-16T14:49:19.868841+0000","last_unstale":"2024-09-16T14:49:45.952464+0000","last_undegraded":"2024-09-16T14:49:45.952464+0000","last_fullsized":"2024-09-16T14:49:45.952464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01518982,"stat_sum":{"num_bytes":9125888,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":1063,"num_write":48,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":9224192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,10,8,9,3,7],"acting":[6,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"2"},{"start":"2e","length":"2"},{"start":"31","length":"1"},{"start":"37","length":"5"},{"start":"40","length":"1"},{"start":"43","length":"1"}]},{"pgid":"3.b","version":"90'77","reported_seq":343,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882753+0000","last_change":"2024-09-16T14:49:49.011747+0000","last_active":"2024-09-16T14:49:49.882753+0000","last_peered":"2024-09-16T14:49:49.882753+0000","last_clean":"2024-09-16T14:49:49.882753+0000","last_became_active":"2024-09-16T14:49:12.972366+0000","last_became_peered":"2024-09-16T14:49:12.972366+0000","last_unstale":"2024-09-16T14:49:49.882753+0000","last_undegraded":"2024-09-16T14:49:49.882753+0000","last_fullsized":"2024-09-16T14:49:49.882753+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:03.456665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043729700000000001,"stat_sum":{"num_bytes":1130496,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":7388,"num_write":44,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1785856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,1,6],"acting":[3,0,4,8,11,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.c","version":"102'89","reported_seq":377,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.876733+0000","last_change":"2024-09-16T14:49:49.029918+0000","last_active":"2024-09-16T14:49:49.876733+0000","last_peered":"2024-09-16T14:49:49.876733+0000","last_clean":"2024-09-16T14:49:49.876733+0000","last_became_active":"2024-09-16T14:49:12.979797+0000","last_became_peered":"2024-09-16T14:49:12.979797+0000","last_unstale":"2024-09-16T14:49:49.876733+0000","last_undegraded":"2024-09-16T14:49:49.876733+0000","last_fullsized":"2024-09-16T14:49:49.876733+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.201481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098966060000000005,"stat_sum":{"num_bytes":7389184,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":12010,"num_write":33,"num_write_kb":9920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3440640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.d","version":"98'82","reported_seq":387,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882964+0000","last_change":"2024-09-16T14:49:49.034992+0000","last_active":"2024-09-16T14:49:49.882964+0000","last_peered":"2024-09-16T14:49:49.882964+0000","last_clean":"2024-09-16T14:49:49.882964+0000","last_became_active":"2024-09-16T14:49:12.979812+0000","last_became_peered":"2024-09-16T14:49:12.979812+0000","last_unstale":"2024-09-16T14:49:49.882964+0000","last_undegraded":"2024-09-16T14:49:49.882964+0000","last_fullsized":"2024-09-16T14:49:49.882964+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010903285,"stat_sum":{"num_bytes":5210112,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":10527,"num_write":78,"num_write_kb":7952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4489216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.e","version":"102'86","reported_seq":334,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.911320+0000","last_change":"2024-09-16T14:49:49.035327+0000","last_active":"2024-09-16T14:49:50.911320+0000","last_peered":"2024-09-16T14:49:50.911320+0000","last_clean":"2024-09-16T14:49:50.911320+0000","last_became_active":"2024-09-16T14:48:35.075626+0000","last_became_peered":"2024-09-16T14:48:35.075626+0000","last_unstale":"2024-09-16T14:49:50.911320+0000","last_undegraded":"2024-09-16T14:49:50.911320+0000","last_fullsized":"2024-09-16T14:49:50.911320+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.908348+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.016608049,"stat_sum":{"num_bytes":13107200,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":4441,"num_write":75,"num_write_kb":13360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.f","version":"104'64","reported_seq":307,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.965499+0000","last_change":"2024-09-16T14:49:49.034518+0000","last_active":"2024-09-16T14:49:50.965499+0000","last_peered":"2024-09-16T14:49:50.965499+0000","last_clean":"2024-09-16T14:49:50.965499+0000","last_became_active":"2024-09-16T14:48:35.075111+0000","last_became_peered":"2024-09-16T14:48:35.075111+0000","last_unstale":"2024-09-16T14:49:50.965499+0000","last_undegraded":"2024-09-16T14:49:50.965499+0000","last_fullsized":"2024-09-16T14:49:50.965499+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:58.979883+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01172438,"stat_sum":{"num_bytes":3915776,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":10587,"num_write":62,"num_write_kb":7728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.10","version":"82'43","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T14:49:45.873772+0000","last_change":"2024-09-16T14:49:44.876682+0000","last_active":"2024-09-16T14:49:45.873772+0000","last_peered":"2024-09-16T14:49:45.873772+0000","last_clean":"2024-09-16T14:49:45.873772+0000","last_became_active":"2024-09-16T14:49:39.890294+0000","last_became_peered":"2024-09-16T14:49:39.890294+0000","last_unstale":"2024-09-16T14:49:45.873772+0000","last_undegraded":"2024-09-16T14:49:45.873772+0000","last_fullsized":"2024-09-16T14:49:45.873772+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"82'43","last_scrub_stamp":"2024-09-16T14:49:42.551604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.551604+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:59.224582+0000","scrub_duration":0.0021534729999999999,"objects_trimmed":0,"snaptrim_duration":0.0032657860000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,5,3,0,2,1,7],"acting":[6,9,5,3,0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"2"},{"start":"2e","length":"2"},{"start":"31","length":"1"},{"start":"37","length":"5"},{"start":"40","length":"1"},{"start":"43","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":93,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.873262+0000","last_change":"2024-09-16T14:49:49.015897+0000","last_active":"2024-09-16T14:49:50.873262+0000","last_peered":"2024-09-16T14:49:50.873262+0000","last_clean":"2024-09-16T14:49:50.873262+0000","last_became_active":"2024-09-16T14:49:39.875227+0000","last_became_peered":"2024-09-16T14:49:39.875227+0000","last_unstale":"2024-09-16T14:49:50.873262+0000","last_undegraded":"2024-09-16T14:49:50.873262+0000","last_fullsized":"2024-09-16T14:49:50.873262+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:40.245626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.245626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:41.418675+0000","scrub_duration":0.0016751209999999999,"objects_trimmed":0,"snaptrim_duration":0.00079420900000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6,0,3,9,1,11],"acting":[10,4,6,0,3,9,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.12","version":"96'106","reported_seq":165,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.868833+0000","last_change":"2024-09-16T14:49:49.018091+0000","last_active":"2024-09-16T14:49:50.868833+0000","last_peered":"2024-09-16T14:49:50.868833+0000","last_clean":"2024-09-16T14:49:50.868833+0000","last_became_active":"2024-09-16T14:49:42.870631+0000","last_became_peered":"2024-09-16T14:49:42.870631+0000","last_unstale":"2024-09-16T14:49:50.868833+0000","last_undegraded":"2024-09-16T14:49:50.868833+0000","last_fullsized":"2024-09-16T14:49:50.868833+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060451400000000001,"stat_sum":{"num_bytes":3088384,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":3413,"num_write":30,"num_write_kb":4640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":7602176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1,2,5,6,11],"acting":[0,7,3,1,2,5,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.13","version":"102'45","reported_seq":95,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883456+0000","last_change":"2024-09-16T14:49:49.034362+0000","last_active":"2024-09-16T14:49:49.883456+0000","last_peered":"2024-09-16T14:49:49.883456+0000","last_clean":"2024-09-16T14:49:49.883456+0000","last_became_active":"2024-09-16T14:49:39.885593+0000","last_became_peered":"2024-09-16T14:49:39.885593+0000","last_unstale":"2024-09-16T14:49:49.883456+0000","last_undegraded":"2024-09-16T14:49:49.883456+0000","last_fullsized":"2024-09-16T14:49:49.883456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.011674103,"stat_sum":{"num_bytes":868352,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":5278,"num_write":19,"num_write_kb":1112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1695744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2,6,1,9,8],"acting":[7,5,4,2,6,1,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"46","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":109944832,"num_objects":52,"num_object_clones":11,"num_object_copies":416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":240,"num_read_kb":86818,"num_write":556,"num_write_kb":90688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":49225728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224763904,"data_stored":252866560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1984,"ondisk_log_size":1984,"up":208,"acting":208,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411348,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77140,"kb_used_data":23880,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94294700,"statfs":{"total":96636764160,"available":96557772800,"internally_reserved":0,"allocated":24453120,"data_stored":28269658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52700000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.38300000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.46700000000000003}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.437},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.311}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.57099999999999995}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.48699999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.503},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51800000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59799999999999998}]},{"osd":9,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.622}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.64800000000000002}]}]},{"osd":10,"up_from":18,"seq":77309411350,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68084,"kb_used_data":18920,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94303756,"statfs":{"total":96636764160,"available":96567046144,"internally_reserved":0,"allocated":19374080,"data_stored":20768301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.505},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.48799999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.60399999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57299999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.46100000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.432}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72299999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.39700000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.37},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.38}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.40799999999999997}]}]},{"osd":6,"up_from":41,"seq":176093659150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46620,"kb_used_data":15632,"kb_used_omap":1,"kb_used_meta":30974,"kb_avail":94325220,"statfs":{"total":96636764160,"available":96589025280,"internally_reserved":0,"allocated":16007168,"data_stored":17012453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1965,"internal_metadata":31717459},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":1,"up_from":18,"seq":77309411350,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91184,"kb_used_data":23972,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94280656,"statfs":{"total":96636764160,"available":96543391744,"internally_reserved":0,"allocated":24547328,"data_stored":26425852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.312}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.0590000000000002,"5min":4.0590000000000002,"15min":4.0590000000000002},"last":0.42999999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.503},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":3.9620000000000002,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.55600000000000005}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.79300000000000004}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.67500000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.68700000000000006}]},{"osd":11,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":4.6589999999999998,"5min":4.6589999999999998,"15min":4.6589999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.66500000000000004}]}]},{"osd":0,"up_from":18,"seq":77309411348,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66224,"kb_used_data":17056,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94305616,"statfs":{"total":96636764160,"available":96568950784,"internally_reserved":0,"allocated":17465344,"data_stored":19409466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.41399999999999998}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.5}]},{"osd":8,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.629},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.77800000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57299999999999995}]},{"osd":10,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.745}]},{"osd":11,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.84299999999999997}]}]},{"osd":5,"up_from":18,"seq":77309411349,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68260,"kb_used_data":19088,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94303580,"statfs":{"total":96636764160,"available":96566865920,"internally_reserved":0,"allocated":19546112,"data_stored":20416939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.46000000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.38}]},{"osd":2,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.436}]},{"osd":3,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.30199999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.34000000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.503}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63800000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.73299999999999998}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.624}]}]},{"osd":9,"up_from":17,"seq":73014444053,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69128,"kb_used_data":19956,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94302712,"statfs":{"total":96636764160,"available":96565977088,"internally_reserved":0,"allocated":20434944,"data_stored":20539800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.47499999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.46200000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.40400000000000003}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.38},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.44400000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.627},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.57499999999999996}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.67700000000000005}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.435}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.53900000000000003}]},{"osd":11,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.58999999999999997}]}]},{"osd":8,"up_from":17,"seq":73014444053,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63980,"kb_used_data":14808,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94307860,"statfs":{"total":96636764160,"available":96571248640,"internally_reserved":0,"allocated":15163392,"data_stored":14997912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.59299999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.79700000000000004}]},{"osd":5,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.69699999999999995}]},{"osd":6,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68200000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.74099999999999999}]},{"osd":9,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.77000000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.81100000000000005}]},{"osd":11,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.75800000000000001}]}]},{"osd":7,"up_from":17,"seq":73014444053,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66072,"kb_used_data":16904,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94305768,"statfs":{"total":96636764160,"available":96569106432,"internally_reserved":0,"allocated":17309696,"data_stored":17337272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":50330060},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5620000000000003,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.58299999999999996}]},{"osd":3,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.65800000000000003}]},{"osd":4,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52000000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.313},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.40600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.442}]},{"osd":9,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57199999999999995}]},{"osd":10,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.50700000000000001}]},{"osd":11,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.49199999999999999}]}]},{"osd":4,"up_from":17,"seq":73014444052,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66264,"kb_used_data":17092,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94305576,"statfs":{"total":96636764160,"available":96568909824,"internally_reserved":0,"allocated":17502208,"data_stored":17336728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64200000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.65100000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.435},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.38300000000000001}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.504}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.58199999999999996}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.621},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.59699999999999998}]},{"osd":11,"last update":"Mon Sep 16 14:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.68200000000000005}]}]},{"osd":3,"up_from":17,"seq":73014444054,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71548,"kb_used_data":22376,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94300292,"statfs":{"total":96636764160,"available":96563499008,"internally_reserved":0,"allocated":22913024,"data_stored":23787928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.253},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.32700000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.28399999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.34999999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.46100000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.436},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.51400000000000001}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58199999999999996}]},{"osd":9,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.55200000000000005}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.63800000000000001}]},{"osd":11,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.62},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.60699999999999998}]}]},{"osd":2,"up_from":17,"seq":73014444053,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47048,"kb_used_data":15932,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94324792,"statfs":{"total":96636764160,"available":96588587008,"internally_reserved":0,"allocated":16314368,"data_stored":18921961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.28499999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.38200000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.32300000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46600000000000003}]},{"osd":8,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.51700000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.69799999999999995}]},{"osd":10,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.68600000000000005}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16232448,"data_stored":18636800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23916544,"data_stored":27471872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15093760,"data_stored":18157568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22286336,"data_stored":24829952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17461248,"data_stored":18247680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20078592,"data_stored":22863872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":14835712,"data_stored":16281600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16736256,"data_stored":17719296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":15122432,"data_stored":15908864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":21561344,"data_stored":23965696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":18751488,"data_stored":21815296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":22687744,"data_stored":26968064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:49:53.550 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T14:49:53.550 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T14:49:53.550 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-09-16T14:49:53.551 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:49:53.553 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:53.552+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:53.562 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:53.562 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:53.572 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T14:49:53.655 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:49:53.656 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:49:53.657 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:53.653+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:53.667 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-16T14:49:52.862669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110535219,"num_objects":56,"num_object_clones":11,"num_object_copies":424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":366,"num_read_kb":86927,"num_write":739,"num_write_kb":93628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":49225728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2124,"ondisk_log_size":2124,"up":226,"acting":226,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":226,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":801552,"kb_used_data":225616,"kb_used_omap":19,"kb_used_meta":575724,"kb_avail":1131660528,"statfs":{"total":1159641169920,"available":1158820380672,"internally_reserved":0,"allocated":231030784,"data_stored":245224270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19502,"internal_metadata":589542354},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18292736,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":-11,"num_read_kb":4382,"num_write":-50,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-19,"num_bytes_recovered":-6012928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.852439"},"pg_stats":[{"pgid":"3.19","version":"105'90","reported_seq":129,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T14:49:51.915643+0000","last_change":"2024-09-16T14:49:49.018972+0000","last_active":"2024-09-16T14:49:51.915643+0000","last_peered":"2024-09-16T14:49:51.915643+0000","last_clean":"2024-09-16T14:49:51.915643+0000","last_became_active":"2024-09-16T14:49:39.892805+0000","last_became_peered":"2024-09-16T14:49:39.892805+0000","last_unstale":"2024-09-16T14:49:51.915643+0000","last_undegraded":"2024-09-16T14:49:51.915643+0000","last_fullsized":"2024-09-16T14:49:51.915643+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"95'81","last_scrub_stamp":"2024-09-16T14:49:42.521147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.521147+0000","objects_scrubbed":3,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:10.802992+0000","scrub_duration":0.003238852,"objects_trimmed":0,"snaptrim_duration":0.0042894789999999997,"stat_sum":{"num_bytes":7438336,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":2320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,10,5,7,11,2],"acting":[1,3,4,10,5,7,11,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.18","version":"102'122","reported_seq":171,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.716793+0000","last_change":"2024-09-16T14:49:50.716793+0000","last_active":"2024-09-16T14:49:50.716793+0000","last_peered":"2024-09-16T14:49:50.716793+0000","last_clean":"2024-09-16T14:49:50.716793+0000","last_became_active":"2024-09-16T14:49:39.893130+0000","last_became_peered":"2024-09-16T14:49:39.893130+0000","last_unstale":"2024-09-16T14:49:50.716793+0000","last_undegraded":"2024-09-16T14:49:50.716793+0000","last_fullsized":"2024-09-16T14:49:50.716793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"102'122","last_scrub_stamp":"2024-09-16T14:49:50.716587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:50.716587+0000","objects_scrubbed":2,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:17.481776+0000","scrub_duration":0.0042360469999999997,"objects_trimmed":1,"snaptrim_duration":0.021640853000000002,"stat_sum":{"num_bytes":6111232,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,1,2,8,7,4,5],"acting":[3,9,1,2,8,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.17","version":"104'172","reported_seq":224,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.926382+0000","last_change":"2024-09-16T14:49:49.034483+0000","last_active":"2024-09-16T14:49:50.926382+0000","last_peered":"2024-09-16T14:49:50.926382+0000","last_clean":"2024-09-16T14:49:50.926382+0000","last_became_active":"2024-09-16T14:49:42.869291+0000","last_became_peered":"2024-09-16T14:49:42.869291+0000","last_unstale":"2024-09-16T14:49:50.926382+0000","last_undegraded":"2024-09-16T14:49:50.926382+0000","last_fullsized":"2024-09-16T14:49:50.926382+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015625354000000001,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":18847,"num_write":90,"num_write_kb":17184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":20987904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,9,3,11,6,2,1],"acting":[0,5,9,3,11,6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.16","version":"83'46","reported_seq":52,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.877236+0000","last_change":"2024-09-16T14:49:49.020712+0000","last_active":"2024-09-16T14:49:49.877236+0000","last_peered":"2024-09-16T14:49:49.877236+0000","last_clean":"2024-09-16T14:49:49.877236+0000","last_became_active":"2024-09-16T14:49:39.885128+0000","last_became_peered":"2024-09-16T14:49:39.885128+0000","last_unstale":"2024-09-16T14:49:49.877236+0000","last_undegraded":"2024-09-16T14:49:49.877236+0000","last_fullsized":"2024-09-16T14:49:49.877236+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078294200000000001,"stat_sum":{"num_bytes":1900544,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":436,"num_write":31,"num_write_kb":2696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3,8,2,10,6],"acting":[5,7,1,3,8,2,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.15","version":"83'61","reported_seq":64,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.976286+0000","last_change":"2024-09-16T14:49:49.016770+0000","last_active":"2024-09-16T14:49:49.976286+0000","last_peered":"2024-09-16T14:49:49.976286+0000","last_clean":"2024-09-16T14:49:49.976286+0000","last_became_active":"2024-09-16T14:49:39.883704+0000","last_became_peered":"2024-09-16T14:49:39.883704+0000","last_unstale":"2024-09-16T14:49:49.976286+0000","last_undegraded":"2024-09-16T14:49:49.976286+0000","last_fullsized":"2024-09-16T14:49:49.976286+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"83'61","last_scrub_stamp":"2024-09-16T14:49:42.159081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.159081+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:49.030695+0000","scrub_duration":0.0019877559999999998,"objects_trimmed":0,"snaptrim_duration":0.0015822239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,4,2,6,0,9],"acting":[8,11,7,4,2,6,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.14","version":"95'82","reported_seq":128,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.859102+0000","last_change":"2024-09-16T14:49:49.022706+0000","last_active":"2024-09-16T14:49:49.859102+0000","last_peered":"2024-09-16T14:49:49.859102+0000","last_clean":"2024-09-16T14:49:49.859102+0000","last_became_active":"2024-09-16T14:49:39.886232+0000","last_became_peered":"2024-09-16T14:49:39.886232+0000","last_unstale":"2024-09-16T14:49:49.859102+0000","last_undegraded":"2024-09-16T14:49:49.859102+0000","last_fullsized":"2024-09-16T14:49:49.859102+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"95'82","last_scrub_stamp":"2024-09-16T14:49:42.377752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.377752+0000","objects_scrubbed":2,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:46.676662+0000","scrub_duration":0.002944898,"objects_trimmed":0,"snaptrim_duration":0.0059355980000000003,"stat_sum":{"num_bytes":2473984,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2116,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,3,11,7,6,2,5],"acting":[4,8,3,11,7,6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T14:49:45.874490+0000","last_change":"2024-09-16T14:49:12.961994+0000","last_active":"2024-09-16T14:49:45.874490+0000","last_peered":"2024-09-16T14:49:45.874490+0000","last_clean":"2024-09-16T14:49:45.874490+0000","last_became_active":"2024-09-16T14:49:12.961697+0000","last_became_peered":"2024-09-16T14:49:12.961697+0000","last_unstale":"2024-09-16T14:49:45.874490+0000","last_undegraded":"2024-09-16T14:49:45.874490+0000","last_fullsized":"2024-09-16T14:49:45.874490+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"104'54","reported_seq":337,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.884052+0000","last_change":"2024-09-16T14:49:49.018845+0000","last_active":"2024-09-16T14:49:50.884052+0000","last_peered":"2024-09-16T14:49:50.884052+0000","last_clean":"2024-09-16T14:49:50.884052+0000","last_became_active":"2024-09-16T14:49:37.968269+0000","last_became_peered":"2024-09-16T14:49:37.968269+0000","last_unstale":"2024-09-16T14:49:50.884052+0000","last_undegraded":"2024-09-16T14:49:50.884052+0000","last_fullsized":"2024-09-16T14:49:50.884052+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'48","last_scrub_stamp":"2024-09-16T14:49:43.049407+0000","last_deep_scrub":"91'48","last_deep_scrub_stamp":"2024-09-16T14:49:43.049407+0000","last_clean_scrub_stamp":"2024-09-16T14:49:43.049407+0000","objects_scrubbed":3,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:55.469156+0000","scrub_duration":0.004290266,"objects_trimmed":0,"snaptrim_duration":0.0010204050000000001,"stat_sum":{"num_bytes":2981888,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,5],"acting":[0,1,4,7,8,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.859648+0000","last_change":"2024-09-16T14:48:26.161840+0000","last_active":"2024-09-16T14:49:50.859648+0000","last_peered":"2024-09-16T14:49:50.859648+0000","last_clean":"2024-09-16T14:49:50.859648+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:49:50.859648+0000","last_undegraded":"2024-09-16T14:49:50.859648+0000","last_fullsized":"2024-09-16T14:49:50.859648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"99'155","reported_seq":434,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883243+0000","last_change":"2024-09-16T14:49:49.012411+0000","last_active":"2024-09-16T14:49:49.883243+0000","last_peered":"2024-09-16T14:49:49.883243+0000","last_clean":"2024-09-16T14:49:49.883243+0000","last_became_active":"2024-09-16T14:49:37.961523+0000","last_became_peered":"2024-09-16T14:49:37.961523+0000","last_unstale":"2024-09-16T14:49:49.883243+0000","last_undegraded":"2024-09-16T14:49:49.883243+0000","last_fullsized":"2024-09-16T14:49:49.883243+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'150","last_scrub_stamp":"2024-09-16T14:49:40.650365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.650365+0000","objects_scrubbed":2,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:01.008992+0000","scrub_duration":0.0038572889999999999,"objects_trimmed":0,"snaptrim_duration":0.00085950499999999999,"stat_sum":{"num_bytes":4767744,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":108,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.976066+0000","last_change":"2024-09-16T14:49:43.127050+0000","last_active":"2024-09-16T14:49:49.976066+0000","last_peered":"2024-09-16T14:49:49.976066+0000","last_clean":"2024-09-16T14:49:49.976066+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:49:49.976066+0000","last_undegraded":"2024-09-16T14:49:49.976066+0000","last_fullsized":"2024-09-16T14:49:49.976066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_clean_scrub_stamp":"2024-09-16T14:49:43.126944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:52.930143+0000","scrub_duration":0.0012658890000000001,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"104'80","reported_seq":394,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.941580+0000","last_change":"2024-09-16T14:49:49.016947+0000","last_active":"2024-09-16T14:49:50.941580+0000","last_peered":"2024-09-16T14:49:50.941580+0000","last_clean":"2024-09-16T14:49:50.941580+0000","last_became_active":"2024-09-16T14:49:37.959809+0000","last_became_peered":"2024-09-16T14:49:37.959809+0000","last_unstale":"2024-09-16T14:49:50.941580+0000","last_undegraded":"2024-09-16T14:49:50.941580+0000","last_fullsized":"2024-09-16T14:49:50.941580+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'74","last_scrub_stamp":"2024-09-16T14:49:40.525004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.525004+0000","objects_scrubbed":2,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:04.420225+0000","scrub_duration":0.0030813329999999999,"objects_trimmed":0,"snaptrim_duration":0.0023863780000000002,"stat_sum":{"num_bytes":3194880,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":1538,"num_write":5,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,9,5,10,2,6,0],"acting":[1,11,9,5,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.859573+0000","last_change":"2024-09-16T14:48:26.161822+0000","last_active":"2024-09-16T14:49:50.859573+0000","last_peered":"2024-09-16T14:49:50.859573+0000","last_clean":"2024-09-16T14:49:50.859573+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:49:50.859573+0000","last_undegraded":"2024-09-16T14:49:50.859573+0000","last_fullsized":"2024-09-16T14:49:50.859573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"103'80","reported_seq":464,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.450013+0000","last_change":"2024-09-16T14:49:50.450013+0000","last_active":"2024-09-16T14:49:50.450013+0000","last_peered":"2024-09-16T14:49:50.450013+0000","last_clean":"2024-09-16T14:49:50.450013+0000","last_became_active":"2024-09-16T14:49:37.958130+0000","last_became_peered":"2024-09-16T14:49:37.958130+0000","last_unstale":"2024-09-16T14:49:50.450013+0000","last_undegraded":"2024-09-16T14:49:50.450013+0000","last_fullsized":"2024-09-16T14:49:50.450013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'80","last_scrub_stamp":"2024-09-16T14:49:50.449861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:50.449861+0000","objects_scrubbed":3,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:17.237609+0000","scrub_duration":0.0035009910000000002,"objects_trimmed":0,"snaptrim_duration":0.00080948000000000001,"stat_sum":{"num_bytes":8781824,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,1,11,10],"acting":[5,3,2,6,7,1,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.3","version":"83'37","reported_seq":344,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.859337+0000","last_change":"2024-09-16T14:49:49.023007+0000","last_active":"2024-09-16T14:49:49.859337+0000","last_peered":"2024-09-16T14:49:49.859337+0000","last_clean":"2024-09-16T14:49:49.859337+0000","last_became_active":"2024-09-16T14:49:37.960053+0000","last_became_peered":"2024-09-16T14:49:37.960053+0000","last_unstale":"2024-09-16T14:49:49.859337+0000","last_undegraded":"2024-09-16T14:49:49.859337+0000","last_fullsized":"2024-09-16T14:49:49.859337+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'37","last_scrub_stamp":"2024-09-16T14:49:38.437020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:38.437020+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:44.378876+0000","scrub_duration":0.0029472029999999998,"objects_trimmed":0,"snaptrim_duration":0.0055016789999999998,"stat_sum":{"num_bytes":81920,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":110,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.876420+0000","last_change":"2024-09-16T14:49:28.351058+0000","last_active":"2024-09-16T14:49:49.876420+0000","last_peered":"2024-09-16T14:49:49.876420+0000","last_clean":"2024-09-16T14:49:49.876420+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:49:49.876420+0000","last_undegraded":"2024-09-16T14:49:49.876420+0000","last_fullsized":"2024-09-16T14:49:49.876420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-16T14:49:28.350938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:28.350938+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:42.292353+0000","scrub_duration":0.0018950309999999999,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"91'47","reported_seq":345,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T14:49:51.917258+0000","last_change":"2024-09-16T14:49:49.021090+0000","last_active":"2024-09-16T14:49:51.917258+0000","last_peered":"2024-09-16T14:49:51.917258+0000","last_clean":"2024-09-16T14:49:51.917258+0000","last_became_active":"2024-09-16T14:49:37.961051+0000","last_became_peered":"2024-09-16T14:49:37.961051+0000","last_unstale":"2024-09-16T14:49:51.917258+0000","last_undegraded":"2024-09-16T14:49:51.917258+0000","last_fullsized":"2024-09-16T14:49:51.917258+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'47","last_scrub_stamp":"2024-09-16T14:49:41.126704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:41.126704+0000","objects_scrubbed":2,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:44.106417+0000","scrub_duration":0.0027967320000000001,"objects_trimmed":0,"snaptrim_duration":0.0026831110000000002,"stat_sum":{"num_bytes":5767168,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":9169,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883009+0000","last_change":"2024-09-16T14:49:27.155463+0000","last_active":"2024-09-16T14:49:49.883009+0000","last_peered":"2024-09-16T14:49:49.883009+0000","last_clean":"2024-09-16T14:49:49.883009+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:49:49.883009+0000","last_undegraded":"2024-09-16T14:49:49.883009+0000","last_fullsized":"2024-09-16T14:49:49.883009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:27.155307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:27.155307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:32.986356+0000","scrub_duration":0.001601157,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":279,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.868799+0000","last_change":"2024-09-16T14:49:49.019384+0000","last_active":"2024-09-16T14:49:50.868799+0000","last_peered":"2024-09-16T14:49:50.868799+0000","last_clean":"2024-09-16T14:49:50.868799+0000","last_became_active":"2024-09-16T14:49:37.959116+0000","last_became_peered":"2024-09-16T14:49:37.959116+0000","last_unstale":"2024-09-16T14:49:50.868799+0000","last_undegraded":"2024-09-16T14:49:50.868799+0000","last_fullsized":"2024-09-16T14:49:50.868799+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:42.094901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.094901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:00.419660+0000","scrub_duration":0.0018558680000000001,"objects_trimmed":0,"snaptrim_duration":0.0010354800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882809+0000","last_change":"2024-09-16T14:48:26.568303+0000","last_active":"2024-09-16T14:49:49.882809+0000","last_peered":"2024-09-16T14:49:49.882809+0000","last_clean":"2024-09-16T14:49:49.882809+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:49:49.882809+0000","last_undegraded":"2024-09-16T14:49:49.882809+0000","last_fullsized":"2024-09-16T14:49:49.882809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"98'110","reported_seq":422,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882601+0000","last_change":"2024-09-16T14:49:49.013432+0000","last_active":"2024-09-16T14:49:49.882601+0000","last_peered":"2024-09-16T14:49:49.882601+0000","last_clean":"2024-09-16T14:49:49.882601+0000","last_became_active":"2024-09-16T14:49:37.958372+0000","last_became_peered":"2024-09-16T14:49:37.958372+0000","last_unstale":"2024-09-16T14:49:49.882601+0000","last_undegraded":"2024-09-16T14:49:49.882601+0000","last_fullsized":"2024-09-16T14:49:49.882601+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'102","last_scrub_stamp":"2024-09-16T14:49:38.639259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:38.639259+0000","objects_scrubbed":1,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:50.153167+0000","scrub_duration":0.0030648189999999999,"objects_trimmed":0,"snaptrim_duration":0.001860719,"stat_sum":{"num_bytes":4112384,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":109,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.876474+0000","last_change":"2024-09-16T14:49:40.527258+0000","last_active":"2024-09-16T14:49:49.876474+0000","last_peered":"2024-09-16T14:49:49.876474+0000","last_clean":"2024-09-16T14:49:49.876474+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:49:49.876474+0000","last_undegraded":"2024-09-16T14:49:49.876474+0000","last_fullsized":"2024-09-16T14:49:49.876474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2024-09-16T14:49:40.527164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.527164+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:46.029615+0000","scrub_duration":0.0012857120000000001,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":267,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883100+0000","last_change":"2024-09-16T14:48:22.359960+0000","last_active":"2024-09-16T14:49:49.883100+0000","last_peered":"2024-09-16T14:49:49.883100+0000","last_clean":"2024-09-16T14:49:49.883100+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:49:49.883100+0000","last_undegraded":"2024-09-16T14:49:49.883100+0000","last_fullsized":"2024-09-16T14:49:49.883100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:48:21.118926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"102'105","reported_seq":382,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.911099+0000","last_change":"2024-09-16T14:49:49.034250+0000","last_active":"2024-09-16T14:49:50.911099+0000","last_peered":"2024-09-16T14:49:50.911099+0000","last_clean":"2024-09-16T14:49:50.911099+0000","last_became_active":"2024-09-16T14:49:37.960827+0000","last_became_peered":"2024-09-16T14:49:37.960827+0000","last_unstale":"2024-09-16T14:49:50.911099+0000","last_undegraded":"2024-09-16T14:49:50.911099+0000","last_fullsized":"2024-09-16T14:49:50.911099+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'97","last_scrub_stamp":"2024-09-16T14:49:40.167318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.167318+0000","objects_scrubbed":2,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:55.504556+0000","scrub_duration":0.002425441,"objects_trimmed":0,"snaptrim_duration":0.016083350999999999,"stat_sum":{"num_bytes":4407296,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2,4,3,5,10,8],"acting":[11,1,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.9","version":"102'83","reported_seq":366,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.976342+0000","last_change":"2024-09-16T14:49:49.031507+0000","last_active":"2024-09-16T14:49:49.976342+0000","last_peered":"2024-09-16T14:49:49.976342+0000","last_clean":"2024-09-16T14:49:49.976342+0000","last_became_active":"2024-09-16T14:49:37.960644+0000","last_became_peered":"2024-09-16T14:49:37.960644+0000","last_unstale":"2024-09-16T14:49:49.976342+0000","last_undegraded":"2024-09-16T14:49:49.976342+0000","last_fullsized":"2024-09-16T14:49:49.976342+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'77","last_scrub_stamp":"2024-09-16T14:49:40.131033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.131033+0000","objects_scrubbed":3,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:51.352479+0000","scrub_duration":0.0026976370000000001,"objects_trimmed":1,"snaptrim_duration":0.016770514,"stat_sum":{"num_bytes":5701632,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.a","version":"99'68","reported_seq":194,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T14:49:45.952464+0000","last_change":"2024-09-16T14:49:44.889238+0000","last_active":"2024-09-16T14:49:45.952464+0000","last_peered":"2024-09-16T14:49:45.952464+0000","last_clean":"2024-09-16T14:49:45.952464+0000","last_became_active":"2024-09-16T14:49:19.868841+0000","last_became_peered":"2024-09-16T14:49:19.868841+0000","last_unstale":"2024-09-16T14:49:45.952464+0000","last_undegraded":"2024-09-16T14:49:45.952464+0000","last_fullsized":"2024-09-16T14:49:45.952464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01518982,"stat_sum":{"num_bytes":9125888,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":1063,"num_write":48,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":9224192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,10,8,9,3,7],"acting":[6,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"2"},{"start":"2e","length":"2"},{"start":"31","length":"1"},{"start":"37","length":"5"},{"start":"40","length":"1"},{"start":"43","length":"1"}]},{"pgid":"3.b","version":"90'77","reported_seq":343,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882753+0000","last_change":"2024-09-16T14:49:49.011747+0000","last_active":"2024-09-16T14:49:49.882753+0000","last_peered":"2024-09-16T14:49:49.882753+0000","last_clean":"2024-09-16T14:49:49.882753+0000","last_became_active":"2024-09-16T14:49:12.972366+0000","last_became_peered":"2024-09-16T14:49:12.972366+0000","last_unstale":"2024-09-16T14:49:49.882753+0000","last_undegraded":"2024-09-16T14:49:49.882753+0000","last_fullsized":"2024-09-16T14:49:49.882753+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:03.456665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043729700000000001,"stat_sum":{"num_bytes":1130496,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":7388,"num_write":44,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1785856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,1,6],"acting":[3,0,4,8,11,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.c","version":"102'89","reported_seq":377,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.876733+0000","last_change":"2024-09-16T14:49:49.029918+0000","last_active":"2024-09-16T14:49:49.876733+0000","last_peered":"2024-09-16T14:49:49.876733+0000","last_clean":"2024-09-16T14:49:49.876733+0000","last_became_active":"2024-09-16T14:49:12.979797+0000","last_became_peered":"2024-09-16T14:49:12.979797+0000","last_unstale":"2024-09-16T14:49:49.876733+0000","last_undegraded":"2024-09-16T14:49:49.876733+0000","last_fullsized":"2024-09-16T14:49:49.876733+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.201481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098966060000000005,"stat_sum":{"num_bytes":7389184,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":12010,"num_write":33,"num_write_kb":9920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3440640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.d","version":"98'82","reported_seq":387,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882964+0000","last_change":"2024-09-16T14:49:49.034992+0000","last_active":"2024-09-16T14:49:49.882964+0000","last_peered":"2024-09-16T14:49:49.882964+0000","last_clean":"2024-09-16T14:49:49.882964+0000","last_became_active":"2024-09-16T14:49:12.979812+0000","last_became_peered":"2024-09-16T14:49:12.979812+0000","last_unstale":"2024-09-16T14:49:49.882964+0000","last_undegraded":"2024-09-16T14:49:49.882964+0000","last_fullsized":"2024-09-16T14:49:49.882964+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010903285,"stat_sum":{"num_bytes":5210112,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":10527,"num_write":78,"num_write_kb":7952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4489216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.e","version":"102'86","reported_seq":334,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.911320+0000","last_change":"2024-09-16T14:49:49.035327+0000","last_active":"2024-09-16T14:49:50.911320+0000","last_peered":"2024-09-16T14:49:50.911320+0000","last_clean":"2024-09-16T14:49:50.911320+0000","last_became_active":"2024-09-16T14:48:35.075626+0000","last_became_peered":"2024-09-16T14:48:35.075626+0000","last_unstale":"2024-09-16T14:49:50.911320+0000","last_undegraded":"2024-09-16T14:49:50.911320+0000","last_fullsized":"2024-09-16T14:49:50.911320+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.908348+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.016608049,"stat_sum":{"num_bytes":13107200,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":4441,"num_write":75,"num_write_kb":13360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.f","version":"104'64","reported_seq":307,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.965499+0000","last_change":"2024-09-16T14:49:49.034518+0000","last_active":"2024-09-16T14:49:50.965499+0000","last_peered":"2024-09-16T14:49:50.965499+0000","last_clean":"2024-09-16T14:49:50.965499+0000","last_became_active":"2024-09-16T14:48:35.075111+0000","last_became_peered":"2024-09-16T14:48:35.075111+0000","last_unstale":"2024-09-16T14:49:50.965499+0000","last_undegraded":"2024-09-16T14:49:50.965499+0000","last_fullsized":"2024-09-16T14:49:50.965499+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:58.979883+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01172438,"stat_sum":{"num_bytes":3915776,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":10587,"num_write":62,"num_write_kb":7728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.10","version":"82'43","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T14:49:45.873772+0000","last_change":"2024-09-16T14:49:44.876682+0000","last_active":"2024-09-16T14:49:45.873772+0000","last_peered":"2024-09-16T14:49:45.873772+0000","last_clean":"2024-09-16T14:49:45.873772+0000","last_became_active":"2024-09-16T14:49:39.890294+0000","last_became_peered":"2024-09-16T14:49:39.890294+0000","last_unstale":"2024-09-16T14:49:45.873772+0000","last_undegraded":"2024-09-16T14:49:45.873772+0000","last_fullsized":"2024-09-16T14:49:45.873772+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"82'43","last_scrub_stamp":"2024-09-16T14:49:42.551604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.551604+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:59.224582+0000","scrub_duration":0.0021534729999999999,"objects_trimmed":0,"snaptrim_duration":0.0032657860000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,5,3,0,2,1,7],"acting":[6,9,5,3,0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"2"},{"start":"2e","length":"2"},{"start":"31","length":"1"},{"start":"37","length":"5"},{"start":"40","length":"1"},{"start":"43","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":93,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.873262+0000","last_change":"2024-09-16T14:49:49.015897+0000","last_active":"2024-09-16T14:49:50.873262+0000","last_peered":"2024-09-16T14:49:50.873262+0000","last_clean":"2024-09-16T14:49:50.873262+0000","last_became_active":"2024-09-16T14:49:39.875227+0000","last_became_peered":"2024-09-16T14:49:39.875227+0000","last_unstale":"2024-09-16T14:49:50.873262+0000","last_undegraded":"2024-09-16T14:49:50.873262+0000","last_fullsized":"2024-09-16T14:49:50.873262+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:40.245626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.245626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:41.418675+0000","scrub_duration":0.0016751209999999999,"objects_trimmed":0,"snaptrim_duration":0.00079420900000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6,0,3,9,1,11],"acting":[10,4,6,0,3,9,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.12","version":"96'106","reported_seq":165,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.868833+0000","last_change":"2024-09-16T14:49:49.018091+0000","last_active":"2024-09-16T14:49:50.868833+0000","last_peered":"2024-09-16T14:49:50.868833+0000","last_clean":"2024-09-16T14:49:50.868833+0000","last_became_active":"2024-09-16T14:49:42.870631+0000","last_became_peered":"2024-09-16T14:49:42.870631+0000","last_unstale":"2024-09-16T14:49:50.868833+0000","last_undegraded":"2024-09-16T14:49:50.868833+0000","last_fullsized":"2024-09-16T14:49:50.868833+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060451400000000001,"stat_sum":{"num_bytes":3088384,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":3413,"num_write":30,"num_write_kb":4640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":7602176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1,2,5,6,11],"acting":[0,7,3,1,2,5,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.13","version":"102'45","reported_seq":95,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883456+0000","last_change":"2024-09-16T14:49:49.034362+0000","last_active":"2024-09-16T14:49:49.883456+0000","last_peered":"2024-09-16T14:49:49.883456+0000","last_clean":"2024-09-16T14:49:49.883456+0000","last_became_active":"2024-09-16T14:49:39.885593+0000","last_became_peered":"2024-09-16T14:49:39.885593+0000","last_unstale":"2024-09-16T14:49:49.883456+0000","last_undegraded":"2024-09-16T14:49:49.883456+0000","last_fullsized":"2024-09-16T14:49:49.883456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.011674103,"stat_sum":{"num_bytes":868352,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":5278,"num_write":19,"num_write_kb":1112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1695744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2,6,1,9,8],"acting":[7,5,4,2,6,1,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"46","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":109944832,"num_objects":52,"num_object_clones":11,"num_object_copies":416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":240,"num_read_kb":86818,"num_write":556,"num_write_kb":90688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":49225728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224763904,"data_stored":252866560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1984,"ondisk_log_size":1984,"up":208,"acting":208,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411348,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77140,"kb_used_data":23880,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94294700,"statfs":{"total":96636764160,"available":96557772800,"internally_reserved":0,"allocated":24453120,"data_stored":28269658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52700000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.38300000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.46700000000000003}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.437},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.311}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.57099999999999995}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.48699999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.503},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51800000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59799999999999998}]},{"osd":9,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.622}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.64800000000000002}]}]},{"osd":10,"up_from":18,"seq":77309411350,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68084,"kb_used_data":18920,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94303756,"statfs":{"total":96636764160,"available":96567046144,"internally_reserved":0,"allocated":19374080,"data_stored":20768301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.505},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.48799999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.60399999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57299999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.46100000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.432}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72299999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.39700000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.37},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.38}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.40799999999999997}]}]},{"osd":6,"up_from":41,"seq":176093659150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46620,"kb_used_data":15632,"kb_used_omap":1,"kb_used_meta":30974,"kb_avail":94325220,"statfs":{"total":96636764160,"available":96589025280,"internally_reserved":0,"allocated":16007168,"data_stored":17012453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1965,"internal_metadata":31717459},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":1,"up_from":18,"seq":77309411350,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91184,"kb_used_data":23972,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94280656,"statfs":{"total":96636764160,"available":96543391744,"internally_reserved":0,"allocated":24547328,"data_stored":26425852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.312}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.0590000000000002,"5min":4.0590000000000002,"15min":4.0590000000000002},"last":0.42999999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.503},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":3.9620000000000002,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.55600000000000005}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.79300000000000004}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.67500000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.68700000000000006}]},{"osd":11,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":4.6589999999999998,"5min":4.6589999999999998,"15min":4.6589999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.66500000000000004}]}]},{"osd":0,"up_from":18,"seq":77309411348,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66224,"kb_used_data":17056,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94305616,"statfs":{"total":96636764160,"available":96568950784,"internally_reserved":0,"allocated":17465344,"data_stored":19409466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.41399999999999998}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.5}]},{"osd":8,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.629},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.77800000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57299999999999995}]},{"osd":10,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.745}]},{"osd":11,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.84299999999999997}]}]},{"osd":5,"up_from":18,"seq":77309411349,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68260,"kb_used_data":19088,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94303580,"statfs":{"total":96636764160,"available":96566865920,"internally_reserved":0,"allocated":19546112,"data_stored":20416939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.46000000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.38}]},{"osd":2,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.436}]},{"osd":3,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.30199999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.34000000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.503}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63800000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.73299999999999998}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.624}]}]},{"osd":9,"up_from":17,"seq":73014444053,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69128,"kb_used_data":19956,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94302712,"statfs":{"total":96636764160,"available":96565977088,"internally_reserved":0,"allocated":20434944,"data_stored":20539800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.47499999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.46200000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.40400000000000003}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.38},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.44400000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.627},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.57499999999999996}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.67700000000000005}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.435}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.53900000000000003}]},{"osd":11,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.58999999999999997}]}]},{"osd":8,"up_from":17,"seq":73014444053,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63980,"kb_used_data":14808,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94307860,"statfs":{"total":96636764160,"available":96571248640,"internally_reserved":0,"allocated":15163392,"data_stored":14997912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.59299999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.79700000000000004}]},{"osd":5,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.69699999999999995}]},{"osd":6,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68200000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.74099999999999999}]},{"osd":9,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.77000000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.81100000000000005}]},{"osd":11,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.75800000000000001}]}]},{"osd":7,"up_from":17,"seq":73014444053,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66072,"kb_used_data":16904,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94305768,"statfs":{"total":96636764160,"available":96569106432,"internally_reserved":0,"allocated":17309696,"data_stored":17337272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":50330060},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5620000000000003,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.58299999999999996}]},{"osd":3,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.65800000000000003}]},{"osd":4,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52000000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.313},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.40600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.442}]},{"osd":9,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57199999999999995}]},{"osd":10,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.50700000000000001}]},{"osd":11,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.49199999999999999}]}]},{"osd":4,"up_from":17,"seq":73014444052,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66264,"kb_used_data":17092,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94305576,"statfs":{"total":96636764160,"available":96568909824,"internally_reserved":0,"allocated":17502208,"data_stored":17336728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64200000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.65100000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.435},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.38300000000000001}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.504}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.58199999999999996}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.621},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.59699999999999998}]},{"osd":11,"last update":"Mon Sep 16 14:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.68200000000000005}]}]},{"osd":3,"up_from":17,"seq":73014444054,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71548,"kb_used_data":22376,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94300292,"statfs":{"total":96636764160,"available":96563499008,"internally_reserved":0,"allocated":22913024,"data_stored":23787928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.253},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.32700000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.28399999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.34999999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.46100000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.436},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.51400000000000001}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58199999999999996}]},{"osd":9,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.55200000000000005}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.63800000000000001}]},{"osd":11,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.62},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.60699999999999998}]}]},{"osd":2,"up_from":17,"seq":73014444053,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47048,"kb_used_data":15932,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94324792,"statfs":{"total":96636764160,"available":96588587008,"internally_reserved":0,"allocated":16314368,"data_stored":18921961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.28499999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.38200000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.32300000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46600000000000003}]},{"osd":8,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.51700000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.69799999999999995}]},{"osd":10,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.68600000000000005}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16232448,"data_stored":18636800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23916544,"data_stored":27471872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15093760,"data_stored":18157568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22286336,"data_stored":24829952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17461248,"data_stored":18247680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20078592,"data_stored":22863872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":14835712,"data_stored":16281600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16736256,"data_stored":17719296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":15122432,"data_stored":15908864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":21561344,"data_stored":23965696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":18751488,"data_stored":21815296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":22687744,"data_stored":26968064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:49:53.722 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:53.722 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:53.731 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T14:49:53.775 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:53.774+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:53.838 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:53.839 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:53.845 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:49:53.846 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:49:53.848 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T14:49:53.855 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:53.854+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:53.857 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-16T14:49:52.862669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110535219,"num_objects":56,"num_object_clones":11,"num_object_copies":424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":366,"num_read_kb":86927,"num_write":739,"num_write_kb":93628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":49225728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2124,"ondisk_log_size":2124,"up":226,"acting":226,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":226,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1132462080,"kb_used":801552,"kb_used_data":225616,"kb_used_omap":19,"kb_used_meta":575724,"kb_avail":1131660528,"statfs":{"total":1159641169920,"available":1158820380672,"internally_reserved":0,"allocated":231030784,"data_stored":245224270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19502,"internal_metadata":589542354},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18292736,"num_objects":4,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":-11,"num_read_kb":4382,"num_write":-50,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-19,"num_bytes_recovered":-6012928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.852439"},"pg_stats":[{"pgid":"3.19","version":"105'90","reported_seq":129,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T14:49:51.915643+0000","last_change":"2024-09-16T14:49:49.018972+0000","last_active":"2024-09-16T14:49:51.915643+0000","last_peered":"2024-09-16T14:49:51.915643+0000","last_clean":"2024-09-16T14:49:51.915643+0000","last_became_active":"2024-09-16T14:49:39.892805+0000","last_became_peered":"2024-09-16T14:49:39.892805+0000","last_unstale":"2024-09-16T14:49:51.915643+0000","last_undegraded":"2024-09-16T14:49:51.915643+0000","last_fullsized":"2024-09-16T14:49:51.915643+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"95'81","last_scrub_stamp":"2024-09-16T14:49:42.521147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.521147+0000","objects_scrubbed":3,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:10.802992+0000","scrub_duration":0.003238852,"objects_trimmed":0,"snaptrim_duration":0.0042894789999999997,"stat_sum":{"num_bytes":7438336,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":2320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,10,5,7,11,2],"acting":[1,3,4,10,5,7,11,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.18","version":"102'122","reported_seq":171,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.716793+0000","last_change":"2024-09-16T14:49:50.716793+0000","last_active":"2024-09-16T14:49:50.716793+0000","last_peered":"2024-09-16T14:49:50.716793+0000","last_clean":"2024-09-16T14:49:50.716793+0000","last_became_active":"2024-09-16T14:49:39.893130+0000","last_became_peered":"2024-09-16T14:49:39.893130+0000","last_unstale":"2024-09-16T14:49:50.716793+0000","last_undegraded":"2024-09-16T14:49:50.716793+0000","last_fullsized":"2024-09-16T14:49:50.716793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"102'122","last_scrub_stamp":"2024-09-16T14:49:50.716587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:50.716587+0000","objects_scrubbed":2,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:17.481776+0000","scrub_duration":0.0042360469999999997,"objects_trimmed":1,"snaptrim_duration":0.021640853000000002,"stat_sum":{"num_bytes":6111232,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,1,2,8,7,4,5],"acting":[3,9,1,2,8,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.17","version":"104'172","reported_seq":224,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.926382+0000","last_change":"2024-09-16T14:49:49.034483+0000","last_active":"2024-09-16T14:49:50.926382+0000","last_peered":"2024-09-16T14:49:50.926382+0000","last_clean":"2024-09-16T14:49:50.926382+0000","last_became_active":"2024-09-16T14:49:42.869291+0000","last_became_peered":"2024-09-16T14:49:42.869291+0000","last_unstale":"2024-09-16T14:49:50.926382+0000","last_undegraded":"2024-09-16T14:49:50.926382+0000","last_fullsized":"2024-09-16T14:49:50.926382+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.015625354000000001,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":18847,"num_write":90,"num_write_kb":17184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":20987904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,9,3,11,6,2,1],"acting":[0,5,9,3,11,6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.16","version":"83'46","reported_seq":52,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.877236+0000","last_change":"2024-09-16T14:49:49.020712+0000","last_active":"2024-09-16T14:49:49.877236+0000","last_peered":"2024-09-16T14:49:49.877236+0000","last_clean":"2024-09-16T14:49:49.877236+0000","last_became_active":"2024-09-16T14:49:39.885128+0000","last_became_peered":"2024-09-16T14:49:39.885128+0000","last_unstale":"2024-09-16T14:49:49.877236+0000","last_undegraded":"2024-09-16T14:49:49.877236+0000","last_fullsized":"2024-09-16T14:49:49.877236+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078294200000000001,"stat_sum":{"num_bytes":1900544,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":436,"num_write":31,"num_write_kb":2696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3,8,2,10,6],"acting":[5,7,1,3,8,2,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.15","version":"83'61","reported_seq":64,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.976286+0000","last_change":"2024-09-16T14:49:49.016770+0000","last_active":"2024-09-16T14:49:49.976286+0000","last_peered":"2024-09-16T14:49:49.976286+0000","last_clean":"2024-09-16T14:49:49.976286+0000","last_became_active":"2024-09-16T14:49:39.883704+0000","last_became_peered":"2024-09-16T14:49:39.883704+0000","last_unstale":"2024-09-16T14:49:49.976286+0000","last_undegraded":"2024-09-16T14:49:49.976286+0000","last_fullsized":"2024-09-16T14:49:49.976286+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"83'61","last_scrub_stamp":"2024-09-16T14:49:42.159081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.159081+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:49.030695+0000","scrub_duration":0.0019877559999999998,"objects_trimmed":0,"snaptrim_duration":0.0015822239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,4,2,6,0,9],"acting":[8,11,7,4,2,6,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.14","version":"95'82","reported_seq":128,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.859102+0000","last_change":"2024-09-16T14:49:49.022706+0000","last_active":"2024-09-16T14:49:49.859102+0000","last_peered":"2024-09-16T14:49:49.859102+0000","last_clean":"2024-09-16T14:49:49.859102+0000","last_became_active":"2024-09-16T14:49:39.886232+0000","last_became_peered":"2024-09-16T14:49:39.886232+0000","last_unstale":"2024-09-16T14:49:49.859102+0000","last_undegraded":"2024-09-16T14:49:49.859102+0000","last_fullsized":"2024-09-16T14:49:49.859102+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"95'82","last_scrub_stamp":"2024-09-16T14:49:42.377752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.377752+0000","objects_scrubbed":2,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:46.676662+0000","scrub_duration":0.002944898,"objects_trimmed":0,"snaptrim_duration":0.0059355980000000003,"stat_sum":{"num_bytes":2473984,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2116,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,3,11,7,6,2,5],"acting":[4,8,3,11,7,6,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T14:49:45.874490+0000","last_change":"2024-09-16T14:49:12.961994+0000","last_active":"2024-09-16T14:49:45.874490+0000","last_peered":"2024-09-16T14:49:45.874490+0000","last_clean":"2024-09-16T14:49:45.874490+0000","last_became_active":"2024-09-16T14:49:12.961697+0000","last_became_peered":"2024-09-16T14:49:12.961697+0000","last_unstale":"2024-09-16T14:49:45.874490+0000","last_undegraded":"2024-09-16T14:49:45.874490+0000","last_fullsized":"2024-09-16T14:49:45.874490+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"104'54","reported_seq":337,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.884052+0000","last_change":"2024-09-16T14:49:49.018845+0000","last_active":"2024-09-16T14:49:50.884052+0000","last_peered":"2024-09-16T14:49:50.884052+0000","last_clean":"2024-09-16T14:49:50.884052+0000","last_became_active":"2024-09-16T14:49:37.968269+0000","last_became_peered":"2024-09-16T14:49:37.968269+0000","last_unstale":"2024-09-16T14:49:50.884052+0000","last_undegraded":"2024-09-16T14:49:50.884052+0000","last_fullsized":"2024-09-16T14:49:50.884052+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'48","last_scrub_stamp":"2024-09-16T14:49:43.049407+0000","last_deep_scrub":"91'48","last_deep_scrub_stamp":"2024-09-16T14:49:43.049407+0000","last_clean_scrub_stamp":"2024-09-16T14:49:43.049407+0000","objects_scrubbed":3,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:55.469156+0000","scrub_duration":0.004290266,"objects_trimmed":0,"snaptrim_duration":0.0010204050000000001,"stat_sum":{"num_bytes":2981888,"num_objects":4,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7,8,10,9,5],"acting":[0,1,4,7,8,10,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.859648+0000","last_change":"2024-09-16T14:48:26.161840+0000","last_active":"2024-09-16T14:49:50.859648+0000","last_peered":"2024-09-16T14:49:50.859648+0000","last_clean":"2024-09-16T14:49:50.859648+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:49:50.859648+0000","last_undegraded":"2024-09-16T14:49:50.859648+0000","last_fullsized":"2024-09-16T14:49:50.859648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"99'155","reported_seq":434,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883243+0000","last_change":"2024-09-16T14:49:49.012411+0000","last_active":"2024-09-16T14:49:49.883243+0000","last_peered":"2024-09-16T14:49:49.883243+0000","last_clean":"2024-09-16T14:49:49.883243+0000","last_became_active":"2024-09-16T14:49:37.961523+0000","last_became_peered":"2024-09-16T14:49:37.961523+0000","last_unstale":"2024-09-16T14:49:49.883243+0000","last_undegraded":"2024-09-16T14:49:49.883243+0000","last_fullsized":"2024-09-16T14:49:49.883243+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'150","last_scrub_stamp":"2024-09-16T14:49:40.650365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.650365+0000","objects_scrubbed":2,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:01.008992+0000","scrub_duration":0.0038572889999999999,"objects_trimmed":0,"snaptrim_duration":0.00085950499999999999,"stat_sum":{"num_bytes":4767744,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7,0,10,9,4,1],"acting":[3,11,7,0,10,9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":108,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.976066+0000","last_change":"2024-09-16T14:49:43.127050+0000","last_active":"2024-09-16T14:49:49.976066+0000","last_peered":"2024-09-16T14:49:49.976066+0000","last_clean":"2024-09-16T14:49:49.976066+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:49:49.976066+0000","last_undegraded":"2024-09-16T14:49:49.976066+0000","last_fullsized":"2024-09-16T14:49:49.976066+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_clean_scrub_stamp":"2024-09-16T14:49:43.126944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:52.930143+0000","scrub_duration":0.0012658890000000001,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"104'80","reported_seq":394,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.941580+0000","last_change":"2024-09-16T14:49:49.016947+0000","last_active":"2024-09-16T14:49:50.941580+0000","last_peered":"2024-09-16T14:49:50.941580+0000","last_clean":"2024-09-16T14:49:50.941580+0000","last_became_active":"2024-09-16T14:49:37.959809+0000","last_became_peered":"2024-09-16T14:49:37.959809+0000","last_unstale":"2024-09-16T14:49:50.941580+0000","last_undegraded":"2024-09-16T14:49:50.941580+0000","last_fullsized":"2024-09-16T14:49:50.941580+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'74","last_scrub_stamp":"2024-09-16T14:49:40.525004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.525004+0000","objects_scrubbed":2,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:04.420225+0000","scrub_duration":0.0030813329999999999,"objects_trimmed":0,"snaptrim_duration":0.0023863780000000002,"stat_sum":{"num_bytes":3194880,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":1538,"num_write":5,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,9,5,10,2,6,0],"acting":[1,11,9,5,10,2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.859573+0000","last_change":"2024-09-16T14:48:26.161822+0000","last_active":"2024-09-16T14:49:50.859573+0000","last_peered":"2024-09-16T14:49:50.859573+0000","last_clean":"2024-09-16T14:49:50.859573+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:49:50.859573+0000","last_undegraded":"2024-09-16T14:49:50.859573+0000","last_fullsized":"2024-09-16T14:49:50.859573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"103'80","reported_seq":464,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.450013+0000","last_change":"2024-09-16T14:49:50.450013+0000","last_active":"2024-09-16T14:49:50.450013+0000","last_peered":"2024-09-16T14:49:50.450013+0000","last_clean":"2024-09-16T14:49:50.450013+0000","last_became_active":"2024-09-16T14:49:37.958130+0000","last_became_peered":"2024-09-16T14:49:37.958130+0000","last_unstale":"2024-09-16T14:49:50.450013+0000","last_undegraded":"2024-09-16T14:49:50.450013+0000","last_fullsized":"2024-09-16T14:49:50.450013+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'80","last_scrub_stamp":"2024-09-16T14:49:50.449861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:50.449861+0000","objects_scrubbed":3,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:17.237609+0000","scrub_duration":0.0035009910000000002,"objects_trimmed":0,"snaptrim_duration":0.00080948000000000001,"stat_sum":{"num_bytes":8781824,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6,7,1,11,10],"acting":[5,3,2,6,7,1,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.3","version":"83'37","reported_seq":344,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.859337+0000","last_change":"2024-09-16T14:49:49.023007+0000","last_active":"2024-09-16T14:49:49.859337+0000","last_peered":"2024-09-16T14:49:49.859337+0000","last_clean":"2024-09-16T14:49:49.859337+0000","last_became_active":"2024-09-16T14:49:37.960053+0000","last_became_peered":"2024-09-16T14:49:37.960053+0000","last_unstale":"2024-09-16T14:49:49.859337+0000","last_undegraded":"2024-09-16T14:49:49.859337+0000","last_fullsized":"2024-09-16T14:49:49.859337+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"83'37","last_scrub_stamp":"2024-09-16T14:49:38.437020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:38.437020+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:44.378876+0000","scrub_duration":0.0029472029999999998,"objects_trimmed":0,"snaptrim_duration":0.0055016789999999998,"stat_sum":{"num_bytes":81920,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,6,11,5,9,10],"acting":[4,3,0,6,11,5,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.2","version":"23'2","reported_seq":110,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.876420+0000","last_change":"2024-09-16T14:49:28.351058+0000","last_active":"2024-09-16T14:49:49.876420+0000","last_peered":"2024-09-16T14:49:49.876420+0000","last_clean":"2024-09-16T14:49:49.876420+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:49:49.876420+0000","last_undegraded":"2024-09-16T14:49:49.876420+0000","last_fullsized":"2024-09-16T14:49:49.876420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-16T14:49:28.350938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:28.350938+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:42.292353+0000","scrub_duration":0.0018950309999999999,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"91'47","reported_seq":345,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-16T14:49:51.917258+0000","last_change":"2024-09-16T14:49:49.021090+0000","last_active":"2024-09-16T14:49:51.917258+0000","last_peered":"2024-09-16T14:49:51.917258+0000","last_clean":"2024-09-16T14:49:51.917258+0000","last_became_active":"2024-09-16T14:49:37.961051+0000","last_became_peered":"2024-09-16T14:49:37.961051+0000","last_unstale":"2024-09-16T14:49:51.917258+0000","last_undegraded":"2024-09-16T14:49:51.917258+0000","last_fullsized":"2024-09-16T14:49:51.917258+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"91'47","last_scrub_stamp":"2024-09-16T14:49:41.126704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:41.126704+0000","objects_scrubbed":2,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:44.106417+0000","scrub_duration":0.0027967320000000001,"objects_trimmed":0,"snaptrim_duration":0.0026831110000000002,"stat_sum":{"num_bytes":5767168,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":9169,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6,1,7,8,0,5],"acting":[11,9,6,1,7,8,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":106,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883009+0000","last_change":"2024-09-16T14:49:27.155463+0000","last_active":"2024-09-16T14:49:49.883009+0000","last_peered":"2024-09-16T14:49:49.883009+0000","last_clean":"2024-09-16T14:49:49.883009+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:49:49.883009+0000","last_undegraded":"2024-09-16T14:49:49.883009+0000","last_fullsized":"2024-09-16T14:49:49.883009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:27.155307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:27.155307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:32.986356+0000","scrub_duration":0.001601157,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":279,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.868799+0000","last_change":"2024-09-16T14:49:49.019384+0000","last_active":"2024-09-16T14:49:50.868799+0000","last_peered":"2024-09-16T14:49:50.868799+0000","last_clean":"2024-09-16T14:49:50.868799+0000","last_became_active":"2024-09-16T14:49:37.959116+0000","last_became_peered":"2024-09-16T14:49:37.959116+0000","last_unstale":"2024-09-16T14:49:50.868799+0000","last_undegraded":"2024-09-16T14:49:50.868799+0000","last_fullsized":"2024-09-16T14:49:50.868799+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:42.094901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.094901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:00.419660+0000","scrub_duration":0.0018558680000000001,"objects_trimmed":0,"snaptrim_duration":0.0010354800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,10,11,5,9,6,1],"acting":[0,4,10,11,5,9,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882809+0000","last_change":"2024-09-16T14:48:26.568303+0000","last_active":"2024-09-16T14:49:49.882809+0000","last_peered":"2024-09-16T14:49:49.882809+0000","last_clean":"2024-09-16T14:49:49.882809+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:49:49.882809+0000","last_undegraded":"2024-09-16T14:49:49.882809+0000","last_fullsized":"2024-09-16T14:49:49.882809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:48:23.132742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"98'110","reported_seq":422,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882601+0000","last_change":"2024-09-16T14:49:49.013432+0000","last_active":"2024-09-16T14:49:49.882601+0000","last_peered":"2024-09-16T14:49:49.882601+0000","last_clean":"2024-09-16T14:49:49.882601+0000","last_became_active":"2024-09-16T14:49:37.958372+0000","last_became_peered":"2024-09-16T14:49:37.958372+0000","last_unstale":"2024-09-16T14:49:49.882601+0000","last_undegraded":"2024-09-16T14:49:49.882601+0000","last_fullsized":"2024-09-16T14:49:49.882601+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"84'102","last_scrub_stamp":"2024-09-16T14:49:38.639259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:38.639259+0000","objects_scrubbed":1,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:50.153167+0000","scrub_duration":0.0030648189999999999,"objects_trimmed":0,"snaptrim_duration":0.001860719,"stat_sum":{"num_bytes":4112384,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,6,4,5,1,8,10],"acting":[3,9,6,4,5,1,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"2.3","version":"21'1","reported_seq":109,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.876474+0000","last_change":"2024-09-16T14:49:40.527258+0000","last_active":"2024-09-16T14:49:49.876474+0000","last_peered":"2024-09-16T14:49:49.876474+0000","last_clean":"2024-09-16T14:49:49.876474+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:49:49.876474+0000","last_undegraded":"2024-09-16T14:49:49.876474+0000","last_fullsized":"2024-09-16T14:49:49.876474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2024-09-16T14:49:40.527164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.527164+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:46.029615+0000","scrub_duration":0.0012857120000000001,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":267,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883100+0000","last_change":"2024-09-16T14:48:22.359960+0000","last_active":"2024-09-16T14:49:49.883100+0000","last_peered":"2024-09-16T14:49:49.883100+0000","last_clean":"2024-09-16T14:49:49.883100+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:49:49.883100+0000","last_undegraded":"2024-09-16T14:49:49.883100+0000","last_fullsized":"2024-09-16T14:49:49.883100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:48:21.118926+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"102'105","reported_seq":382,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.911099+0000","last_change":"2024-09-16T14:49:49.034250+0000","last_active":"2024-09-16T14:49:50.911099+0000","last_peered":"2024-09-16T14:49:50.911099+0000","last_clean":"2024-09-16T14:49:50.911099+0000","last_became_active":"2024-09-16T14:49:37.960827+0000","last_became_peered":"2024-09-16T14:49:37.960827+0000","last_unstale":"2024-09-16T14:49:50.911099+0000","last_undegraded":"2024-09-16T14:49:50.911099+0000","last_fullsized":"2024-09-16T14:49:50.911099+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'97","last_scrub_stamp":"2024-09-16T14:49:40.167318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.167318+0000","objects_scrubbed":2,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:55.504556+0000","scrub_duration":0.002425441,"objects_trimmed":0,"snaptrim_duration":0.016083350999999999,"stat_sum":{"num_bytes":4407296,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,2,4,3,5,10,8],"acting":[11,1,2,4,3,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.9","version":"102'83","reported_seq":366,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.976342+0000","last_change":"2024-09-16T14:49:49.031507+0000","last_active":"2024-09-16T14:49:49.976342+0000","last_peered":"2024-09-16T14:49:49.976342+0000","last_clean":"2024-09-16T14:49:49.976342+0000","last_became_active":"2024-09-16T14:49:37.960644+0000","last_became_peered":"2024-09-16T14:49:37.960644+0000","last_unstale":"2024-09-16T14:49:49.976342+0000","last_undegraded":"2024-09-16T14:49:49.976342+0000","last_fullsized":"2024-09-16T14:49:49.976342+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'77","last_scrub_stamp":"2024-09-16T14:49:40.131033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.131033+0000","objects_scrubbed":3,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:51.352479+0000","scrub_duration":0.0026976370000000001,"objects_trimmed":1,"snaptrim_duration":0.016770514,"stat_sum":{"num_bytes":5701632,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7,9,5,1,4,10],"acting":[8,11,7,9,5,1,4,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.a","version":"99'68","reported_seq":194,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T14:49:45.952464+0000","last_change":"2024-09-16T14:49:44.889238+0000","last_active":"2024-09-16T14:49:45.952464+0000","last_peered":"2024-09-16T14:49:45.952464+0000","last_clean":"2024-09-16T14:49:45.952464+0000","last_became_active":"2024-09-16T14:49:19.868841+0000","last_became_peered":"2024-09-16T14:49:19.868841+0000","last_unstale":"2024-09-16T14:49:45.952464+0000","last_undegraded":"2024-09-16T14:49:45.952464+0000","last_fullsized":"2024-09-16T14:49:45.952464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01518982,"stat_sum":{"num_bytes":9125888,"num_objects":3,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":1063,"num_write":48,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":9224192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,10,8,9,3,7],"acting":[6,4,1,10,8,9,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"2"},{"start":"2e","length":"2"},{"start":"31","length":"1"},{"start":"37","length":"5"},{"start":"40","length":"1"},{"start":"43","length":"1"}]},{"pgid":"3.b","version":"90'77","reported_seq":343,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882753+0000","last_change":"2024-09-16T14:49:49.011747+0000","last_active":"2024-09-16T14:49:49.882753+0000","last_peered":"2024-09-16T14:49:49.882753+0000","last_clean":"2024-09-16T14:49:49.882753+0000","last_became_active":"2024-09-16T14:49:12.972366+0000","last_became_peered":"2024-09-16T14:49:12.972366+0000","last_unstale":"2024-09-16T14:49:49.882753+0000","last_undegraded":"2024-09-16T14:49:49.882753+0000","last_fullsized":"2024-09-16T14:49:49.882753+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:03.456665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043729700000000001,"stat_sum":{"num_bytes":1130496,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":7388,"num_write":44,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1785856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,8,11,7,1,6],"acting":[3,0,4,8,11,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.c","version":"102'89","reported_seq":377,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.876733+0000","last_change":"2024-09-16T14:49:49.029918+0000","last_active":"2024-09-16T14:49:49.876733+0000","last_peered":"2024-09-16T14:49:49.876733+0000","last_clean":"2024-09-16T14:49:49.876733+0000","last_became_active":"2024-09-16T14:49:12.979797+0000","last_became_peered":"2024-09-16T14:49:12.979797+0000","last_unstale":"2024-09-16T14:49:49.876733+0000","last_undegraded":"2024-09-16T14:49:49.876733+0000","last_fullsized":"2024-09-16T14:49:49.876733+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.201481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098966060000000005,"stat_sum":{"num_bytes":7389184,"num_objects":5,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":12010,"num_write":33,"num_write_kb":9920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3440640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,10,9,2,0,11],"acting":[5,3,6,10,9,2,0,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.d","version":"98'82","reported_seq":387,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.882964+0000","last_change":"2024-09-16T14:49:49.034992+0000","last_active":"2024-09-16T14:49:49.882964+0000","last_peered":"2024-09-16T14:49:49.882964+0000","last_clean":"2024-09-16T14:49:49.882964+0000","last_became_active":"2024-09-16T14:49:12.979812+0000","last_became_peered":"2024-09-16T14:49:12.979812+0000","last_unstale":"2024-09-16T14:49:49.882964+0000","last_undegraded":"2024-09-16T14:49:49.882964+0000","last_fullsized":"2024-09-16T14:49:49.882964+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010903285,"stat_sum":{"num_bytes":5210112,"num_objects":3,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":10527,"num_write":78,"num_write_kb":7952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4489216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,9,6,11,3,4,0],"acting":[7,5,9,6,11,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.e","version":"102'86","reported_seq":334,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.911320+0000","last_change":"2024-09-16T14:49:49.035327+0000","last_active":"2024-09-16T14:49:50.911320+0000","last_peered":"2024-09-16T14:49:50.911320+0000","last_clean":"2024-09-16T14:49:50.911320+0000","last_became_active":"2024-09-16T14:48:35.075626+0000","last_became_peered":"2024-09-16T14:48:35.075626+0000","last_unstale":"2024-09-16T14:49:50.911320+0000","last_undegraded":"2024-09-16T14:49:50.911320+0000","last_fullsized":"2024-09-16T14:49:50.911320+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:54.908348+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.016608049,"stat_sum":{"num_bytes":13107200,"num_objects":2,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":4441,"num_write":75,"num_write_kb":13360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,1,4,0,9,3,10],"acting":[11,8,1,4,0,9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.f","version":"104'64","reported_seq":307,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.965499+0000","last_change":"2024-09-16T14:49:49.034518+0000","last_active":"2024-09-16T14:49:50.965499+0000","last_peered":"2024-09-16T14:49:50.965499+0000","last_clean":"2024-09-16T14:49:50.965499+0000","last_became_active":"2024-09-16T14:48:35.075111+0000","last_became_peered":"2024-09-16T14:48:35.075111+0000","last_unstale":"2024-09-16T14:49:50.965499+0000","last_undegraded":"2024-09-16T14:49:50.965499+0000","last_fullsized":"2024-09-16T14:49:50.965499+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:49:58.979883+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01172438,"stat_sum":{"num_bytes":3915776,"num_objects":2,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":10587,"num_write":62,"num_write_kb":7728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8,2,9,10,0,4],"acting":[7,11,8,2,9,10,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.10","version":"82'43","reported_seq":55,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T14:49:45.873772+0000","last_change":"2024-09-16T14:49:44.876682+0000","last_active":"2024-09-16T14:49:45.873772+0000","last_peered":"2024-09-16T14:49:45.873772+0000","last_clean":"2024-09-16T14:49:45.873772+0000","last_became_active":"2024-09-16T14:49:39.890294+0000","last_became_peered":"2024-09-16T14:49:39.890294+0000","last_unstale":"2024-09-16T14:49:45.873772+0000","last_undegraded":"2024-09-16T14:49:45.873772+0000","last_fullsized":"2024-09-16T14:49:45.873772+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"82'43","last_scrub_stamp":"2024-09-16T14:49:42.551604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:42.551604+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:59.224582+0000","scrub_duration":0.0021534729999999999,"objects_trimmed":0,"snaptrim_duration":0.0032657860000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,5,3,0,2,1,7],"acting":[6,9,5,3,0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"2"},{"start":"2e","length":"2"},{"start":"31","length":"1"},{"start":"37","length":"5"},{"start":"40","length":"1"},{"start":"43","length":"1"}]},{"pgid":"3.11","version":"0'0","reported_seq":93,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.873262+0000","last_change":"2024-09-16T14:49:49.015897+0000","last_active":"2024-09-16T14:49:50.873262+0000","last_peered":"2024-09-16T14:49:50.873262+0000","last_clean":"2024-09-16T14:49:50.873262+0000","last_became_active":"2024-09-16T14:49:39.875227+0000","last_became_peered":"2024-09-16T14:49:39.875227+0000","last_unstale":"2024-09-16T14:49:50.873262+0000","last_undegraded":"2024-09-16T14:49:50.873262+0000","last_fullsized":"2024-09-16T14:49:50.873262+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:40.245626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.245626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:41.418675+0000","scrub_duration":0.0016751209999999999,"objects_trimmed":0,"snaptrim_duration":0.00079420900000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6,0,3,9,1,11],"acting":[10,4,6,0,3,9,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.12","version":"96'106","reported_seq":165,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T14:49:50.868833+0000","last_change":"2024-09-16T14:49:49.018091+0000","last_active":"2024-09-16T14:49:50.868833+0000","last_peered":"2024-09-16T14:49:50.868833+0000","last_clean":"2024-09-16T14:49:50.868833+0000","last_became_active":"2024-09-16T14:49:42.870631+0000","last_became_peered":"2024-09-16T14:49:42.870631+0000","last_unstale":"2024-09-16T14:49:50.868833+0000","last_undegraded":"2024-09-16T14:49:50.868833+0000","last_fullsized":"2024-09-16T14:49:50.868833+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060451400000000001,"stat_sum":{"num_bytes":3088384,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":3413,"num_write":30,"num_write_kb":4640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":7602176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1,2,5,6,11],"acting":[0,7,3,1,2,5,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"46","length":"1"}]},{"pgid":"3.13","version":"102'45","reported_seq":95,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T14:49:49.883456+0000","last_change":"2024-09-16T14:49:49.034362+0000","last_active":"2024-09-16T14:49:49.883456+0000","last_peered":"2024-09-16T14:49:49.883456+0000","last_clean":"2024-09-16T14:49:49.883456+0000","last_became_active":"2024-09-16T14:49:39.885593+0000","last_became_peered":"2024-09-16T14:49:39.885593+0000","last_unstale":"2024-09-16T14:49:49.883456+0000","last_undegraded":"2024-09-16T14:49:49.883456+0000","last_fullsized":"2024-09-16T14:49:49.883456+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:33.849880+0000","last_clean_scrub_stamp":"2024-09-16T14:48:33.849880+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.011674103,"stat_sum":{"num_bytes":868352,"num_objects":1,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":5278,"num_write":19,"num_write_kb":1112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1695744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2,6,1,9,8],"acting":[7,5,4,2,6,1,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"46","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":26,"stat_sum":{"num_bytes":109944832,"num_objects":52,"num_object_clones":11,"num_object_copies":416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":240,"num_read_kb":86818,"num_write":556,"num_write_kb":90688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":49225728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":224763904,"data_stored":252866560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1984,"ondisk_log_size":1984,"up":208,"acting":208,"num_store_stats":12},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411348,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77140,"kb_used_data":23880,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94294700,"statfs":{"total":96636764160,"available":96557772800,"internally_reserved":0,"allocated":24453120,"data_stored":28269658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.52700000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.38300000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.46700000000000003}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.437},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.311}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.57099999999999995}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.48699999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.503},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51800000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59799999999999998}]},{"osd":9,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.622}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.64800000000000002}]}]},{"osd":10,"up_from":18,"seq":77309411350,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68084,"kb_used_data":18920,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94303756,"statfs":{"total":96636764160,"available":96567046144,"internally_reserved":0,"allocated":19374080,"data_stored":20768301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.505},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.48799999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.60399999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57299999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.46100000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.432}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72299999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.39700000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.37},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.38}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.40799999999999997}]}]},{"osd":6,"up_from":41,"seq":176093659150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46620,"kb_used_data":15632,"kb_used_omap":1,"kb_used_meta":30974,"kb_avail":94325220,"statfs":{"total":96636764160,"available":96589025280,"internally_reserved":0,"allocated":16007168,"data_stored":17012453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1965,"internal_metadata":31717459},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":1,"up_from":18,"seq":77309411350,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91184,"kb_used_data":23972,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94280656,"statfs":{"total":96636764160,"available":96543391744,"internally_reserved":0,"allocated":24547328,"data_stored":26425852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.312}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":4.0590000000000002,"5min":4.0590000000000002,"15min":4.0590000000000002},"last":0.42999999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.503},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.379}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.49399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":3.9620000000000002,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.55600000000000005}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.0030000000000001,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.79300000000000004}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.67500000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.68700000000000006}]},{"osd":11,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":4.6589999999999998,"5min":4.6589999999999998,"15min":4.6589999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.66500000000000004}]}]},{"osd":0,"up_from":18,"seq":77309411348,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66224,"kb_used_data":17056,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94305616,"statfs":{"total":96636764160,"available":96568950784,"internally_reserved":0,"allocated":17465344,"data_stored":19409466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.41399999999999998}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.59999999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.5}]},{"osd":8,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.629},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.77800000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.57299999999999995}]},{"osd":10,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.745}]},{"osd":11,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.84299999999999997}]}]},{"osd":5,"up_from":18,"seq":77309411349,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68260,"kb_used_data":19088,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94303580,"statfs":{"total":96636764160,"available":96566865920,"internally_reserved":0,"allocated":19546112,"data_stored":20416939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.46000000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.38}]},{"osd":2,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.436}]},{"osd":3,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.30199999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.34000000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.503}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63800000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.73299999999999998}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.624}]}]},{"osd":9,"up_from":17,"seq":73014444053,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69128,"kb_used_data":19956,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94302712,"statfs":{"total":96636764160,"available":96565977088,"internally_reserved":0,"allocated":20434944,"data_stored":20539800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.47499999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.46200000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.40400000000000003}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.38},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.44400000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.627},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.57499999999999996}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.67700000000000005}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.435}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.53900000000000003}]},{"osd":11,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.58999999999999997}]}]},{"osd":8,"up_from":17,"seq":73014444053,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63980,"kb_used_data":14808,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94307860,"statfs":{"total":96636764160,"available":96571248640,"internally_reserved":0,"allocated":15163392,"data_stored":14997912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.59299999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.79700000000000004}]},{"osd":5,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.69699999999999995}]},{"osd":6,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68200000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.74099999999999999}]},{"osd":9,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.77000000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.81100000000000005}]},{"osd":11,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.75800000000000001}]}]},{"osd":7,"up_from":17,"seq":73014444053,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66072,"kb_used_data":16904,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94305768,"statfs":{"total":96636764160,"available":96569106432,"internally_reserved":0,"allocated":17309696,"data_stored":17337272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":50330060},"hb_peers":[0,1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5620000000000003,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.58299999999999996}]},{"osd":3,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.65800000000000003}]},{"osd":4,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52000000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.313},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.40600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.442}]},{"osd":9,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57199999999999995}]},{"osd":10,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.50700000000000001}]},{"osd":11,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.49199999999999999}]}]},{"osd":4,"up_from":17,"seq":73014444052,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66264,"kb_used_data":17092,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94305576,"statfs":{"total":96636764160,"available":96568909824,"internally_reserved":0,"allocated":17502208,"data_stored":17336728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.64200000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.65100000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.435},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.47999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.38300000000000001}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.504}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.58199999999999996}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.621},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.59699999999999998}]},{"osd":11,"last update":"Mon Sep 16 14:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.68200000000000005}]}]},{"osd":3,"up_from":17,"seq":73014444054,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71548,"kb_used_data":22376,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94300292,"statfs":{"total":96636764160,"available":96563499008,"internally_reserved":0,"allocated":22913024,"data_stored":23787928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.253},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.32700000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.28399999999999997}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.34999999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.46100000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.436},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.47299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.51400000000000001}]},{"osd":8,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58199999999999996}]},{"osd":9,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.55200000000000005}]},{"osd":10,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.63800000000000001}]},{"osd":11,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.62},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.60699999999999998}]}]},{"osd":2,"up_from":17,"seq":73014444053,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47048,"kb_used_data":15932,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94324792,"statfs":{"total":96636764160,"available":96588587008,"internally_reserved":0,"allocated":16314368,"data_stored":18921961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.28499999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.38200000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.32300000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.46600000000000003}]},{"osd":8,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.51700000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.69799999999999995}]},{"osd":10,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.68600000000000005}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16232448,"data_stored":18636800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23916544,"data_stored":27471872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15093760,"data_stored":18157568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22286336,"data_stored":24829952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17461248,"data_stored":18247680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20078592,"data_stored":22863872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":14835712,"data_stored":16281600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16736256,"data_stored":17719296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":15122432,"data_stored":15908864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":21561344,"data_stored":23965696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":18751488,"data_stored":21815296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":22687744,"data_stored":26968064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:49:53.861 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pgp_num 26 2024-09-16T14:49:53.956 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:53.955+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:54.006 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:54.006 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:54.016 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T14:49:54.057 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-16T14:49:54.056+0000 7f7dfceb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32911) UID: 0 2024-09-16T14:49:54.157 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:54.156+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:54.162 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:54.163 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:54.172 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T14:49:54.259 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:54.258+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:54.278 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:54.278 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:54.287 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T14:49:54.359 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:54.358+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:54.394 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:54.394 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:54.403 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T14:49:54.460 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:54.459+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:54.509 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:54.509 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:54.518 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T14:49:54.563 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:54.562+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:54.626 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:54.626 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:54.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T14:49:54.676 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:54.675+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:54.763 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:54.762+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:54.782 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:54.782 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:54.790 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T14:49:54.864 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:54.863+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:54.879 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T14:49:54.898 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:54.898 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:54.907 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T14:49:54.965 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:54.964+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:55.016 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T14:49:55.034 INFO:teuthology.orchestra.run.smithi040.stderr:set pool 3 pgp_num to 26 2024-09-16T14:49:55.055 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-16T14:49:55.055 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-16T14:49:55.063 INFO:teuthology.orchestra.run.smithi040.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:55.063 INFO:teuthology.orchestra.run.smithi040.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:55.066 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-16T14:49:55.064+0000 7f552914a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32900) UID: 0 2024-09-16T14:49:55.073 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T14:49:55.146 INFO:teuthology.orchestra.run.smithi040.stdout:osd.0: {} 2024-09-16T14:49:55.146 INFO:teuthology.orchestra.run.smithi040.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T14:49:55.163 INFO:teuthology.orchestra.run.smithi040.stdout:osd.1: {} 2024-09-16T14:49:55.163 INFO:teuthology.orchestra.run.smithi040.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T14:49:55.166 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-16T14:49:55.165+0000 7f7ccf5d3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32901) UID: 0 2024-09-16T14:49:55.180 INFO:teuthology.orchestra.run.smithi040.stdout:osd.2: {} 2024-09-16T14:49:55.181 INFO:teuthology.orchestra.run.smithi040.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T14:49:55.189 INFO:teuthology.orchestra.run.smithi111.stdout:ERROR: (22) Invalid argument 2024-09-16T14:49:55.189 INFO:teuthology.orchestra.run.smithi111.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T14:49:55.196 INFO:teuthology.orchestra.run.smithi040.stdout:osd.3: {} 2024-09-16T14:49:55.197 INFO:teuthology.orchestra.run.smithi040.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T14:49:55.198 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T14:49:55.213 INFO:teuthology.orchestra.run.smithi040.stdout:osd.4: {} 2024-09-16T14:49:55.213 INFO:teuthology.orchestra.run.smithi040.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T14:49:55.230 INFO:teuthology.orchestra.run.smithi040.stdout:osd.5: {} 2024-09-16T14:49:55.230 INFO:teuthology.orchestra.run.smithi040.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T14:49:55.246 INFO:teuthology.orchestra.run.smithi040.stdout:osd.6: {} 2024-09-16T14:49:55.247 INFO:teuthology.orchestra.run.smithi040.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T14:49:55.262 INFO:teuthology.orchestra.run.smithi040.stdout:osd.7: {} 2024-09-16T14:49:55.263 INFO:teuthology.orchestra.run.smithi040.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T14:49:55.267 INFO:tasks.ceph.osd.11.smithi114.stderr:2024-09-16T14:49:55.266+0000 7fe30d672640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 33119) UID: 0 2024-09-16T14:49:55.279 INFO:teuthology.orchestra.run.smithi040.stdout:osd.8: {} 2024-09-16T14:49:55.279 INFO:teuthology.orchestra.run.smithi040.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-16T14:49:55.297 INFO:teuthology.orchestra.run.smithi040.stdout:osd.9: {} 2024-09-16T14:49:55.297 INFO:teuthology.orchestra.run.smithi040.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-16T14:49:55.302 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:55.302 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:55.302 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:55.303 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:55.303 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:55.311 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T14:49:55.313 INFO:teuthology.orchestra.run.smithi040.stdout:osd.10: {} 2024-09-16T14:49:55.314 INFO:teuthology.orchestra.run.smithi040.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-16T14:49:55.330 INFO:teuthology.orchestra.run.smithi040.stdout:osd.11: {} 2024-09-16T14:49:55.331 INFO:teuthology.orchestra.run.smithi040.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-16T14:49:55.368 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:55.366+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:55.416 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:55.416 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:55.416 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:55.416 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:55.416 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.417 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs2 37/27 ECSubWrite(tid=116, reqid=osd.3.0:116, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:55.417 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:55.417 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 71.493331287999993, 2024-09-16T14:49:55.417 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010231874, 2024-09-16T14:49:55.417 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.417 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.417 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.417 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.417 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.417 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:55.417 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.417 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.417 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.417 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.418 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:55.418 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.418 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.418 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.418 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.418 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920703+0000", 2024-09-16T14:49:55.418 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.418 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.418 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.418 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.418 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920725+0000", 2024-09-16T14:49:55.418 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2198999999999999e-05 2024-09-16T14:49:55.418 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.418 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.419 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.419 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920735+0000", 2024-09-16T14:49:55.419 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0122000000000001e-05 2024-09-16T14:49:55.419 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.419 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.419 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.419 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920742+0000", 2024-09-16T14:49:55.419 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7179999999999999e-06 2024-09-16T14:49:55.419 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.419 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.419 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.419 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921079+0000", 2024-09-16T14:49:55.419 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00033676000000000002 2024-09-16T14:49:55.420 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.420 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.420 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.420 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921088+0000", 2024-09-16T14:49:55.420 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9290000000000004e-06 2024-09-16T14:49:55.420 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.420 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.420 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.420 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930916+0000", 2024-09-16T14:49:55.420 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0098279749999999992 2024-09-16T14:49:55.420 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.420 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.420 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.421 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930935+0000", 2024-09-16T14:49:55.421 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9510000000000001e-05 2024-09-16T14:49:55.421 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.421 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.421 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.421 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.421 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.421 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.fs7 37/27 ECSubWrite(tid=189, reqid=osd.7.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=37'12))", 2024-09-16T14:49:55.421 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:55.421 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 71.493143588999999, 2024-09-16T14:49:55.421 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010007947999999999, 2024-09-16T14:49:55.421 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.421 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.421 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.422 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.422 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.422 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:55.422 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.422 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.422 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.422 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.422 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920891+0000", 2024-09-16T14:49:55.422 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.422 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.422 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.422 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.422 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920890+0000", 2024-09-16T14:49:55.422 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:55.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.423 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.423 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.423 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920900+0000", 2024-09-16T14:49:55.423 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9839999999999996e-06 2024-09-16T14:49:55.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.423 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.423 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.423 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920910+0000", 2024-09-16T14:49:55.423 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4220000000000006e-06 2024-09-16T14:49:55.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.424 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.424 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.424 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920917+0000", 2024-09-16T14:49:55.424 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5349999999999999e-06 2024-09-16T14:49:55.424 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.424 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.424 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.424 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920967+0000", 2024-09-16T14:49:55.424 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9348000000000003e-05 2024-09-16T14:49:55.424 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.424 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.424 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.424 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.920976+0000", 2024-09-16T14:49:55.424 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8810000000000002e-06 2024-09-16T14:49:55.425 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.425 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.425 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.425 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930871+0000", 2024-09-16T14:49:55.425 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0098950500000000007 2024-09-16T14:49:55.425 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.425 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.425 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.425 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.930899+0000", 2024-09-16T14:49:55.425 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.847e-05 2024-09-16T14:49:55.425 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.425 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.425 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.426 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.426 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.2s3 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:55.426 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:55.426 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 71.492146547000004, 2024-09-16T14:49:55.426 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011669621999999999, 2024-09-16T14:49:55.426 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.426 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.426 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.426 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.426 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.426 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:55.426 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921888+0000", 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921887+0000", 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921911+0000", 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4012999999999999e-05 2024-09-16T14:49:55.427 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.428 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.428 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.428 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921923+0000", 2024-09-16T14:49:55.428 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1642e-05 2024-09-16T14:49:55.428 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.428 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.428 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.428 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921932+0000", 2024-09-16T14:49:55.428 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0210000000000005e-06 2024-09-16T14:49:55.428 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.428 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.428 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.428 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921988+0000", 2024-09-16T14:49:55.428 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5822999999999997e-05 2024-09-16T14:49:55.429 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.429 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.429 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.429 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921997+0000", 2024-09-16T14:49:55.429 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7900000000000005e-06 2024-09-16T14:49:55.429 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.429 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.429 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.429 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933539+0000", 2024-09-16T14:49:55.429 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011542248 2024-09-16T14:49:55.429 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.429 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.429 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.429 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933558+0000", 2024-09-16T14:49:55.430 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8964999999999999e-05 2024-09-16T14:49:55.430 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.430 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.430 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.430 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.430 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.430 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds6 37/27 ECSubWrite(tid=190, reqid=osd.7.0:188, at_version=0'0, trim_to=0'0, roll_forward_to=37'20))", 2024-09-16T14:49:55.430 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:55.430 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 71.492064947000003, 2024-09-16T14:49:55.430 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0090799120000000007, 2024-09-16T14:49:55.430 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.430 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.430 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.430 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.431 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.431 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:55.431 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.431 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.431 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.431 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.431 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921970+0000", 2024-09-16T14:49:55.431 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.431 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.431 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.431 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.431 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921969+0000", 2024-09-16T14:49:55.431 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:55.431 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.432 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.432 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.432 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921983+0000", 2024-09-16T14:49:55.432 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4403000000000001e-05 2024-09-16T14:49:55.432 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.432 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.432 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.432 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.921996+0000", 2024-09-16T14:49:55.432 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.259e-05 2024-09-16T14:49:55.432 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.432 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.432 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.432 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922005+0000", 2024-09-16T14:49:55.432 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9080000000000001e-06 2024-09-16T14:49:55.433 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.433 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.433 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.433 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922073+0000", 2024-09-16T14:49:55.433 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8127999999999996e-05 2024-09-16T14:49:55.433 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.433 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.433 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.433 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.922085+0000", 2024-09-16T14:49:55.433 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1712999999999999e-05 2024-09-16T14:49:55.433 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.433 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.433 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.433 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931031+0000", 2024-09-16T14:49:55.434 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089464669999999996 2024-09-16T14:49:55.434 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.434 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.434 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.434 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.931050+0000", 2024-09-16T14:49:55.434 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8471999999999999e-05 2024-09-16T14:49:55.434 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.434 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.434 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.434 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.434 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.434 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as1 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:55.434 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:55.434 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 71.489969697999996, 2024-09-16T14:49:55.435 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0095052339999999996, 2024-09-16T14:49:55.435 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.435 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.435 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.435 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.435 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.435 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:55.435 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.435 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.435 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.435 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.435 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:55.435 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.435 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.436 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.436 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.436 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924065+0000", 2024-09-16T14:49:55.436 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:55.436 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.436 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.436 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.436 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924089+0000", 2024-09-16T14:49:55.436 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3768999999999999e-05 2024-09-16T14:49:55.436 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.436 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.436 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.436 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924101+0000", 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2294e-05 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924109+0000", 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7389999999999999e-06 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924163+0000", 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.4052999999999999e-05 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.437 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924172+0000", 2024-09-16T14:49:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7189999999999996e-06 2024-09-16T14:49:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933561+0000", 2024-09-16T14:49:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0093884659999999998 2024-09-16T14:49:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933570+0000", 2024-09-16T14:49:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4120000000000007e-06 2024-09-16T14:49:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.438 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.439 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.439 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.439 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.es3 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:55.439 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:55.439 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 71.489912262999994, 2024-09-16T14:49:55.439 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0094028089999999998, 2024-09-16T14:49:55.439 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.439 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.439 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.439 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.440 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.440 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:55.440 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.440 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.440 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.440 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.440 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:55.440 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.440 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.440 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.440 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.440 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924122+0000", 2024-09-16T14:49:55.440 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.441 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.441 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.441 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.441 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924142+0000", 2024-09-16T14:49:55.441 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0162000000000001e-05 2024-09-16T14:49:55.441 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.441 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.441 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.441 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924150+0000", 2024-09-16T14:49:55.441 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0919999999999998e-06 2024-09-16T14:49:55.441 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.441 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.441 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.441 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924157+0000", 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6069999999999996e-06 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924215+0000", 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8192000000000001e-05 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.924225+0000", 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9760000000000007e-06 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.442 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933494+0000", 2024-09-16T14:49:55.443 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0092692720000000003 2024-09-16T14:49:55.443 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.443 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.443 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.443 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:43.933525+0000", 2024-09-16T14:49:55.443 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0976999999999998e-05 2024-09-16T14:49:55.443 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.443 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.443 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.443 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.443 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.443 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:594 3.3s0 3:c9d06351:::smithi04040151-33:head [append 0~491520] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:55.443 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.981536+0000", 2024-09-16T14:49:55.443 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 43.432498490999997, 2024-09-16T14:49:55.444 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0295208730000001, 2024-09-16T14:49:55.444 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.444 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:55.444 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.444 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.444 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.444 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 594 2024-09-16T14:49:55.444 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.444 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.444 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.444 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.444 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.981536+0000", 2024-09-16T14:49:55.444 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.445 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.445 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.445 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.445 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.981536+0000", 2024-09-16T14:49:55.445 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.445 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.445 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.445 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.445 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.981535+0000", 2024-09-16T14:49:55.445 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:55.445 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.445 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.445 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.445 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982118+0000", 2024-09-16T14:49:55.446 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00058284500000000002 2024-09-16T14:49:55.446 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.446 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.446 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.446 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982129+0000", 2024-09-16T14:49:55.446 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1644999999999999e-05 2024-09-16T14:49:55.446 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.446 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.446 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.446 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982134+0000", 2024-09-16T14:49:55.446 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.152e-06 2024-09-16T14:49:55.446 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.446 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.447 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.447 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982188+0000", 2024-09-16T14:49:55.447 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3829999999999998e-05 2024-09-16T14:49:55.447 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.447 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.447 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:55.447 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.982192+0000", 2024-09-16T14:49:55.447 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5980000000000001e-06 2024-09-16T14:49:55.447 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.447 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.447 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.447 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980648+0000", 2024-09-16T14:49:55.447 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99845620300000004 2024-09-16T14:49:55.448 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.448 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.448 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:55.448 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980918+0000", 2024-09-16T14:49:55.448 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00026981999999999999 2024-09-16T14:49:55.448 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.448 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.448 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.448 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.002491+0000", 2024-09-16T14:49:55.448 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.021573093000000002 2024-09-16T14:49:55.448 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.448 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.448 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.448 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.002715+0000", 2024-09-16T14:49:55.449 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00022431799999999999 2024-09-16T14:49:55.449 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.449 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.449 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.449 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004518+0000", 2024-09-16T14:49:55.449 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001802339 2024-09-16T14:49:55.449 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.449 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.449 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.449 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009950+0000", 2024-09-16T14:49:55.449 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0054329260000000002 2024-09-16T14:49:55.449 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.449 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.449 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:55.450 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.010633+0000", 2024-09-16T14:49:55.450 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00068230700000000005 2024-09-16T14:49:55.450 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.450 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.450 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.450 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.011057+0000", 2024-09-16T14:49:55.450 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000424313 2024-09-16T14:49:55.450 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.450 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.450 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.450 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.450 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.450 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:595 3.3s0 3:c9d06351:::smithi04040151-33:head [append 0~589824] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.985758+0000", 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 43.428276562999997, 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0310086730000001, 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 595 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.985758+0000", 2024-09-16T14:49:55.451 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.452 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.452 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.452 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.452 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.985758+0000", 2024-09-16T14:49:55.452 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.452 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.452 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.452 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.452 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.985757+0000", 2024-09-16T14:49:55.452 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:55.452 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.452 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.452 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.453 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986224+0000", 2024-09-16T14:49:55.453 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00046679099999999998 2024-09-16T14:49:55.453 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.453 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.453 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.453 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986234+0000", 2024-09-16T14:49:55.453 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0614999999999999e-05 2024-09-16T14:49:55.453 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.453 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.453 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.453 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986239+0000", 2024-09-16T14:49:55.453 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0000000000000004e-06 2024-09-16T14:49:55.453 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.453 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.454 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.454 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986272+0000", 2024-09-16T14:49:55.454 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2861999999999998e-05 2024-09-16T14:49:55.454 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.454 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.454 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:55.454 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.986274+0000", 2024-09-16T14:49:55.454 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.013e-06 2024-09-16T14:49:55.454 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.454 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.454 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.454 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.981881+0000", 2024-09-16T14:49:55.454 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99560661699999997 2024-09-16T14:49:55.454 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.455 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.455 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:55.455 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.981947+0000", 2024-09-16T14:49:55.455 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6390000000000006e-05 2024-09-16T14:49:55.455 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.455 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.455 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.455 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005200+0000", 2024-09-16T14:49:55.455 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.023253165999999999 2024-09-16T14:49:55.455 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.455 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.455 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.455 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005354+0000", 2024-09-16T14:49:55.455 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000153611 2024-09-16T14:49:55.456 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.456 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.456 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.456 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007291+0000", 2024-09-16T14:49:55.456 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001936716 2024-09-16T14:49:55.456 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.456 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.456 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.456 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009953+0000", 2024-09-16T14:49:55.456 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0026618190000000002 2024-09-16T14:49:55.456 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.456 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.456 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:55.456 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.016469+0000", 2024-09-16T14:49:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0065166629999999998 2024-09-16T14:49:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.016767+0000", 2024-09-16T14:49:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00029760600000000001 2024-09-16T14:49:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:596 3.3s0 3:c9d06351:::smithi04040151-33:head [append 0~557056] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.989693+0000", 2024-09-16T14:49:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 43.424342007999996, 2024-09-16T14:49:55.458 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.027510892, 2024-09-16T14:49:55.458 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.458 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:55.458 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.458 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.458 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.458 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 596 2024-09-16T14:49:55.458 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.458 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.458 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.458 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.458 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.989693+0000", 2024-09-16T14:49:55.458 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.458 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.459 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.459 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.459 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.989693+0000", 2024-09-16T14:49:55.459 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.459 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.459 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.459 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.459 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.989692+0000", 2024-09-16T14:49:55.459 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.459 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.459 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.459 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.459 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990180+0000", 2024-09-16T14:49:55.459 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00048803300000000002 2024-09-16T14:49:55.460 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.460 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.460 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.460 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990189+0000", 2024-09-16T14:49:55.460 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0440000000000001e-06 2024-09-16T14:49:55.460 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.460 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.460 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.460 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990194+0000", 2024-09-16T14:49:55.461 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8369999999999996e-06 2024-09-16T14:49:55.461 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.461 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.461 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.461 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990239+0000", 2024-09-16T14:49:55.461 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4898999999999998e-05 2024-09-16T14:49:55.461 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.461 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.461 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:55.461 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990241+0000", 2024-09-16T14:49:55.462 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9819999999999998e-06 2024-09-16T14:49:55.462 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.462 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.462 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.462 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.981982+0000", 2024-09-16T14:49:55.462 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99174135900000004 2024-09-16T14:49:55.462 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.462 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.462 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:55.462 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982045+0000", 2024-09-16T14:49:55.462 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2890000000000003e-05 2024-09-16T14:49:55.462 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.462 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.007979+0000", 2024-09-16T14:49:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.025933905 2024-09-16T14:49:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.008137+0000", 2024-09-16T14:49:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015834399999999999 2024-09-16T14:49:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.009359+0000", 2024-09-16T14:49:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0012212499999999999 2024-09-16T14:49:55.463 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014320+0000", 2024-09-16T14:49:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049609850000000002 2024-09-16T14:49:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.017003+0000", 2024-09-16T14:49:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002683389 2024-09-16T14:49:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.017204+0000", 2024-09-16T14:49:55.464 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00020068700000000001 2024-09-16T14:49:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:597 3.3s0 3:c9d06351:::smithi04040151-33:head [append 0~16384] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.990199+0000", 2024-09-16T14:49:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 43.423835662999998, 2024-09-16T14:49:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.029187326, 2024-09-16T14:49:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.465 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 597 2024-09-16T14:49:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990199+0000", 2024-09-16T14:49:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990199+0000", 2024-09-16T14:49:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.466 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990199+0000", 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990216+0000", 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7176999999999998e-05 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990220+0000", 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4739999999999999e-06 2024-09-16T14:49:55.467 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.468 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.468 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.468 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990224+0000", 2024-09-16T14:49:55.468 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3799999999999998e-06 2024-09-16T14:49:55.468 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.468 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.468 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.468 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990262+0000", 2024-09-16T14:49:55.468 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8461999999999999e-05 2024-09-16T14:49:55.468 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.468 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.469 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:55.469 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990264+0000", 2024-09-16T14:49:55.469 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6330000000000001e-06 2024-09-16T14:49:55.469 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.469 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.469 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.469 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982110+0000", 2024-09-16T14:49:55.469 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99184635300000001 2024-09-16T14:49:55.469 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.469 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.470 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:55.470 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982190+0000", 2024-09-16T14:49:55.470 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0255999999999999e-05 2024-09-16T14:49:55.470 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.470 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.470 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.470 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.011434+0000", 2024-09-16T14:49:55.470 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.029243331000000001 2024-09-16T14:49:55.470 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.470 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.011595+0000", 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00016157699999999999 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012441+0000", 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00084541900000000003 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.016197+0000", 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003756073 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.471 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.472 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:55.472 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.019019+0000", 2024-09-16T14:49:55.472 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00282168 2024-09-16T14:49:55.472 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.472 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.472 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.472 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.019386+0000", 2024-09-16T14:49:55.472 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00036785099999999999 2024-09-16T14:49:55.472 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.472 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.472 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.472 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.472 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.472 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:598 3.3s0 3:c9d06351:::smithi04040151-33:head [setxattr _header (57) in=7b] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:55.473 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.990235+0000", 2024-09-16T14:49:55.473 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 43.423800102999998, 2024-09-16T14:49:55.473 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.029738257, 2024-09-16T14:49:55.473 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.473 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:55.473 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.473 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.473 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.473 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 598 2024-09-16T14:49:55.473 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.473 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.473 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.473 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.474 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990235+0000", 2024-09-16T14:49:55.474 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.474 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.474 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.474 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.474 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990235+0000", 2024-09-16T14:49:55.474 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.474 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.474 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.474 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.474 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990234+0000", 2024-09-16T14:49:55.474 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.474 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.474 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.475 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.475 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990237+0000", 2024-09-16T14:49:55.475 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3910000000000001e-06 2024-09-16T14:49:55.475 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.475 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.475 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.475 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990240+0000", 2024-09-16T14:49:55.475 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.625e-06 2024-09-16T14:49:55.475 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.475 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.475 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.475 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990243+0000", 2024-09-16T14:49:55.476 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.9000000000000002e-06 2024-09-16T14:49:55.476 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.476 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.476 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.476 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990290+0000", 2024-09-16T14:49:55.476 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7154000000000003e-05 2024-09-16T14:49:55.476 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.476 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.476 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:55.476 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990292+0000", 2024-09-16T14:49:55.477 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.237e-06 2024-09-16T14:49:55.477 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.477 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.477 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.477 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982229+0000", 2024-09-16T14:49:55.477 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.991937069 2024-09-16T14:49:55.477 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.477 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.477 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:55.477 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982316+0000", 2024-09-16T14:49:55.477 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7495000000000002e-05 2024-09-16T14:49:55.477 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.477 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.477 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.478 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012980+0000", 2024-09-16T14:49:55.478 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.030663620999999999 2024-09-16T14:49:55.478 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.478 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.478 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.478 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013119+0000", 2024-09-16T14:49:55.478 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00013920800000000001 2024-09-16T14:49:55.478 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.478 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.478 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.478 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.013756+0000", 2024-09-16T14:49:55.478 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00063710700000000004 2024-09-16T14:49:55.478 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.478 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.016198+0000", 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002442222 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.019618+0000", 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.003420083 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.019973+0000", 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00035448499999999998 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.479 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.480 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.480 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.480 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.480 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:599 3.3s0 3:c9d06351:::smithi04040151-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:55.480 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.990261+0000", 2024-09-16T14:49:55.480 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 43.423773507999996, 2024-09-16T14:49:55.480 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.034782844, 2024-09-16T14:49:55.480 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.480 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:55.480 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.480 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.480 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.480 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 599 2024-09-16T14:49:55.480 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.481 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.481 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.481 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.481 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990261+0000", 2024-09-16T14:49:55.481 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.481 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.481 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.481 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.481 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990261+0000", 2024-09-16T14:49:55.481 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.481 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.481 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.481 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.481 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990261+0000", 2024-09-16T14:49:55.482 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.482 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.482 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.482 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.482 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990263+0000", 2024-09-16T14:49:55.482 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8059999999999999e-06 2024-09-16T14:49:55.482 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.482 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.482 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.482 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990267+0000", 2024-09-16T14:49:55.482 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8850000000000001e-06 2024-09-16T14:49:55.482 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.482 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.482 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.483 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990269+0000", 2024-09-16T14:49:55.483 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7839999999999999e-06 2024-09-16T14:49:55.483 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.483 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.483 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.483 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990311+0000", 2024-09-16T14:49:55.483 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1971000000000002e-05 2024-09-16T14:49:55.483 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.483 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.483 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:55.483 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.990312+0000", 2024-09-16T14:49:55.484 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2e-07 2024-09-16T14:49:55.484 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.484 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.484 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.484 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982653+0000", 2024-09-16T14:49:55.484 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.99234119499999995 2024-09-16T14:49:55.484 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.484 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.484 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for missing object", 2024-09-16T14:49:55.484 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.982733+0000", 2024-09-16T14:49:55.484 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9537999999999996e-05 2024-09-16T14:49:55.484 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.484 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014037+0000", 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.031304580999999998 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014200+0000", 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00016317999999999999 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.020590+0000", 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0063896650000000001 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.485 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.020693+0000", 2024-09-16T14:49:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010270800000000001 2024-09-16T14:49:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.025044+0000", 2024-09-16T14:49:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.004351293 2024-09-16T14:49:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDPGRecoveryDeleteReply(3.3s0 e66,65 3:c09a69fe:::smithi04040151-5:2,35'7)", 2024-09-16T14:49:55.487 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.005793+0000", 2024-09-16T14:49:55.487 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 42.408241476000001, 2024-09-16T14:49:55.487 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0091592329999999993, 2024-09-16T14:49:55.487 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.487 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.487 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.487 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.487 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.487 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005793+0000", 2024-09-16T14:49:55.487 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.487 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.487 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.487 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.487 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005793+0000", 2024-09-16T14:49:55.488 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.488 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.488 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.488 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.488 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005792+0000", 2024-09-16T14:49:55.488 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:55.488 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.488 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.488 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.488 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005800+0000", 2024-09-16T14:49:55.488 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0309999999999998e-06 2024-09-16T14:49:55.488 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.488 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.488 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.489 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005813+0000", 2024-09-16T14:49:55.489 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.287e-05 2024-09-16T14:49:55.489 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.489 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.489 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.489 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005822+0000", 2024-09-16T14:49:55.489 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3920000000000003e-06 2024-09-16T14:49:55.489 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.489 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.489 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.489 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014351+0000", 2024-09-16T14:49:55.489 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0085288429999999995 2024-09-16T14:49:55.489 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.489 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.490 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.490 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.014952+0000", 2024-09-16T14:49:55.490 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00060190500000000002 2024-09-16T14:49:55.490 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.490 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.490 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.490 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.490 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.490 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.7s6 82/27 ECSubWrite(tid=342, reqid=osd.3.0:342, at_version=82'110, trim_to=0'0, roll_forward_to=81'108))", 2024-09-16T14:49:55.490 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.934271+0000", 2024-09-16T14:49:55.490 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 26.479763741999999, 2024-09-16T14:49:55.490 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.009767197, 2024-09-16T14:49:55.490 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.490 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.491 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.491 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.491 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.491 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934271+0000", 2024-09-16T14:49:55.491 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.491 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.491 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.491 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.491 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934271+0000", 2024-09-16T14:49:55.491 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.491 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.491 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.491 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.491 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934271+0000", 2024-09-16T14:49:55.492 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.492 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.492 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.492 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.492 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934296+0000", 2024-09-16T14:49:55.492 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5639e-05 2024-09-16T14:49:55.492 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.492 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.492 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.492 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934306+0000", 2024-09-16T14:49:55.492 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3270000000000007e-06 2024-09-16T14:49:55.492 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.492 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.492 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934313+0000", 2024-09-16T14:49:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9240000000000007e-06 2024-09-16T14:49:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934382+0000", 2024-09-16T14:49:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8567999999999998e-05 2024-09-16T14:49:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934391+0000", 2024-09-16T14:49:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3810000000000005e-06 2024-09-16T14:49:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.493 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.494 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.494 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.944016+0000", 2024-09-16T14:49:55.494 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0096248849999999997 2024-09-16T14:49:55.494 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.494 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.494 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.494 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.944038+0000", 2024-09-16T14:49:55.494 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1866999999999999e-05 2024-09-16T14:49:55.494 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.494 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.494 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.494 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.494 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.494 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs2 82/65 ECSubWrite(tid=343, reqid=osd.3.0:343, at_version=82'67, trim_to=0'0, roll_forward_to=81'65))", 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.934324+0000", 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 26.479710842999999, 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0097367909999999998, 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934324+0000", 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934324+0000", 2024-09-16T14:49:55.495 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.496 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.496 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.496 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.496 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934324+0000", 2024-09-16T14:49:55.496 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.496 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.496 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.496 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.496 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934349+0000", 2024-09-16T14:49:55.496 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5507000000000002e-05 2024-09-16T14:49:55.496 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.496 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.496 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.496 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934357+0000", 2024-09-16T14:49:55.497 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8550000000000002e-06 2024-09-16T14:49:55.497 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.497 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.497 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.497 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934363+0000", 2024-09-16T14:49:55.497 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.1890000000000004e-06 2024-09-16T14:49:55.497 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.497 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.497 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.497 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934406+0000", 2024-09-16T14:49:55.497 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2775000000000003e-05 2024-09-16T14:49:55.497 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.497 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.497 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.498 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934415+0000", 2024-09-16T14:49:55.498 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9190000000000005e-06 2024-09-16T14:49:55.498 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.498 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.498 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.498 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.944047+0000", 2024-09-16T14:49:55.498 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0096324600000000007 2024-09-16T14:49:55.498 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.498 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.498 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.498 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.944061+0000", 2024-09-16T14:49:55.498 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3397000000000001e-05 2024-09-16T14:49:55.498 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.498 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.499 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.499 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.499 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.499 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs2 82/65 ECSubWrite(tid=345, reqid=osd.3.0:345, at_version=82'69, trim_to=0'0, roll_forward_to=81'65))", 2024-09-16T14:49:55.499 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.934374+0000", 2024-09-16T14:49:55.499 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 26.479661021999998, 2024-09-16T14:49:55.499 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012306037000000001, 2024-09-16T14:49:55.499 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.499 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.499 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.499 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.499 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.499 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934374+0000", 2024-09-16T14:49:55.499 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934374+0000", 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934373+0000", 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934395+0000", 2024-09-16T14:49:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1424999999999999e-05 2024-09-16T14:49:55.501 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.501 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.501 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.501 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934403+0000", 2024-09-16T14:49:55.501 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7260000000000002e-06 2024-09-16T14:49:55.501 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.501 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.501 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.501 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934409+0000", 2024-09-16T14:49:55.501 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.2940000000000002e-06 2024-09-16T14:49:55.501 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.501 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.501 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.502 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936858+0000", 2024-09-16T14:49:55.502 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002448895 2024-09-16T14:49:55.502 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.502 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.502 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.502 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936868+0000", 2024-09-16T14:49:55.502 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.044e-05 2024-09-16T14:49:55.502 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.502 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.502 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.502 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.946645+0000", 2024-09-16T14:49:55.502 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0097768009999999999 2024-09-16T14:49:55.502 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.502 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.503 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.503 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.946680+0000", 2024-09-16T14:49:55.503 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.4755999999999997e-05 2024-09-16T14:49:55.503 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.503 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.503 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.503 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.503 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.503 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.7s6 82/27 ECSubWrite(tid=344, reqid=osd.3.0:344, at_version=82'112, trim_to=0'0, roll_forward_to=81'108))", 2024-09-16T14:49:55.503 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.934415+0000", 2024-09-16T14:49:55.503 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 26.479620135000001, 2024-09-16T14:49:55.503 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012309166999999999, 2024-09-16T14:49:55.503 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934415+0000", 2024-09-16T14:49:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934415+0000", 2024-09-16T14:49:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.505 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934414+0000", 2024-09-16T14:49:55.505 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.505 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.505 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.505 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.505 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934430+0000", 2024-09-16T14:49:55.505 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5923999999999999e-05 2024-09-16T14:49:55.505 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.505 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.505 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.505 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934453+0000", 2024-09-16T14:49:55.505 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2855000000000001e-05 2024-09-16T14:49:55.505 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.505 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.506 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.506 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934462+0000", 2024-09-16T14:49:55.506 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8289999999999997e-06 2024-09-16T14:49:55.506 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.506 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.506 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.506 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.936997+0000", 2024-09-16T14:49:55.506 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025347780000000001 2024-09-16T14:49:55.506 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.506 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.508 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.508 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.937007+0000", 2024-09-16T14:49:55.508 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0010000000000001e-05 2024-09-16T14:49:55.508 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.508 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.508 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.508 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.946708+0000", 2024-09-16T14:49:55.508 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0097015279999999992 2024-09-16T14:49:55.508 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.508 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.508 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.509 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.946724+0000", 2024-09-16T14:49:55.509 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5514000000000001e-05 2024-09-16T14:49:55.509 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.509 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.509 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.509 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.509 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.509 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs2 90/65 ECSubWrite(tid=379, reqid=osd.3.0:379, at_version=90'75, trim_to=0'0, roll_forward_to=83'73))", 2024-09-16T14:49:55.509 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.945451+0000", 2024-09-16T14:49:55.509 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.468583486, 2024-09-16T14:49:55.509 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0092287910000000001, 2024-09-16T14:49:55.509 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.509 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.509 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.510 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.510 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.510 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.945451+0000", 2024-09-16T14:49:55.510 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.510 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.510 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.510 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.510 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.945451+0000", 2024-09-16T14:49:55.510 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.510 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.510 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.510 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.510 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.945451+0000", 2024-09-16T14:49:55.510 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.945480+0000", 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.9660999999999999e-05 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.945492+0000", 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1337e-05 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.945503+0000", 2024-09-16T14:49:55.511 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1243999999999999e-05 2024-09-16T14:49:55.512 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.512 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.512 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.512 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.949925+0000", 2024-09-16T14:49:55.512 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044218149999999999 2024-09-16T14:49:55.512 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.512 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.512 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.512 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.949935+0000", 2024-09-16T14:49:55.512 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0329999999999999e-05 2024-09-16T14:49:55.512 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.512 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.512 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.512 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.954651+0000", 2024-09-16T14:49:55.513 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0047154670000000001 2024-09-16T14:49:55.513 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.513 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.513 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.513 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.954680+0000", 2024-09-16T14:49:55.513 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.9374999999999999e-05 2024-09-16T14:49:55.513 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.513 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.513 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.513 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.513 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.513 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs2 90/65 ECSubWrite(tid=380, reqid=osd.3.0:380, at_version=90'77, trim_to=0'0, roll_forward_to=83'73))", 2024-09-16T14:49:55.513 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.947706+0000", 2024-09-16T14:49:55.513 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.466328939, 2024-09-16T14:49:55.514 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011257764, 2024-09-16T14:49:55.514 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.514 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.514 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.514 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.514 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.514 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947706+0000", 2024-09-16T14:49:55.514 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.514 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.514 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.514 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.514 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947706+0000", 2024-09-16T14:49:55.514 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.514 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947705+0000", 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947732+0000", 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6228999999999999e-05 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947742+0000", 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0543000000000001e-05 2024-09-16T14:49:55.515 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.516 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.516 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.516 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947751+0000", 2024-09-16T14:49:55.516 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4079999999999998e-06 2024-09-16T14:49:55.516 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.516 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.516 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.516 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.950263+0000", 2024-09-16T14:49:55.516 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025118559999999998 2024-09-16T14:49:55.516 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.516 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.516 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.516 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.950271+0000", 2024-09-16T14:49:55.516 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7649999999999999e-06 2024-09-16T14:49:55.517 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.517 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.517 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.517 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958938+0000", 2024-09-16T14:49:55.517 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0086666590000000002 2024-09-16T14:49:55.517 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.517 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.517 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.517 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958964+0000", 2024-09-16T14:49:55.517 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5624999999999999e-05 2024-09-16T14:49:55.517 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.517 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.517 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.517 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.518 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.518 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.19s6 91/90 ECSubWrite(tid=102, reqid=client.4500.0:822, at_version=91'77, trim_to=0'0, roll_forward_to=91'75))", 2024-09-16T14:49:55.518 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:38.005940+0000", 2024-09-16T14:49:55.518 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 17.408094668, 2024-09-16T14:49:55.518 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0092427350000000002, 2024-09-16T14:49:55.518 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.518 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.518 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.518 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.518 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.518 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:38.005940+0000", 2024-09-16T14:49:55.518 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.518 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.518 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:38.005940+0000", 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:38.005939+0000", 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:38.009598+0000", 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0036592220000000002 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.519 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.520 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.520 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:38.009614+0000", 2024-09-16T14:49:55.520 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5209e-05 2024-09-16T14:49:55.520 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.520 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.520 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.520 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:38.009636+0000", 2024-09-16T14:49:55.520 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2167999999999998e-05 2024-09-16T14:49:55.520 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.520 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.520 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.520 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:38.009736+0000", 2024-09-16T14:49:55.520 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000100598 2024-09-16T14:49:55.520 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.521 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.521 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.521 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:38.009747+0000", 2024-09-16T14:49:55.521 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0346999999999999e-05 2024-09-16T14:49:55.521 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.521 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.521 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.521 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:38.015160+0000", 2024-09-16T14:49:55.521 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0054136710000000001 2024-09-16T14:49:55.521 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.521 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.521 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.521 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:38.015183+0000", 2024-09-16T14:49:55.521 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2325000000000001e-05 2024-09-16T14:49:55.522 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.522 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.522 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.522 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.522 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.522 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:55.523 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T14:49:55.524 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:55.522+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:55.624 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:55.623+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:55.625 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:55.625 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:55.625 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:55.625 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:55.634 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T14:49:55.724 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-16T14:49:55.723+0000 7f18ac34a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32907) UID: 0 2024-09-16T14:49:55.735 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:55.736 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:55.736 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:55.736 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:55.736 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:55.744 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T14:49:55.825 INFO:tasks.ceph.osd.5.smithi111.stderr:2024-09-16T14:49:55.825+0000 7fe1eff3a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32771) UID: 0 2024-09-16T14:49:55.849 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:55.850 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:55.850 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:55.850 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:55.850 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.850 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:449 3.5s0 3:a7111aa9:::smithi04040151-45:head [stat,delete] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:55.850 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:55.850 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 57.971386553999999, 2024-09-16T14:49:55.850 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013571421, 2024-09-16T14:49:55.850 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.850 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:55.850 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.851 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.851 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.851 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 449 2024-09-16T14:49:55.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.851 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.851 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.851 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.851 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:55.851 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.851 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.851 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.851 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876386+0000", 2024-09-16T14:49:55.851 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.852 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.852 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.852 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876385+0000", 2024-09-16T14:49:55.852 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:55.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.852 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.852 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.852 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876396+0000", 2024-09-16T14:49:55.852 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1094000000000001e-05 2024-09-16T14:49:55.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.852 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.852 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876412+0000", 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6563e-05 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876421+0000", 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5189999999999992e-06 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876477+0000", 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.6679999999999999e-05 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.853 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:55.854 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.876487+0000", 2024-09-16T14:49:55.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4509999999999998e-06 2024-09-16T14:49:55.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.854 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.854 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.878486+0000", 2024-09-16T14:49:55.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001999635 2024-09-16T14:49:55.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.854 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.854 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.854 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.878592+0000", 2024-09-16T14:49:55.854 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000105398 2024-09-16T14:49:55.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.855 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.855 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.855 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.889958+0000", 2024-09-16T14:49:55.855 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01136595 2024-09-16T14:49:55.855 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.855 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.855 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.855 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.855 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.855 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:467 3.5s0 3:a29e72cc:::smithi04040151-39:head [rollback 14] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:55.855 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:55.855 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 57.852193995999997, 2024-09-16T14:49:55.855 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012554012, 2024-09-16T14:49:55.855 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 467 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.856 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:55.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.857 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.857 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995579+0000", 2024-09-16T14:49:55.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.857 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.857 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995581+0000", 2024-09-16T14:49:55.857 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.9009999999999998e-06 2024-09-16T14:49:55.857 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.857 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.858 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.858 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995587+0000", 2024-09-16T14:49:55.858 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8850000000000004e-06 2024-09-16T14:49:55.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.858 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.858 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.858 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995592+0000", 2024-09-16T14:49:55.858 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9749999999999998e-06 2024-09-16T14:49:55.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.858 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.858 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.858 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.996914+0000", 2024-09-16T14:49:55.858 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00132168 2024-09-16T14:49:55.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.859 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.859 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.859 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997029+0000", 2024-09-16T14:49:55.859 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000115203 2024-09-16T14:49:55.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.859 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.859 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.859 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.997570+0000", 2024-09-16T14:49:55.859 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00054107999999999999 2024-09-16T14:49:55.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.859 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.859 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.859 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.003265+0000", 2024-09-16T14:49:55.859 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0056945679999999997 2024-09-16T14:49:55.860 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.860 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.860 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:55.860 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008067+0000", 2024-09-16T14:49:55.860 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048018039999999998 2024-09-16T14:49:55.860 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.860 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.860 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.860 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008133+0000", 2024-09-16T14:49:55.860 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6365999999999994e-05 2024-09-16T14:49:55.860 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.860 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.860 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.861 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.861 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:468 3.5s0 3:a29e72cc:::smithi04040151-39:head [append 0~0] snapc 15=[15,14,13,10,c] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:55.861 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:55.861 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 57.852024935999999, 2024-09-16T14:49:55.861 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012562663999999999, 2024-09-16T14:49:55.861 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.861 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:55.861 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.861 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.861 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 468 2024-09-16T14:49:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995748+0000", 2024-09-16T14:49:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995747+0000", 2024-09-16T14:49:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995752+0000", 2024-09-16T14:49:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.916e-06 2024-09-16T14:49:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995762+0000", 2024-09-16T14:49:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4609999999999997e-06 2024-09-16T14:49:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.995768+0000", 2024-09-16T14:49:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8109999999999997e-06 2024-09-16T14:49:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.998577+0000", 2024-09-16T14:49:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0028088369999999998 2024-09-16T14:49:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.998680+0000", 2024-09-16T14:49:55.865 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010318399999999999 2024-09-16T14:49:55.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.865 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.865 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:57.999182+0000", 2024-09-16T14:49:55.865 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00050101099999999999 2024-09-16T14:49:55.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.865 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.005964+0000", 2024-09-16T14:49:55.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0067821490000000003 2024-09-16T14:49:55.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.866 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:55.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008246+0000", 2024-09-16T14:49:55.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0022827670000000002 2024-09-16T14:49:55.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.866 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.866 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.866 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:58.008311+0000", 2024-09-16T14:49:55.866 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.4269000000000004e-05 2024-09-16T14:49:55.867 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.867 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.867 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.867 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.8s5 61/27 ECSubWrite(tid=224, reqid=osd.11.0:224, at_version=61'80, trim_to=0'0, roll_forward_to=60'78))", 2024-09-16T14:49:55.867 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932515+0000", 2024-09-16T14:49:55.867 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 47.915258203, 2024-09-16T14:49:55.867 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010770217, 2024-09-16T14:49:55.867 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.867 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.867 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.868 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.868 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932515+0000", 2024-09-16T14:49:55.868 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.868 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.868 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.868 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932515+0000", 2024-09-16T14:49:55.868 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.868 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.868 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.868 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.868 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932514+0000", 2024-09-16T14:49:55.868 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.869 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.869 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932539+0000", 2024-09-16T14:49:55.869 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4494e-05 2024-09-16T14:49:55.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.869 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.869 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932549+0000", 2024-09-16T14:49:55.869 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0028e-05 2024-09-16T14:49:55.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.869 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.870 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.870 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932556+0000", 2024-09-16T14:49:55.870 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.6920000000000002e-06 2024-09-16T14:49:55.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.870 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.870 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.870 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932618+0000", 2024-09-16T14:49:55.870 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.1600000000000007e-05 2024-09-16T14:49:55.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.870 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.871 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.871 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932626+0000", 2024-09-16T14:49:55.871 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4139999999999994e-06 2024-09-16T14:49:55.871 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.871 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.871 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943268+0000", 2024-09-16T14:49:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010641695 2024-09-16T14:49:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943285+0000", 2024-09-16T14:49:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6929e-05 2024-09-16T14:49:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds1 61/27 ECSubWrite(tid=234, reqid=osd.7.0:234, at_version=61'48, trim_to=0'0, roll_forward_to=60'46))", 2024-09-16T14:49:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:55.873 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 47.915005522999998, 2024-09-16T14:49:55.873 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010600091000000001, 2024-09-16T14:49:55.873 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.873 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.873 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.873 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.873 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.873 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:55.873 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.873 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.873 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.873 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:55.873 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932767+0000", 2024-09-16T14:49:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932785+0000", 2024-09-16T14:49:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7829000000000002e-05 2024-09-16T14:49:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932794+0000", 2024-09-16T14:49:55.875 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5179999999999993e-06 2024-09-16T14:49:55.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.875 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.875 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.875 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932801+0000", 2024-09-16T14:49:55.875 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.793e-06 2024-09-16T14:49:55.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.875 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.875 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.875 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933189+0000", 2024-09-16T14:49:55.875 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00038759100000000001 2024-09-16T14:49:55.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.875 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.875 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933197+0000", 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.5290000000000008e-06 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943351+0000", 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010153673 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943368+0000", 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6538999999999999e-05 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.876 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.877 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.877 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.877 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/27 ECSubWrite(tid=63, reqid=osd.1.0:63, at_version=61'50, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:55.877 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:55.877 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 47.914952788000001, 2024-09-16T14:49:55.877 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010214164, 2024-09-16T14:49:55.877 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.877 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.877 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.877 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.877 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.877 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:55.878 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.878 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.878 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.878 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.878 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:55.878 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.878 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.878 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.878 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.878 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932820+0000", 2024-09-16T14:49:55.878 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.878 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932838+0000", 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8576e-05 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932847+0000", 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6519999999999995e-06 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932855+0000", 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7600000000000002e-06 2024-09-16T14:49:55.879 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.880 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.880 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.880 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932929+0000", 2024-09-16T14:49:55.880 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4134999999999994e-05 2024-09-16T14:49:55.880 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.880 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.880 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.880 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932937+0000", 2024-09-16T14:49:55.880 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.1519999999999992e-06 2024-09-16T14:49:55.880 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.880 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.880 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.881 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941947+0000", 2024-09-16T14:49:55.881 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0090100430000000006 2024-09-16T14:49:55.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.881 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.881 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.881 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943034+0000", 2024-09-16T14:49:55.881 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001087111 2024-09-16T14:49:55.881 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.881 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.881 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.881 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.881 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.as2 61/53 ECSubWrite(tid=225, reqid=osd.11.0:225, at_version=61'26, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 47.914858801000001, 2024-09-16T14:49:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010782797, 2024-09-16T14:49:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932914+0000", 2024-09-16T14:49:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932954+0000", 2024-09-16T14:49:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0166000000000001e-05 2024-09-16T14:49:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.883 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932980+0000", 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5843e-05 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932987+0000", 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0079999999999996e-06 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933034+0000", 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6934999999999997e-05 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.884 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933042+0000", 2024-09-16T14:49:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8429999999999999e-06 2024-09-16T14:49:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943652+0000", 2024-09-16T14:49:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010609288999999999 2024-09-16T14:49:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943697+0000", 2024-09-16T14:49:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5296999999999997e-05 2024-09-16T14:49:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.885 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.886 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.886 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.886 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.886 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.4s3 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:55.886 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:55.886 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 47.914808716000003, 2024-09-16T14:49:55.886 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010665579999999999, 2024-09-16T14:49:55.886 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.886 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.886 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.886 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.886 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.886 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:55.886 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932964+0000", 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.932995+0000", 2024-09-16T14:49:55.887 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1084000000000003e-05 2024-09-16T14:49:55.888 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.888 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.888 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.888 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933003+0000", 2024-09-16T14:49:55.888 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2199999999999992e-06 2024-09-16T14:49:55.888 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.888 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.888 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.888 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933010+0000", 2024-09-16T14:49:55.888 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.4359999999999999e-06 2024-09-16T14:49:55.888 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.888 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935460+0000", 2024-09-16T14:49:55.889 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0024503350000000001 2024-09-16T14:49:55.889 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.889 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.935469+0000", 2024-09-16T14:49:55.889 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.602e-06 2024-09-16T14:49:55.889 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.889 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.889 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.889 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943610+0000", 2024-09-16T14:49:55.889 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0081417390000000003 2024-09-16T14:49:55.889 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.943630+0000", 2024-09-16T14:49:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9584000000000001e-05 2024-09-16T14:49:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:579 3.5s0 3:a29e72cc:::smithi04040151-39:head [read 0~671744,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.946788+0000", 2024-09-16T14:49:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 43.900985220000003, 2024-09-16T14:49:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.037038522, 2024-09-16T14:49:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.891 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:55.891 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.891 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.891 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.891 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 579 2024-09-16T14:49:55.891 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.891 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.891 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.891 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.891 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946788+0000", 2024-09-16T14:49:55.891 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.891 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.891 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.891 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946788+0000", 2024-09-16T14:49:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946786+0000", 2024-09-16T14:49:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946795+0000", 2024-09-16T14:49:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9549999999999998e-06 2024-09-16T14:49:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946806+0000", 2024-09-16T14:49:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1174e-05 2024-09-16T14:49:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946816+0000", 2024-09-16T14:49:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.1949999999999992e-06 2024-09-16T14:49:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.947936+0000", 2024-09-16T14:49:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011208679999999999 2024-09-16T14:49:55.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.894 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.894 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:55.894 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.947940+0000", 2024-09-16T14:49:55.894 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.365e-06 2024-09-16T14:49:55.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.894 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.894 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.894 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.976737+0000", 2024-09-16T14:49:55.894 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.028796684 2024-09-16T14:49:55.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.894 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.894 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.894 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.977002+0000", 2024-09-16T14:49:55.895 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00026509800000000002 2024-09-16T14:49:55.895 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.895 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.895 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.895 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.983826+0000", 2024-09-16T14:49:55.895 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0068246660000000001 2024-09-16T14:49:55.895 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.895 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.895 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.895 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.895 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.895 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:580 3.5s0 3:a29e72cc:::smithi04040151-39:head [read 0~671744,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:55.895 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.946829+0000", 2024-09-16T14:49:55.895 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 43.900944252000002, 2024-09-16T14:49:55.896 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0381929240000001, 2024-09-16T14:49:55.896 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.896 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:55.896 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.896 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.896 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.896 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 580 2024-09-16T14:49:55.896 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.896 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.896 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.896 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.896 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946829+0000", 2024-09-16T14:49:55.896 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.896 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.897 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.897 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.897 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946829+0000", 2024-09-16T14:49:55.897 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.897 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.897 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.897 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.897 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946828+0000", 2024-09-16T14:49:55.897 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.897 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.897 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.897 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.897 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946832+0000", 2024-09-16T14:49:55.897 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9009999999999996e-06 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946839+0000", 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9619999999999996e-06 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946846+0000", 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7789999999999998e-06 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.948182+0000", 2024-09-16T14:49:55.898 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0013367609999999999 2024-09-16T14:49:55.899 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.899 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.899 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:55.899 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.948185+0000", 2024-09-16T14:49:55.899 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.357e-06 2024-09-16T14:49:55.899 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.899 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.899 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.899 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.979390+0000", 2024-09-16T14:49:55.899 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0312049299999999 2024-09-16T14:49:55.899 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.899 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.899 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.899 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.979672+0000", 2024-09-16T14:49:55.900 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00028239299999999998 2024-09-16T14:49:55.900 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.900 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.900 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.900 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.985022+0000", 2024-09-16T14:49:55.900 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053494620000000001 2024-09-16T14:49:55.900 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.900 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.900 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.900 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.900 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.900 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:581 3.5s0 3:a29e72cc:::smithi04040151-39:head [read 0~671744,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:55.900 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.946851+0000", 2024-09-16T14:49:55.901 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 43.900921953000001, 2024-09-16T14:49:55.901 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0393245289999999, 2024-09-16T14:49:55.901 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.901 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:55.901 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.901 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.901 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.901 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 581 2024-09-16T14:49:55.901 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.901 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.901 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.901 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.901 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946851+0000", 2024-09-16T14:49:55.901 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.902 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.902 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.902 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.902 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946851+0000", 2024-09-16T14:49:55.902 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.902 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.902 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.902 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.902 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946851+0000", 2024-09-16T14:49:55.902 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.902 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.902 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.902 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.902 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946854+0000", 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.937e-06 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946860+0000", 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.1120000000000001e-06 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.946864+0000", 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.634e-06 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.948218+0000", 2024-09-16T14:49:55.904 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0013536209999999999 2024-09-16T14:49:55.904 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.904 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.904 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:55.904 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.948220+0000", 2024-09-16T14:49:55.904 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.866e-06 2024-09-16T14:49:55.904 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.904 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.904 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.904 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980141+0000", 2024-09-16T14:49:55.904 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0319210940000001 2024-09-16T14:49:55.904 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.904 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.904 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.905 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980309+0000", 2024-09-16T14:49:55.905 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00016819599999999999 2024-09-16T14:49:55.905 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.905 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.905 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.905 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.986176+0000", 2024-09-16T14:49:55.905 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0058664870000000001 2024-09-16T14:49:55.905 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.905 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.905 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.905 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.905 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.905 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:690 3.cs0 3:3bbed9f6:::smithi04040151-38:head [append 0~720896] snapc 2e=[2e,2b,29,28,25] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-16T14:49:55.905 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:21.919040+0000", 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 33.928732564999997, 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012801389999999999, 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 690 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.919040+0000", 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.906 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.907 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.907 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.907 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.919040+0000", 2024-09-16T14:49:55.907 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.907 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.907 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.907 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.907 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.919038+0000", 2024-09-16T14:49:55.907 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:55.907 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.907 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.907 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.907 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.920351+0000", 2024-09-16T14:49:55.908 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001312913 2024-09-16T14:49:55.908 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.908 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.908 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.908 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.920366+0000", 2024-09-16T14:49:55.908 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4728e-05 2024-09-16T14:49:55.908 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.908 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.908 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.908 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.920389+0000", 2024-09-16T14:49:55.908 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3456999999999999e-05 2024-09-16T14:49:55.908 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.908 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.909 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.909 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.920466+0000", 2024-09-16T14:49:55.909 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7377000000000001e-05 2024-09-16T14:49:55.909 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.909 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.909 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.909 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.920545+0000", 2024-09-16T14:49:55.909 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8739000000000006e-05 2024-09-16T14:49:55.909 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.909 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.910 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.910 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.921681+0000", 2024-09-16T14:49:55.910 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011363079999999999 2024-09-16T14:49:55.910 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.910 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.910 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.910 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.922719+0000", 2024-09-16T14:49:55.910 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0010372249999999999 2024-09-16T14:49:55.910 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.910 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.910 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:55.910 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.931783+0000", 2024-09-16T14:49:55.910 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0090647539999999995 2024-09-16T14:49:55.911 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.911 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.911 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.911 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.931842+0000", 2024-09-16T14:49:55.911 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.8369000000000003e-05 2024-09-16T14:49:55.911 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.911 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.911 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.911 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.911 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.911 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:694 3.cs0 3:3bbed9f6:::smithi04040151-38:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-09-16T14:49:55.911 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:21.927339+0000", 2024-09-16T14:49:55.911 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 33.920433926000001, 2024-09-16T14:49:55.911 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089489630000000008, 2024-09-16T14:49:55.912 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.912 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:55.912 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.912 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.912 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.912 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 694 2024-09-16T14:49:55.912 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.912 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.912 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.912 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.912 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.927339+0000", 2024-09-16T14:49:55.912 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.912 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.912 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.913 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.913 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.927339+0000", 2024-09-16T14:49:55.913 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.913 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.913 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.913 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.913 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.927339+0000", 2024-09-16T14:49:55.913 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.913 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.913 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.913 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.913 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.927341+0000", 2024-09-16T14:49:55.916 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.894e-06 2024-09-16T14:49:55.916 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.916 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.916 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.916 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.927344+0000", 2024-09-16T14:49:55.916 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.7419999999999999e-06 2024-09-16T14:49:55.916 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.916 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.916 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.916 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.927353+0000", 2024-09-16T14:49:55.917 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3489999999999992e-06 2024-09-16T14:49:55.917 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.917 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.917 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.917 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.929608+0000", 2024-09-16T14:49:55.917 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0022548749999999999 2024-09-16T14:49:55.917 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.917 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.917 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:55.917 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.929682+0000", 2024-09-16T14:49:55.917 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.4530000000000006e-05 2024-09-16T14:49:55.917 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.917 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.917 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.918 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.933769+0000", 2024-09-16T14:49:55.918 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0040868349999999996 2024-09-16T14:49:55.918 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.918 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.918 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.918 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.933819+0000", 2024-09-16T14:49:55.918 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0268000000000001e-05 2024-09-16T14:49:55.918 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.918 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.918 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.918 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:21.936288+0000", 2024-09-16T14:49:55.918 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0024688150000000001 2024-09-16T14:49:55.918 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.918 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:760 3.5s0 3:a29e72cc:::smithi04040151-39:head [append 0~0] snapc 31=[2e,28] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.926013+0000", 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 26.921760002999999, 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.032715963000000001, 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 760 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.919 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926013+0000", 2024-09-16T14:49:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926013+0000", 2024-09-16T14:49:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926011+0000", 2024-09-16T14:49:55.920 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926020+0000", 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6729999999999998e-06 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926033+0000", 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3030000000000001e-05 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926041+0000", 2024-09-16T14:49:55.921 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.0120000000000005e-06 2024-09-16T14:49:55.922 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.922 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.922 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.922 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928219+0000", 2024-09-16T14:49:55.922 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0021786000000000002 2024-09-16T14:49:55.922 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.922 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.922 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:55.922 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928222+0000", 2024-09-16T14:49:55.922 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.943e-06 2024-09-16T14:49:55.922 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.922 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.922 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.922 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.942660+0000", 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014438124 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.943382+0000", 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00072144099999999997 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.955102+0000", 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011719985 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.923 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.955185+0000", 2024-09-16T14:49:55.924 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3676000000000003e-05 2024-09-16T14:49:55.924 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.924 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.924 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.924 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.955753+0000", 2024-09-16T14:49:55.924 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00056795200000000004 2024-09-16T14:49:55.924 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.924 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.924 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.924 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.958254+0000", 2024-09-16T14:49:55.924 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025006130000000001 2024-09-16T14:49:55.924 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.924 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.924 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:55.925 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.958435+0000", 2024-09-16T14:49:55.925 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00018093800000000001 2024-09-16T14:49:55.925 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.925 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.925 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.925 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.958729+0000", 2024-09-16T14:49:55.925 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00029398599999999999 2024-09-16T14:49:55.925 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.925 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.925 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.925 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.925 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.925 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:761 3.5s0 3:a29e72cc:::smithi04040151-39:head [rollback 2e] snapc 31=[2e,28] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:49:55.925 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.926050+0000", 2024-09-16T14:49:55.926 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 26.921722608, 2024-09-16T14:49:55.926 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.037148401999999997, 2024-09-16T14:49:55.926 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.926 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:55.926 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.926 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.926 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.926 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 761 2024-09-16T14:49:55.926 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.926 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.926 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.926 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.926 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926050+0000", 2024-09-16T14:49:55.926 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.927 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.927 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.927 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.927 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926050+0000", 2024-09-16T14:49:55.927 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.927 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.927 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.927 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.927 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926050+0000", 2024-09-16T14:49:55.927 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.927 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.927 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.927 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.927 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926053+0000", 2024-09-16T14:49:55.928 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0350000000000002e-06 2024-09-16T14:49:55.928 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.928 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.928 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.928 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926060+0000", 2024-09-16T14:49:55.928 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0790000000000003e-06 2024-09-16T14:49:55.928 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.928 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.928 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.928 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926065+0000", 2024-09-16T14:49:55.928 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.3650000000000003e-06 2024-09-16T14:49:55.928 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.928 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.929 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.929 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928321+0000", 2024-09-16T14:49:55.929 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0022560089999999998 2024-09-16T14:49:55.929 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.929 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.929 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:55.929 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928323+0000", 2024-09-16T14:49:55.929 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2580000000000001e-06 2024-09-16T14:49:55.929 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.929 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.929 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.929 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.944730+0000", 2024-09-16T14:49:55.929 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.016407234999999999 2024-09-16T14:49:55.930 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.930 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.930 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:55.930 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.944848+0000", 2024-09-16T14:49:55.930 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000118245 2024-09-16T14:49:55.930 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.930 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.930 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.930 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.956932+0000", 2024-09-16T14:49:55.930 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012083943 2024-09-16T14:49:55.930 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.930 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.930 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.930 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.957039+0000", 2024-09-16T14:49:55.931 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000107112 2024-09-16T14:49:55.931 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.931 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.931 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.931 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.957584+0000", 2024-09-16T14:49:55.931 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00054457099999999999 2024-09-16T14:49:55.931 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.931 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.931 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.931 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.960280+0000", 2024-09-16T14:49:55.931 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002696585 2024-09-16T14:49:55.931 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.931 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.931 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:55.932 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.962848+0000", 2024-09-16T14:49:55.932 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025671560000000001 2024-09-16T14:49:55.932 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.932 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.932 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.932 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.963199+0000", 2024-09-16T14:49:55.932 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00035125400000000001 2024-09-16T14:49:55.932 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.932 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.932 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.932 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.932 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.932 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:762 3.5s0 3:a29e72cc:::smithi04040151-39:head [append 0~0] snapc 31=[2e,28] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:49:55.932 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.926070+0000", 2024-09-16T14:49:55.933 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 26.921702633999999, 2024-09-16T14:49:55.933 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.037652156999999999, 2024-09-16T14:49:55.933 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.933 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:55.933 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.933 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.933 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.933 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 762 2024-09-16T14:49:55.933 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.933 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.933 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.933 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.933 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926070+0000", 2024-09-16T14:49:55.933 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926070+0000", 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926070+0000", 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926076+0000", 2024-09-16T14:49:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9100000000000002e-06 2024-09-16T14:49:55.935 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.935 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.935 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.935 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926082+0000", 2024-09-16T14:49:55.935 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.489e-06 2024-09-16T14:49:55.935 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.935 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.935 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.935 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.926087+0000", 2024-09-16T14:49:55.935 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.3900000000000003e-06 2024-09-16T14:49:55.935 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.935 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.935 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.935 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928926+0000", 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0028388900000000002 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.928928+0000", 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.221e-06 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.944921+0000", 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015993395000000001 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:55.936 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.945026+0000", 2024-09-16T14:49:55.937 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000104676 2024-09-16T14:49:55.937 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.937 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.937 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.937 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.958770+0000", 2024-09-16T14:49:55.937 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013743663999999999 2024-09-16T14:49:55.937 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.937 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.937 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.937 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.958841+0000", 2024-09-16T14:49:55.937 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.1088999999999996e-05 2024-09-16T14:49:55.937 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.937 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.937 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.938 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.959195+0000", 2024-09-16T14:49:55.938 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00035468199999999999 2024-09-16T14:49:55.938 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.938 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.938 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.938 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.963255+0000", 2024-09-16T14:49:55.938 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0040595680000000004 2024-09-16T14:49:55.938 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.938 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.938 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:55.938 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.963371+0000", 2024-09-16T14:49:55.938 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00011607099999999999 2024-09-16T14:49:55.938 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.938 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.939 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.939 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.963722+0000", 2024-09-16T14:49:55.939 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000351498 2024-09-16T14:49:55.939 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.939 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.939 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.939 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.939 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.939 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:787 3.cs0 3:31a6e1c2:::smithi04040151-23:head [append 0~507904] snapc 39=[39,38,37,28] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T14:49:55.939 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:32.874333+0000", 2024-09-16T14:49:55.939 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 22.973440191999998, 2024-09-16T14:49:55.939 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0086053910000000004, 2024-09-16T14:49:55.939 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.940 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:55.940 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:55.940 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:55.940 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:55.940 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 787 2024-09-16T14:49:55.940 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.940 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.940 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.940 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.940 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.874333+0000", 2024-09-16T14:49:55.940 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.940 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.940 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.940 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.941 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.874333+0000", 2024-09-16T14:49:55.941 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.941 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.941 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.941 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.941 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.874332+0000", 2024-09-16T14:49:55.941 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:55.941 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.941 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.941 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.941 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.874978+0000", 2024-09-16T14:49:55.941 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00064625499999999998 2024-09-16T14:49:55.941 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.874991+0000", 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3125999999999999e-05 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.874997+0000", 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5999999999999997e-06 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.875342+0000", 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00034574399999999999 2024-09-16T14:49:55.942 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.943 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.943 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:55.943 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.875435+0000", 2024-09-16T14:49:55.943 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2326999999999999e-05 2024-09-16T14:49:55.943 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.943 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.943 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.943 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.876197+0000", 2024-09-16T14:49:55.943 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00076206500000000003 2024-09-16T14:49:55.943 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.943 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.944 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.944 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.876979+0000", 2024-09-16T14:49:55.944 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00078201699999999998 2024-09-16T14:49:55.944 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.944 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.944 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:55.944 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.882865+0000", 2024-09-16T14:49:55.944 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0058867609999999999 2024-09-16T14:49:55.944 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.944 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.944 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.944 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.882938+0000", 2024-09-16T14:49:55.944 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2671999999999998e-05 2024-09-16T14:49:55.945 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.945 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.945 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.945 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.945 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.945 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 89/65 ECSubWriteReply(tid=189, last_complete=90'73, committed=1, applied=1))", 2024-09-16T14:49:55.945 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.946058+0000", 2024-09-16T14:49:55.945 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.901714629000001, 2024-09-16T14:49:55.945 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011478993999999999, 2024-09-16T14:49:55.945 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.945 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.945 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.945 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.945 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946058+0000", 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946058+0000", 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946057+0000", 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.946 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.947 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946063+0000", 2024-09-16T14:49:55.947 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.0979999999999999e-06 2024-09-16T14:49:55.947 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.947 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.947 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.947 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946076+0000", 2024-09-16T14:49:55.947 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2495e-05 2024-09-16T14:49:55.947 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.947 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.947 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.947 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946085+0000", 2024-09-16T14:49:55.947 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6500000000000008e-06 2024-09-16T14:49:55.947 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.947 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.948 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.948 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957431+0000", 2024-09-16T14:49:55.948 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011346096999999999 2024-09-16T14:49:55.948 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.948 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.948 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.948 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957537+0000", 2024-09-16T14:49:55.948 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000105917 2024-09-16T14:49:55.948 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.948 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWriteReply(3.cs0 89/65 ECSubWriteReply(tid=189, last_complete=90'73, committed=1, applied=1))", 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.946453+0000", 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.901320444, 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011123697, 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946453+0000", 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.949 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.950 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.950 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.950 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946453+0000", 2024-09-16T14:49:55.950 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.950 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.950 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.950 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.950 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946452+0000", 2024-09-16T14:49:55.950 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:55.950 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.950 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.950 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.950 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946458+0000", 2024-09-16T14:49:55.950 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7009999999999999e-06 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946471+0000", 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2968e-05 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.946481+0000", 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3370000000000006e-06 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957563+0000", 2024-09-16T14:49:55.951 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011082949999999999 2024-09-16T14:49:55.952 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.952 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.952 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.952 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957576+0000", 2024-09-16T14:49:55.952 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2758000000000001e-05 2024-09-16T14:49:55.952 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.952 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.952 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.952 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.952 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.952 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds1 90/65 ECSubWrite(tid=311, reqid=osd.7.0:311, at_version=90'75, trim_to=0'0, roll_forward_to=89'73))", 2024-09-16T14:49:55.952 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.957718+0000", 2024-09-16T14:49:55.952 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 18.890054735, 2024-09-16T14:49:55.952 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010726197, 2024-09-16T14:49:55.953 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:55.953 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:55.953 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:55.953 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.953 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:55.953 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957718+0000", 2024-09-16T14:49:55.953 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.953 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.953 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.953 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:55.953 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957718+0000", 2024-09-16T14:49:55.953 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:55.953 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.953 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957718+0000", 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957737+0000", 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8757999999999999e-05 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957746+0000", 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0129999999999999e-06 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.955 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:55.955 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957753+0000", 2024-09-16T14:49:55.955 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9809999999999997e-06 2024-09-16T14:49:55.955 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.955 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.955 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:55.955 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957967+0000", 2024-09-16T14:49:55.955 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000214026 2024-09-16T14:49:55.955 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.955 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.955 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:55.955 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957976+0000", 2024-09-16T14:49:55.955 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3249999999999997e-06 2024-09-16T14:49:55.955 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.968424+0000", 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01044819 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.968444+0000", 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0103999999999999e-05 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:55.956 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:55.958 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T14:49:55.958 INFO:tasks.ceph.osd.6.smithi111.stderr:2024-09-16T14:49:55.957+0000 7fcde87d0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 42447) UID: 0 2024-09-16T14:49:56.040 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'unique_pool_0' removed 2024-09-16T14:49:56.040 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is set 2024-09-16T14:49:56.058 INFO:tasks.ceph.osd.9.smithi114.stderr:2024-09-16T14:49:56.057+0000 7fda965a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 33113) UID: 0 2024-09-16T14:49:56.064 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-16T14:49:56.073 INFO:tasks.thrashosds:joining thrashosds 2024-09-16T14:49:56.109 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:56.109 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:56.109 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:56.109 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:56.118 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T14:49:56.219 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:56.220 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:56.220 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:56.220 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:56.220 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:56.228 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T14:49:56.332 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:56.333 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:56.333 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:56.333 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:56.333 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.333 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4225.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-16T14:49:56.333 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:56.333 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 94.191038516000006, 2024-09-16T14:49:56.333 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.223181782, 2024-09-16T14:49:56.333 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.333 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:56.334 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.334 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4225", 2024-09-16T14:49:56.334 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.111:0/1404405087", 2024-09-16T14:49:56.334 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 1 2024-09-16T14:49:56.334 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.334 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.334 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.334 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.334 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:56.334 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.334 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.334 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.334 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.335 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139497+0000", 2024-09-16T14:49:56.335 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.335 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.335 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.335 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.335 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139495+0000", 2024-09-16T14:49:56.335 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:56.335 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.335 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.335 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.335 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139504+0000", 2024-09-16T14:49:56.335 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6119999999999999e-06 2024-09-16T14:49:56.335 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.335 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.336 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.336 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139514+0000", 2024-09-16T14:49:56.336 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8889999999999997e-06 2024-09-16T14:49:56.336 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.336 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.336 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.336 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139533+0000", 2024-09-16T14:49:56.336 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9573e-05 2024-09-16T14:49:56.336 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.336 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.336 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.336 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139585+0000", 2024-09-16T14:49:56.337 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2170999999999999e-05 2024-09-16T14:49:56.337 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.337 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.337 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:56.337 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.139594+0000", 2024-09-16T14:49:56.337 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.884e-06 2024-09-16T14:49:56.337 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.337 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.337 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.337 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359233+0000", 2024-09-16T14:49:56.337 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.21963890899999999 2024-09-16T14:49:56.337 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.337 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.338 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.338 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359437+0000", 2024-09-16T14:49:56.338 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00020374999999999999 2024-09-16T14:49:56.338 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.338 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.338 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for subops from 11", 2024-09-16T14:49:56.338 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.359611+0000", 2024-09-16T14:49:56.338 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017445899999999999 2024-09-16T14:49:56.338 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.338 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.338 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op_commit", 2024-09-16T14:49:56.338 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.360506+0000", 2024-09-16T14:49:56.338 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00089459299999999995 2024-09-16T14:49:56.339 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.339 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.339 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_commit_rec", 2024-09-16T14:49:56.339 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362594+0000", 2024-09-16T14:49:56.339 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002087776 2024-09-16T14:49:56.339 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.339 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.339 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:56.339 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362636+0000", 2024-09-16T14:49:56.339 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1933e-05 2024-09-16T14:49:56.339 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.339 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.339 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.339 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:48:22.362679+0000", 2024-09-16T14:49:56.340 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2877999999999998e-05 2024-09-16T14:49:56.340 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.340 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.340 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.340 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.340 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.340 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:535 3.fs0 3:ff6b916c:::smithi04040151-25:head [append 0~786432] snapc 1e=[1e,1c,1b] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:56.340 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:56.340 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 48.399173439999998, 2024-09-16T14:49:56.340 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013876462000000001, 2024-09-16T14:49:56.340 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.340 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:56.340 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.341 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.341 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.341 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 535 2024-09-16T14:49:56.341 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.341 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.341 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.341 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.341 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:56.341 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.341 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.341 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.341 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.341 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931362+0000", 2024-09-16T14:49:56.342 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.342 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.342 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.342 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.342 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.931360+0000", 2024-09-16T14:49:56.342 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:56.342 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.342 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.342 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.343 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933014+0000", 2024-09-16T14:49:56.343 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001653715 2024-09-16T14:49:56.343 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.343 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.343 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.343 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933028+0000", 2024-09-16T14:49:56.343 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4239999999999999e-05 2024-09-16T14:49:56.343 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.343 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.343 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.343 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933036+0000", 2024-09-16T14:49:56.343 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.8019999999999992e-06 2024-09-16T14:49:56.343 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.344 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.344 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.344 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933084+0000", 2024-09-16T14:49:56.344 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7663000000000001e-05 2024-09-16T14:49:56.344 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.344 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.344 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.344 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.933184+0000", 2024-09-16T14:49:56.344 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000100549 2024-09-16T14:49:56.344 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.344 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.344 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.344 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.934324+0000", 2024-09-16T14:49:56.345 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011399229999999999 2024-09-16T14:49:56.345 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.345 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.345 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.345 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.941715+0000", 2024-09-16T14:49:56.345 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.007391023 2024-09-16T14:49:56.345 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.345 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.346 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:56.346 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.945187+0000", 2024-09-16T14:49:56.346 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0034717709999999998 2024-09-16T14:49:56.346 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.346 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.346 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.346 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.945238+0000", 2024-09-16T14:49:56.346 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.1459999999999999e-05 2024-09-16T14:49:56.346 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.346 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.346 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.346 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.347 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.347 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:539 3.fs0 3:ff6b916c:::smithi04040151-25:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e61)", 2024-09-16T14:49:56.347 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:56.347 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 48.392051907999999, 2024-09-16T14:49:56.347 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01551078, 2024-09-16T14:49:56.347 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.347 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:56.347 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.347 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.347 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.347 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 539 2024-09-16T14:49:56.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.347 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.348 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.348 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.348 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:56.348 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.348 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.348 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.348 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.348 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:56.348 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.348 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.349 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.349 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.349 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938483+0000", 2024-09-16T14:49:56.349 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.349 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.349 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.349 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.349 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938486+0000", 2024-09-16T14:49:56.349 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7779999999999999e-06 2024-09-16T14:49:56.349 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.349 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.349 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.349 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938492+0000", 2024-09-16T14:49:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8120000000000002e-06 2024-09-16T14:49:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.938495+0000", 2024-09-16T14:49:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.948e-06 2024-09-16T14:49:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942566+0000", 2024-09-16T14:49:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0040700839999999999 2024-09-16T14:49:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:56.351 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.942659+0000", 2024-09-16T14:49:56.351 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3887000000000004e-05 2024-09-16T14:49:56.351 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.351 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.351 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.351 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.948875+0000", 2024-09-16T14:49:56.351 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0062154630000000001 2024-09-16T14:49:56.351 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.351 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.351 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.351 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.948968+0000", 2024-09-16T14:49:56.351 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2758999999999997e-05 2024-09-16T14:49:56.351 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.351 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.352 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.352 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:07.953994+0000", 2024-09-16T14:49:56.352 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0050265580000000004 2024-09-16T14:49:56.352 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.352 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.352 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.352 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.352 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.352 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:600 3.ds0 3:bcba132a:::smithi04040151-1:head [append 0~507904] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:56.352 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.993737+0000", 2024-09-16T14:49:56.352 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 44.336798545999997, 2024-09-16T14:49:56.352 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.008095819, 2024-09-16T14:49:56.352 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.353 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:56.353 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.353 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.353 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.353 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 600 2024-09-16T14:49:56.353 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.353 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.353 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.353 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.353 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.993737+0000", 2024-09-16T14:49:56.353 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.353 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.353 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.353 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.354 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.993737+0000", 2024-09-16T14:49:56.354 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.354 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.354 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.354 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.354 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.993735+0000", 2024-09-16T14:49:56.354 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:56.354 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.354 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.354 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.354 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994263+0000", 2024-09-16T14:49:56.354 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00052718199999999995 2024-09-16T14:49:56.354 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994274+0000", 2024-09-16T14:49:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1535999999999999e-05 2024-09-16T14:49:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994279+0000", 2024-09-16T14:49:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0839999999999999e-06 2024-09-16T14:49:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994330+0000", 2024-09-16T14:49:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.0467999999999999e-05 2024-09-16T14:49:56.356 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.356 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.356 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:56.356 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.994332+0000", 2024-09-16T14:49:56.356 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1390000000000002e-06 2024-09-16T14:49:56.356 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.356 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.356 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.356 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.980708+0000", 2024-09-16T14:49:56.356 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98637637199999995 2024-09-16T14:49:56.356 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.356 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.357 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:56.357 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.981437+0000", 2024-09-16T14:49:56.357 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00072860299999999998 2024-09-16T14:49:56.357 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.357 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.357 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.357 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.989211+0000", 2024-09-16T14:49:56.357 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077741490000000002 2024-09-16T14:49:56.357 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.357 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.357 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.357 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.989340+0000", 2024-09-16T14:49:56.357 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00012908900000000001 2024-09-16T14:49:56.357 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.358 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.358 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.358 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.990969+0000", 2024-09-16T14:49:56.358 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001628612 2024-09-16T14:49:56.358 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.358 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.358 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.358 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996293+0000", 2024-09-16T14:49:56.358 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0053247440000000002 2024-09-16T14:49:56.358 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.358 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.358 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:56.358 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.001434+0000", 2024-09-16T14:49:56.359 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0051404349999999996 2024-09-16T14:49:56.359 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.359 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.359 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.359 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.001833+0000", 2024-09-16T14:49:56.359 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00039878500000000002 2024-09-16T14:49:56.359 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.359 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.359 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.359 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.359 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.359 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:601 3.ds0 3:bcba132a:::smithi04040151-1:head [append 0~606208] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:56.359 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:11.998248+0000", 2024-09-16T14:49:56.360 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 44.332287227999998, 2024-09-16T14:49:56.360 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0062125260000001, 2024-09-16T14:49:56.360 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.360 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:56.360 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.360 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.360 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.360 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 601 2024-09-16T14:49:56.360 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.360 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.360 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.360 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.360 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998248+0000", 2024-09-16T14:49:56.360 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.361 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.361 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.361 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.361 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998248+0000", 2024-09-16T14:49:56.361 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.361 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.361 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.361 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.361 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998246+0000", 2024-09-16T14:49:56.361 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:56.361 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.361 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.361 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.362 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998747+0000", 2024-09-16T14:49:56.362 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00050040399999999997 2024-09-16T14:49:56.362 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.362 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.362 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.362 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998755+0000", 2024-09-16T14:49:56.362 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3329999999999997e-06 2024-09-16T14:49:56.362 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.362 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.362 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.362 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998759+0000", 2024-09-16T14:49:56.362 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.8380000000000004e-06 2024-09-16T14:49:56.362 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.363 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.363 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.363 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998846+0000", 2024-09-16T14:49:56.363 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6456e-05 2024-09-16T14:49:56.363 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.363 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.363 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:56.363 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:11.998847+0000", 2024-09-16T14:49:56.363 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.635e-06 2024-09-16T14:49:56.363 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.363 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.363 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.363 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984175+0000", 2024-09-16T14:49:56.363 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98532833399999997 2024-09-16T14:49:56.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.364 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.364 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:56.364 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984375+0000", 2024-09-16T14:49:56.364 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00019923299999999999 2024-09-16T14:49:56.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.364 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.364 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.364 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992431+0000", 2024-09-16T14:49:56.364 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0080562129999999996 2024-09-16T14:49:56.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.364 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.364 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.364 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.992552+0000", 2024-09-16T14:49:56.365 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000120986 2024-09-16T14:49:56.365 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.365 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.365 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.365 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.993913+0000", 2024-09-16T14:49:56.365 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001361206 2024-09-16T14:49:56.365 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.365 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.365 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.366 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996295+0000", 2024-09-16T14:49:56.366 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0023814629999999999 2024-09-16T14:49:56.366 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.366 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.366 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:56.366 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004101+0000", 2024-09-16T14:49:56.366 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0078063500000000001 2024-09-16T14:49:56.366 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.366 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.366 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.366 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004461+0000", 2024-09-16T14:49:56.366 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00035967799999999999 2024-09-16T14:49:56.366 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.367 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.367 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.367 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.367 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.367 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:602 3.ds0 3:bcba132a:::smithi04040151-1:head [append 0~557056] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:56.367 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:56.370 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 44.328475009000002, 2024-09-16T14:49:56.370 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0032171249999999, 2024-09-16T14:49:56.370 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.370 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:56.370 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.370 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.370 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.370 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 602 2024-09-16T14:49:56.370 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.371 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.371 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.371 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.371 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:56.371 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.371 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.371 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.371 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.371 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:56.371 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.371 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.372 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.372 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.372 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002060+0000", 2024-09-16T14:49:56.372 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.372 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.372 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.372 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.372 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002547+0000", 2024-09-16T14:49:56.372 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00048700700000000002 2024-09-16T14:49:56.373 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.373 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.373 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.373 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002554+0000", 2024-09-16T14:49:56.373 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.435e-06 2024-09-16T14:49:56.373 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.373 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.373 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.373 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002558+0000", 2024-09-16T14:49:56.373 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.501e-06 2024-09-16T14:49:56.373 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.373 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.374 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.374 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002641+0000", 2024-09-16T14:49:56.374 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2874999999999995e-05 2024-09-16T14:49:56.374 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.374 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.374 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:56.374 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002642+0000", 2024-09-16T14:49:56.374 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6670000000000001e-06 2024-09-16T14:49:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984450+0000", 2024-09-16T14:49:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98180768500000004 2024-09-16T14:49:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984591+0000", 2024-09-16T14:49:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014090700000000001 2024-09-16T14:49:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.375 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996586+0000", 2024-09-16T14:49:56.376 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011994898 2024-09-16T14:49:56.376 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.376 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.376 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.376 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.996695+0000", 2024-09-16T14:49:56.376 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000108962 2024-09-16T14:49:56.376 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.376 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.376 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.376 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.997804+0000", 2024-09-16T14:49:56.376 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0011095090000000001 2024-09-16T14:49:56.376 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.376 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.376 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.377 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000509+0000", 2024-09-16T14:49:56.377 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0027042440000000002 2024-09-16T14:49:56.377 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.377 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.377 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:56.377 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.004755+0000", 2024-09-16T14:49:56.377 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0042461269999999997 2024-09-16T14:49:56.378 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.378 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.378 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.379 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005277+0000", 2024-09-16T14:49:56.379 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00052277599999999997 2024-09-16T14:49:56.379 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.379 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.379 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.379 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.379 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.379 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:603 3.ds0 3:bcba132a:::smithi04040151-1:head [setxattr _header (56) in=7b] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:56.379 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:56.379 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 44.327955420000002, 2024-09-16T14:49:56.379 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.003299656, 2024-09-16T14:49:56.380 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.380 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:56.380 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.380 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.380 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.380 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 603 2024-09-16T14:49:56.380 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.380 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.380 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.380 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.380 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:56.380 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.380 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.381 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.381 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.381 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:56.381 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.381 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.381 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.381 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.381 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002580+0000", 2024-09-16T14:49:56.381 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.382 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.382 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.382 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.382 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002582+0000", 2024-09-16T14:49:56.382 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.345e-06 2024-09-16T14:49:56.382 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.382 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.382 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.382 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002586+0000", 2024-09-16T14:49:56.382 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4710000000000001e-06 2024-09-16T14:49:56.382 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.382 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.382 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.382 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002589+0000", 2024-09-16T14:49:56.383 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.694e-06 2024-09-16T14:49:56.383 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.383 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.383 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.383 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002673+0000", 2024-09-16T14:49:56.383 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4349000000000002e-05 2024-09-16T14:49:56.383 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.383 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.383 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:56.383 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002674+0000", 2024-09-16T14:49:56.383 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9300000000000006e-07 2024-09-16T14:49:56.383 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.384 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.384 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.384 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984634+0000", 2024-09-16T14:49:56.384 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98195955099999999 2024-09-16T14:49:56.384 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.384 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.384 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:56.384 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984742+0000", 2024-09-16T14:49:56.384 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000108491 2024-09-16T14:49:56.384 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.384 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.384 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.384 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.998850+0000", 2024-09-16T14:49:56.384 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014108005 2024-09-16T14:49:56.385 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.385 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.385 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.385 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.999006+0000", 2024-09-16T14:49:56.385 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015594799999999999 2024-09-16T14:49:56.385 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.385 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.385 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.385 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.999686+0000", 2024-09-16T14:49:56.385 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00067984999999999996 2024-09-16T14:49:56.386 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.386 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.386 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.386 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000622+0000", 2024-09-16T14:49:56.386 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00093585200000000004 2024-09-16T14:49:56.386 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.386 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.386 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:56.386 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005500+0000", 2024-09-16T14:49:56.386 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0048775049999999999 2024-09-16T14:49:56.386 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.386 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.386 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.386 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.005880+0000", 2024-09-16T14:49:56.387 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00037999399999999998 2024-09-16T14:49:56.387 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.387 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.387 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.387 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.387 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.387 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:604 3.ds0 3:bcba132a:::smithi04040151-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:56.387 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:56.387 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 44.327943877999999, 2024-09-16T14:49:56.387 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.00580336, 2024-09-16T14:49:56.387 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.387 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:56.387 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.387 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.388 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.388 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 604 2024-09-16T14:49:56.388 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.388 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.388 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.388 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.388 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:56.388 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.388 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.388 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.388 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.388 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:56.388 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.388 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.389 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.389 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.389 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002591+0000", 2024-09-16T14:49:56.389 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.389 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.389 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.389 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.389 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002593+0000", 2024-09-16T14:49:56.389 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0559999999999999e-06 2024-09-16T14:49:56.389 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.389 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.389 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.389 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002597+0000", 2024-09-16T14:49:56.389 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.7349999999999998e-06 2024-09-16T14:49:56.390 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.390 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.390 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.390 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002601+0000", 2024-09-16T14:49:56.390 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6519999999999999e-06 2024-09-16T14:49:56.390 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.390 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.390 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.390 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002698+0000", 2024-09-16T14:49:56.390 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7378000000000002e-05 2024-09-16T14:49:56.390 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.390 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.390 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for peered", 2024-09-16T14:49:56.390 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.002699+0000", 2024-09-16T14:49:56.391 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.5099999999999999e-07 2024-09-16T14:49:56.391 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.391 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.391 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.391 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984866+0000", 2024-09-16T14:49:56.391 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.98216700499999998 2024-09-16T14:49:56.391 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.391 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.391 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for degraded object", 2024-09-16T14:49:56.391 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:12.984953+0000", 2024-09-16T14:49:56.392 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6847000000000004e-05 2024-09-16T14:49:56.392 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.392 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.392 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.392 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000753+0000", 2024-09-16T14:49:56.392 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015800455000000001 2024-09-16T14:49:56.392 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.392 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.392 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:56.392 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.000883+0000", 2024-09-16T14:49:56.392 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00013054000000000001 2024-09-16T14:49:56.392 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006482+0000", 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0055987839999999999 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.006630+0000", 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00014820600000000001 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.008395+0000", 2024-09-16T14:49:56.393 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0017643139999999999 2024-09-16T14:49:56.394 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.394 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.394 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.394 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.394 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.394 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:751 3.ds0 3:bcba132a:::smithi04040151-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e80)", 2024-09-16T14:49:56.394 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:27.015103+0000", 2024-09-16T14:49:56.394 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.315432129000001, 2024-09-16T14:49:56.394 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013724121000000001, 2024-09-16T14:49:56.394 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.394 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:56.394 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.394 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.394 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.395 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 751 2024-09-16T14:49:56.395 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.395 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.395 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.395 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.395 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015103+0000", 2024-09-16T14:49:56.395 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.395 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.395 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.396 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.396 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015103+0000", 2024-09-16T14:49:56.396 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.396 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.396 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.396 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.396 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015103+0000", 2024-09-16T14:49:56.396 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.396 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.396 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.396 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.396 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015104+0000", 2024-09-16T14:49:56.396 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.584e-06 2024-09-16T14:49:56.396 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.397 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.397 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.397 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015108+0000", 2024-09-16T14:49:56.397 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9389999999999999e-06 2024-09-16T14:49:56.397 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.397 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.397 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.397 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.015111+0000", 2024-09-16T14:49:56.397 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7889999999999999e-06 2024-09-16T14:49:56.397 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.397 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.397 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.397 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.017354+0000", 2024-09-16T14:49:56.398 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0022426849999999999 2024-09-16T14:49:56.398 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.398 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.398 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:56.398 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.017423+0000", 2024-09-16T14:49:56.398 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.9228999999999997e-05 2024-09-16T14:49:56.398 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.398 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.398 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.398 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.022608+0000", 2024-09-16T14:49:56.398 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0051849920000000002 2024-09-16T14:49:56.398 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.398 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.398 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.022679+0000", 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0444000000000005e-05 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.028827+0000", 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0061487649999999996 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:756 3.fs0 3:f17be48c:::smithi04040151-41:2e [read 0~3588096,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:27.016084+0000", 2024-09-16T14:49:56.399 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.314450878999999, 2024-09-16T14:49:56.400 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012061232, 2024-09-16T14:49:56.400 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.400 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:56.400 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.400 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.400 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.400 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 756 2024-09-16T14:49:56.400 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.400 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.400 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.400 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.400 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016084+0000", 2024-09-16T14:49:56.400 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.401 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.401 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.401 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.401 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016084+0000", 2024-09-16T14:49:56.401 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.401 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.401 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.401 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.401 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016084+0000", 2024-09-16T14:49:56.401 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:56.401 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.401 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.401 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.401 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016090+0000", 2024-09-16T14:49:56.402 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7279999999999998e-06 2024-09-16T14:49:56.402 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.402 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.402 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.402 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016099+0000", 2024-09-16T14:49:56.402 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9649999999999997e-06 2024-09-16T14:49:56.402 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.402 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.402 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.402 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016106+0000", 2024-09-16T14:49:56.402 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3670000000000002e-06 2024-09-16T14:49:56.402 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.402 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.403 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.403 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.017376+0000", 2024-09-16T14:49:56.403 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0012697279999999999 2024-09-16T14:49:56.403 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.403 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.403 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.403 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.017477+0000", 2024-09-16T14:49:56.403 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010165700000000001 2024-09-16T14:49:56.403 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.403 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.403 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.403 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.028146+0000", 2024-09-16T14:49:56.403 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010668502999999999 2024-09-16T14:49:56.403 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.404 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.404 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.404 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.404 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.404 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:757 3.fs0 3:f17be48c:::smithi04040151-41:2e [read 0~3588096,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-16T14:49:56.404 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:27.016111+0000", 2024-09-16T14:49:56.404 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.314424237000001, 2024-09-16T14:49:56.404 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013568108000000001, 2024-09-16T14:49:56.404 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.404 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:56.404 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.404 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.404 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.404 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 757 2024-09-16T14:49:56.405 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.405 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.405 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.405 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.405 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016111+0000", 2024-09-16T14:49:56.405 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.405 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.405 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.405 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.405 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016111+0000", 2024-09-16T14:49:56.406 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.406 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.406 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.406 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.406 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016111+0000", 2024-09-16T14:49:56.406 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.406 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.406 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.406 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.406 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016115+0000", 2024-09-16T14:49:56.406 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9380000000000002e-06 2024-09-16T14:49:56.406 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.406 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.406 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.407 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016121+0000", 2024-09-16T14:49:56.407 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9909999999999999e-06 2024-09-16T14:49:56.407 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.407 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.407 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.407 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.016125+0000", 2024-09-16T14:49:56.407 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.6600000000000003e-06 2024-09-16T14:49:56.407 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.407 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.407 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.407 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.020236+0000", 2024-09-16T14:49:56.408 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0041107019999999999 2024-09-16T14:49:56.408 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.408 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.408 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.408 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.020334+0000", 2024-09-16T14:49:56.408 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8166999999999997e-05 2024-09-16T14:49:56.408 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.408 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.408 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.408 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:27.029679+0000", 2024-09-16T14:49:56.408 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0093450909999999998 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.7s2 82/27 ECSubWrite(tid=342, reqid=osd.3.0:342, at_version=82'110, trim_to=0'0, roll_forward_to=81'108))", 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.933809+0000", 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 27.396726579999999, 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013532219, 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.409 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933809+0000", 2024-09-16T14:49:56.410 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.410 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.410 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.410 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.410 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933809+0000", 2024-09-16T14:49:56.410 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.410 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.410 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.410 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.410 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933808+0000", 2024-09-16T14:49:56.410 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.410 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.410 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.411 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.411 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933834+0000", 2024-09-16T14:49:56.411 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5735e-05 2024-09-16T14:49:56.411 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.411 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.411 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.411 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933844+0000", 2024-09-16T14:49:56.411 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0081999999999999e-05 2024-09-16T14:49:56.411 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.411 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.411 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.411 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933851+0000", 2024-09-16T14:49:56.411 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.8070000000000002e-06 2024-09-16T14:49:56.411 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.412 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.412 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.412 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934134+0000", 2024-09-16T14:49:56.412 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00028271999999999998 2024-09-16T14:49:56.412 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.412 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.412 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.412 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.934143+0000", 2024-09-16T14:49:56.412 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4120000000000007e-06 2024-09-16T14:49:56.412 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.412 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.412 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.947319+0000", 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013176155 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.947341+0000", 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1603e-05 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs5 82/65 ECSubWrite(tid=345, reqid=osd.3.0:345, at_version=82'69, trim_to=0'0, roll_forward_to=81'65))", 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.933907+0000", 2024-09-16T14:49:56.413 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 27.396628739000001, 2024-09-16T14:49:56.414 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012550997, 2024-09-16T14:49:56.414 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.414 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:56.414 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.414 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.414 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.414 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933907+0000", 2024-09-16T14:49:56.414 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.414 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.414 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.414 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.414 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933907+0000", 2024-09-16T14:49:56.414 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.415 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.415 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.415 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.415 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933906+0000", 2024-09-16T14:49:56.415 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.415 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.415 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.415 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.415 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933922+0000", 2024-09-16T14:49:56.415 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5930000000000002e-05 2024-09-16T14:49:56.415 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.415 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.415 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.415 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933929+0000", 2024-09-16T14:49:56.416 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6730000000000003e-06 2024-09-16T14:49:56.416 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.416 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.416 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.416 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933934+0000", 2024-09-16T14:49:56.416 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.4360000000000001e-06 2024-09-16T14:49:56.416 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.416 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.416 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.416 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.938626+0000", 2024-09-16T14:49:56.416 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0046912070000000002 2024-09-16T14:49:56.416 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.416 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.416 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.417 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.938635+0000", 2024-09-16T14:49:56.417 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.3589999999999997e-06 2024-09-16T14:49:56.417 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.417 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.417 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.417 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.946337+0000", 2024-09-16T14:49:56.417 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0077023760000000004 2024-09-16T14:49:56.417 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.417 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.417 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.417 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.946458+0000", 2024-09-16T14:49:56.417 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000120277 2024-09-16T14:49:56.417 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.417 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.418 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.418 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.418 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.418 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.7s2 82/27 ECSubWrite(tid=344, reqid=osd.3.0:344, at_version=82'112, trim_to=0'0, roll_forward_to=81'108))", 2024-09-16T14:49:56.418 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:28.933939+0000", 2024-09-16T14:49:56.418 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 27.396596165999998, 2024-09-16T14:49:56.418 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014103087, 2024-09-16T14:49:56.418 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.418 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:56.418 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.418 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.418 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.418 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933939+0000", 2024-09-16T14:49:56.419 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.419 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.419 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.419 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.419 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933939+0000", 2024-09-16T14:49:56.419 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.419 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.419 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.419 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.419 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933939+0000", 2024-09-16T14:49:56.419 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:56.419 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.419 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.419 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.420 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933954+0000", 2024-09-16T14:49:56.420 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.491e-05 2024-09-16T14:49:56.420 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.420 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.420 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.420 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933960+0000", 2024-09-16T14:49:56.420 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6109999999999999e-06 2024-09-16T14:49:56.420 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.420 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.420 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.420 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.933965+0000", 2024-09-16T14:49:56.420 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9599999999999999e-06 2024-09-16T14:49:56.420 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.420 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.421 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.421 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.941466+0000", 2024-09-16T14:49:56.421 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0075007529999999998 2024-09-16T14:49:56.421 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.421 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.421 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.421 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.941478+0000", 2024-09-16T14:49:56.421 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1501999999999999e-05 2024-09-16T14:49:56.421 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.421 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.421 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.421 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.948023+0000", 2024-09-16T14:49:56.421 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0065449870000000004 2024-09-16T14:49:56.421 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.422 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.422 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.422 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:28.948042+0000", 2024-09-16T14:49:56.422 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9545999999999999e-05 2024-09-16T14:49:56.422 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.422 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.422 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.422 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.422 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.422 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:860 3.ds0 3:bd112605:::smithi04040151-44:3b [read 0~737280,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:56.422 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.862879+0000", 2024-09-16T14:49:56.422 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 14.467656179, 2024-09-16T14:49:56.423 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013219919, 2024-09-16T14:49:56.423 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.423 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:56.423 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.423 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.423 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.423 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 860 2024-09-16T14:49:56.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.423 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.423 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.423 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.424 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862879+0000", 2024-09-16T14:49:56.424 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.424 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.424 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.424 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.424 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862879+0000", 2024-09-16T14:49:56.424 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.424 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.424 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.424 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.424 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862877+0000", 2024-09-16T14:49:56.424 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:56.424 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.424 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.425 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.425 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862886+0000", 2024-09-16T14:49:56.425 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4190000000000002e-06 2024-09-16T14:49:56.425 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.425 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.425 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.425 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862898+0000", 2024-09-16T14:49:56.425 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1994000000000001e-05 2024-09-16T14:49:56.425 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.425 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.425 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.425 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862906+0000", 2024-09-16T14:49:56.425 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.1459999999999996e-06 2024-09-16T14:49:56.425 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.426 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.426 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.426 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862955+0000", 2024-09-16T14:49:56.426 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9147999999999998e-05 2024-09-16T14:49:56.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.426 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.426 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:56.426 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862965+0000", 2024-09-16T14:49:56.426 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.5519999999999993e-06 2024-09-16T14:49:56.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.426 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.426 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.426 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866479+0000", 2024-09-16T14:49:56.426 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0035142139999999999 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866601+0000", 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00012230699999999999 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.876099+0000", 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0094979060000000004 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.427 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.428 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:861 3.ds0 3:bd112605:::smithi04040151-44:3b [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:56.428 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.862920+0000", 2024-09-16T14:49:56.428 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 14.467615713000001, 2024-09-16T14:49:56.428 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013865529, 2024-09-16T14:49:56.428 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.428 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:56.428 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.428 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.428 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.428 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 861 2024-09-16T14:49:56.428 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.428 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.428 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.428 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.429 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862920+0000", 2024-09-16T14:49:56.429 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.429 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.429 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.429 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.429 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862920+0000", 2024-09-16T14:49:56.429 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.429 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.429 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.430 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.430 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862919+0000", 2024-09-16T14:49:56.430 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.430 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.430 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.430 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.430 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862922+0000", 2024-09-16T14:49:56.430 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1489999999999998e-06 2024-09-16T14:49:56.430 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.430 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.430 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.430 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862926+0000", 2024-09-16T14:49:56.430 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.0969999999999999e-06 2024-09-16T14:49:56.430 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.431 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.431 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.431 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862929+0000", 2024-09-16T14:49:56.431 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.061e-06 2024-09-16T14:49:56.431 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.431 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.431 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.431 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863026+0000", 2024-09-16T14:49:56.431 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6824999999999995e-05 2024-09-16T14:49:56.431 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.431 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.431 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:56.431 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863031+0000", 2024-09-16T14:49:56.432 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1919999999999998e-06 2024-09-16T14:49:56.432 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.432 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.432 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.432 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866928+0000", 2024-09-16T14:49:56.432 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0038978099999999998 2024-09-16T14:49:56.432 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.432 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.432 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.432 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867034+0000", 2024-09-16T14:49:56.432 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010561299999999999 2024-09-16T14:49:56.432 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.432 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.432 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.433 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.876785+0000", 2024-09-16T14:49:56.433 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0097512020000000005 2024-09-16T14:49:56.433 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.433 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.433 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.433 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.433 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.433 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:862 3.ds0 3:bd112605:::smithi04040151-44:3b [read 0~737280,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:56.433 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.862941+0000", 2024-09-16T14:49:56.433 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 14.467594454, 2024-09-16T14:49:56.433 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014724180999999999, 2024-09-16T14:49:56.433 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.433 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:56.433 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 862 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862941+0000", 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862941+0000", 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.434 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.435 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.435 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.435 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862941+0000", 2024-09-16T14:49:56.435 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.435 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.435 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.435 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.435 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862943+0000", 2024-09-16T14:49:56.435 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3180000000000001e-06 2024-09-16T14:49:56.435 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.435 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.435 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.435 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862946+0000", 2024-09-16T14:49:56.436 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.6160000000000002e-06 2024-09-16T14:49:56.436 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.436 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.436 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.436 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862949+0000", 2024-09-16T14:49:56.436 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6400000000000001e-06 2024-09-16T14:49:56.436 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.436 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.436 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.436 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863054+0000", 2024-09-16T14:49:56.436 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010529100000000001 2024-09-16T14:49:56.436 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.436 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.436 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:56.437 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863061+0000", 2024-09-16T14:49:56.437 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6590000000000001e-06 2024-09-16T14:49:56.437 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.437 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.437 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.437 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867329+0000", 2024-09-16T14:49:56.437 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0042681100000000003 2024-09-16T14:49:56.437 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.437 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.437 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.437 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867443+0000", 2024-09-16T14:49:56.437 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000113596 2024-09-16T14:49:56.437 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.437 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.438 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.438 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.877665+0000", 2024-09-16T14:49:56.438 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.01022227 2024-09-16T14:49:56.438 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.438 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.438 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.438 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.438 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.438 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:863 3.13s0 3:c9d06351:::smithi04040151-33:3b [read 0~1654784,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:56.438 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.862960+0000", 2024-09-16T14:49:56.438 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 14.467575384, 2024-09-16T14:49:56.438 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013471337999999999, 2024-09-16T14:49:56.438 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.438 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 863 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862960+0000", 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862960+0000", 2024-09-16T14:49:56.439 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.440 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.440 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.440 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.440 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862960+0000", 2024-09-16T14:49:56.440 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.440 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.440 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.440 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.440 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862962+0000", 2024-09-16T14:49:56.440 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4420000000000001e-06 2024-09-16T14:49:56.440 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.440 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.440 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.440 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862966+0000", 2024-09-16T14:49:56.441 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5810000000000001e-06 2024-09-16T14:49:56.441 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.441 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.441 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.441 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862968+0000", 2024-09-16T14:49:56.441 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7369999999999998e-06 2024-09-16T14:49:56.441 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.441 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.441 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.441 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.862999+0000", 2024-09-16T14:49:56.441 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1031000000000002e-05 2024-09-16T14:49:56.441 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.442 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.442 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:56.442 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863008+0000", 2024-09-16T14:49:56.442 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.4200000000000007e-06 2024-09-16T14:49:56.442 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.442 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.442 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.442 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.865719+0000", 2024-09-16T14:49:56.442 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002710877 2024-09-16T14:49:56.442 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.442 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.442 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.442 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866147+0000", 2024-09-16T14:49:56.442 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000428124 2024-09-16T14:49:56.443 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.443 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.443 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.443 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.876431+0000", 2024-09-16T14:49:56.443 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.010284422 2024-09-16T14:49:56.443 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.443 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.443 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.443 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.443 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.443 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:864 3.13s0 3:c9d06351:::smithi04040151-33:3b [read 0~1654784,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:56.443 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.863012+0000", 2024-09-16T14:49:56.443 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 14.467523234, 2024-09-16T14:49:56.443 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014700533999999999, 2024-09-16T14:49:56.444 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.444 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:56.444 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.444 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.444 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.444 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 864 2024-09-16T14:49:56.444 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.444 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.444 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.444 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.444 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863012+0000", 2024-09-16T14:49:56.444 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.445 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.445 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.445 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.445 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863012+0000", 2024-09-16T14:49:56.445 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.445 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.445 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.445 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.445 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863011+0000", 2024-09-16T14:49:56.445 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:56.445 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.445 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.445 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.445 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863018+0000", 2024-09-16T14:49:56.446 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.5679999999999996e-06 2024-09-16T14:49:56.446 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.446 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.446 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.446 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863026+0000", 2024-09-16T14:49:56.446 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8980000000000002e-06 2024-09-16T14:49:56.446 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.446 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.446 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.446 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863032+0000", 2024-09-16T14:49:56.446 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.7400000000000001e-06 2024-09-16T14:49:56.446 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.446 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.447 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.447 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863063+0000", 2024-09-16T14:49:56.447 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.0474e-05 2024-09-16T14:49:56.447 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.447 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.447 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:56.447 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863069+0000", 2024-09-16T14:49:56.447 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.9939999999999997e-06 2024-09-16T14:49:56.447 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.447 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.447 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.447 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866510+0000", 2024-09-16T14:49:56.447 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00344123 2024-09-16T14:49:56.447 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.448 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.448 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.448 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.866610+0000", 2024-09-16T14:49:56.448 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000100385 2024-09-16T14:49:56.448 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.448 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.448 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.448 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.877713+0000", 2024-09-16T14:49:56.448 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011102386000000001 2024-09-16T14:49:56.448 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.448 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.448 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.448 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.448 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.449 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:865 3.13s0 3:c9d06351:::smithi04040151-33:3b [read 0~1654784,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:56.449 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.863041+0000", 2024-09-16T14:49:56.449 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 14.4674943, 2024-09-16T14:49:56.449 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015790143999999999, 2024-09-16T14:49:56.449 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:56.449 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:56.449 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:56.449 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:56.449 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.449 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 865 2024-09-16T14:49:56.449 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.449 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:56.449 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.449 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:56.450 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863041+0000", 2024-09-16T14:49:56.450 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.450 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.450 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.450 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:56.450 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863041+0000", 2024-09-16T14:49:56.450 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:56.450 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.450 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.450 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:56.450 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863041+0000", 2024-09-16T14:49:56.450 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.450 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.450 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.451 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:56.451 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863043+0000", 2024-09-16T14:49:56.451 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0549999999999998e-06 2024-09-16T14:49:56.451 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.451 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.451 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:56.451 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863047+0000", 2024-09-16T14:49:56.451 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.941e-06 2024-09-16T14:49:56.451 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.451 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.451 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.451 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863050+0000", 2024-09-16T14:49:56.451 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8399999999999999e-06 2024-09-16T14:49:56.452 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.452 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.452 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.452 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863083+0000", 2024-09-16T14:49:56.452 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3009999999999997e-05 2024-09-16T14:49:56.452 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.452 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.452 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:56.452 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.863085+0000", 2024-09-16T14:49:56.452 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8849999999999999e-06 2024-09-16T14:49:56.452 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.452 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.452 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:56.452 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867224+0000", 2024-09-16T14:49:56.453 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0041382240000000002 2024-09-16T14:49:56.453 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.453 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.453 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:56.453 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.867316+0000", 2024-09-16T14:49:56.453 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2559000000000006e-05 2024-09-16T14:49:56.453 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:56.453 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:56.453 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:56.453 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.878831+0000", 2024-09-16T14:49:56.453 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.011514912 2024-09-16T14:49:56.453 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.453 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.453 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.454 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:56.454 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:56.454 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:56.455 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-16T14:49:56.569 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:56.569 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:56.569 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:56.569 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:56.579 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-16T14:49:56.681 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:56.681 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:56.681 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:56.681 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:56.681 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:56.690 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-16T14:49:56.794 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:56.794 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:56.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:56.795 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:56.795 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.795 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:295 3.9s0 3:96d2b183:::smithi04040151-29:head [copy-from ver 0] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-16T14:49:56.795 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:56.795 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 72.876817580999997, 2024-09-16T14:49:56.795 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010216596999999999, 2024-09-16T14:49:56.795 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.795 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:56.795 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:56.795 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:56.795 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.796 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 295 2024-09-16T14:49:56.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.796 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:56.796 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.796 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.796 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.796 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915433+0000", 2024-09-16T14:49:56.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915430+0000", 2024-09-16T14:49:56.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:56.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.797 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.797 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915441+0000", 2024-09-16T14:49:56.797 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0789e-05 2024-09-16T14:49:56.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.797 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915462+0000", 2024-09-16T14:49:56.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0845000000000001e-05 2024-09-16T14:49:56.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915473+0000", 2024-09-16T14:49:56.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0805000000000001e-05 2024-09-16T14:49:56.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.798 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.798 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.798 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915528+0000", 2024-09-16T14:49:56.798 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4772999999999999e-05 2024-09-16T14:49:56.798 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:56.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915537+0000", 2024-09-16T14:49:56.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6700000000000006e-06 2024-09-16T14:49:56.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.799 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.919813+0000", 2024-09-16T14:49:56.799 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0042757840000000004 2024-09-16T14:49:56.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.799 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.799 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:56.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.919907+0000", 2024-09-16T14:49:56.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3504999999999993e-05 2024-09-16T14:49:56.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.800 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.800 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925649+0000", 2024-09-16T14:49:56.800 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057427870000000001 2024-09-16T14:49:56.800 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.800 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.800 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.800 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.800 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:296 3.9s0 3:96d2b183:::smithi04040151-29:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e37)", 2024-09-16T14:49:56.801 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:56.801 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 72.876765664000004, 2024-09-16T14:49:56.801 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010423623, 2024-09-16T14:49:56.801 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.801 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:56.801 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:56.801 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:56.801 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.801 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 296 2024-09-16T14:49:56.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.801 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.801 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.801 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:56.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915485+0000", 2024-09-16T14:49:56.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.802 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.802 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915484+0000", 2024-09-16T14:49:56.802 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.802 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915489+0000", 2024-09-16T14:49:56.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6079999999999998e-06 2024-09-16T14:49:56.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915496+0000", 2024-09-16T14:49:56.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7589999999999997e-06 2024-09-16T14:49:56.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.803 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.803 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.803 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915506+0000", 2024-09-16T14:49:56.803 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0340000000000002e-06 2024-09-16T14:49:56.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915588+0000", 2024-09-16T14:49:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2198000000000001e-05 2024-09-16T14:49:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.915594+0000", 2024-09-16T14:49:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6229999999999999e-06 2024-09-16T14:49:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920099+0000", 2024-09-16T14:49:56.804 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0045048700000000002 2024-09-16T14:49:56.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:56.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920147+0000", 2024-09-16T14:49:56.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.8239999999999999e-05 2024-09-16T14:49:56.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.805 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925726+0000", 2024-09-16T14:49:56.805 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055787529999999997 2024-09-16T14:49:56.805 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.805 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.805 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925816+0000", 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0185000000000005e-05 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925908+0000", 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1948999999999997e-05 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as4 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:56.806 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 72.868211013000007, 2024-09-16T14:49:56.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088260140000000001, 2024-09-16T14:49:56.807 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.807 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:56.807 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:56.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.807 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.807 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.807 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:56.807 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924039+0000", 2024-09-16T14:49:56.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924062+0000", 2024-09-16T14:49:56.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2892e-05 2024-09-16T14:49:56.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.808 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.808 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.808 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924070+0000", 2024-09-16T14:49:56.808 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8690000000000004e-06 2024-09-16T14:49:56.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924076+0000", 2024-09-16T14:49:56.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.0739999999999998e-06 2024-09-16T14:49:56.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.809 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924119+0000", 2024-09-16T14:49:56.809 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.2549999999999997e-05 2024-09-16T14:49:56.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.809 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.809 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924126+0000", 2024-09-16T14:49:56.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8960000000000003e-06 2024-09-16T14:49:56.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932835+0000", 2024-09-16T14:49:56.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087089200000000002 2024-09-16T14:49:56.810 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.810 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.810 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.810 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932865+0000", 2024-09-16T14:49:56.810 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0054000000000001e-05 2024-09-16T14:49:56.810 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.810 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es1 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 72.868096425000005, 2024-09-16T14:49:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087127249999999993, 2024-09-16T14:49:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.811 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:56.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:56.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.812 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.812 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924154+0000", 2024-09-16T14:49:56.812 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.812 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924168+0000", 2024-09-16T14:49:56.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4800000000000001e-05 2024-09-16T14:49:56.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924176+0000", 2024-09-16T14:49:56.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.25e-06 2024-09-16T14:49:56.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.813 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.813 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.813 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924182+0000", 2024-09-16T14:49:56.813 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3640000000000004e-06 2024-09-16T14:49:56.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924229+0000", 2024-09-16T14:49:56.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6477999999999998e-05 2024-09-16T14:49:56.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924238+0000", 2024-09-16T14:49:56.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4429999999999992e-06 2024-09-16T14:49:56.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.814 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.814 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.814 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932848+0000", 2024-09-16T14:49:56.814 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086097910000000003 2024-09-16T14:49:56.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.815 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.815 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.932867+0000", 2024-09-16T14:49:56.815 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8927999999999999e-05 2024-09-16T14:49:56.815 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.815 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.815 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.815 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.815 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:394 3.9s0 3:93498a0b:::smithi04040151-32:head [append 0~491520] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:56.815 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:56.815 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.922677944, 2024-09-16T14:49:56.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016461614999999999, 2024-09-16T14:49:56.816 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.816 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:56.816 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:56.816 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:56.816 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.816 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 394 2024-09-16T14:49:56.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.816 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.816 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.816 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.816 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:56.816 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869572+0000", 2024-09-16T14:49:56.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.869570+0000", 2024-09-16T14:49:56.817 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:56.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.817 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.817 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.817 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872427+0000", 2024-09-16T14:49:56.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0028569070000000001 2024-09-16T14:49:56.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872450+0000", 2024-09-16T14:49:56.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2943e-05 2024-09-16T14:49:56.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.818 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872461+0000", 2024-09-16T14:49:56.818 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0882e-05 2024-09-16T14:49:56.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.818 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.818 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872521+0000", 2024-09-16T14:49:56.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9639000000000003e-05 2024-09-16T14:49:56.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:56.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.872530+0000", 2024-09-16T14:49:56.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8810000000000002e-06 2024-09-16T14:49:56.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.819 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.819 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.878309+0000", 2024-09-16T14:49:56.819 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057788520000000001 2024-09-16T14:49:56.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.819 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:56.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.878434+0000", 2024-09-16T14:49:56.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012550899999999999 2024-09-16T14:49:56.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.880126+0000", 2024-09-16T14:49:56.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016919019999999999 2024-09-16T14:49:56.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.820 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.820 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.820 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885887+0000", 2024-09-16T14:49:56.820 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057608959999999997 2024-09-16T14:49:56.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:56.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885957+0000", 2024-09-16T14:49:56.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0504000000000001e-05 2024-09-16T14:49:56.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.821 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.821 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886034+0000", 2024-09-16T14:49:56.821 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6661000000000003e-05 2024-09-16T14:49:56.821 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.821 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.821 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.822 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:395 3.9s0 3:93498a0b:::smithi04040151-32:head [append 0~770048] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:56.822 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:56.822 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.917950529000002, 2024-09-16T14:49:56.822 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.018911853999999999, 2024-09-16T14:49:56.822 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.822 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:56.822 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:56.822 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:56.822 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.822 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 395 2024-09-16T14:49:56.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.822 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.822 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:56.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874300+0000", 2024-09-16T14:49:56.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.823 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.823 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.823 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.874298+0000", 2024-09-16T14:49:56.823 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:56.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875671+0000", 2024-09-16T14:49:56.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0013727069999999999 2024-09-16T14:49:56.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875689+0000", 2024-09-16T14:49:56.824 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7832000000000001e-05 2024-09-16T14:49:56.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.824 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.824 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.824 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875700+0000", 2024-09-16T14:49:56.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1267000000000001e-05 2024-09-16T14:49:56.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875758+0000", 2024-09-16T14:49:56.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8060999999999999e-05 2024-09-16T14:49:56.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:56.825 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875766+0000", 2024-09-16T14:49:56.825 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9580000000000007e-06 2024-09-16T14:49:56.825 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.825 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.825 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881004+0000", 2024-09-16T14:49:56.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0052375169999999997 2024-09-16T14:49:56.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:56.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881098+0000", 2024-09-16T14:49:56.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3875999999999994e-05 2024-09-16T14:49:56.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.826 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.826 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882500+0000", 2024-09-16T14:49:56.826 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014019600000000001 2024-09-16T14:49:56.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.826 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886041+0000", 2024-09-16T14:49:56.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035415210000000002 2024-09-16T14:49:56.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:56.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893136+0000", 2024-09-16T14:49:56.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070949330000000003 2024-09-16T14:49:56.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.827 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.827 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.827 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893212+0000", 2024-09-16T14:49:56.827 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5739000000000001e-05 2024-09-16T14:49:56.827 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:396 3.9s0 3:93498a0b:::smithi04040151-32:head [setxattr _header (57) in=7b] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.916533373999997, 2024-09-16T14:49:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017781597999999999, 2024-09-16T14:49:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:56.828 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.829 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 396 2024-09-16T14:49:56.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.829 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:56.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.829 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.829 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.829 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875717+0000", 2024-09-16T14:49:56.829 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875715+0000", 2024-09-16T14:49:56.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999976 2024-09-16T14:49:56.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875721+0000", 2024-09-16T14:49:56.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6989999999999997e-06 2024-09-16T14:49:56.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.830 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.830 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.830 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875728+0000", 2024-09-16T14:49:56.830 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9040000000000003e-06 2024-09-16T14:49:56.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875734+0000", 2024-09-16T14:49:56.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4519999999999996e-06 2024-09-16T14:49:56.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875800+0000", 2024-09-16T14:49:56.831 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6538000000000006e-05 2024-09-16T14:49:56.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.831 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.831 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:56.831 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875804+0000", 2024-09-16T14:49:56.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.771e-06 2024-09-16T14:49:56.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883232+0000", 2024-09-16T14:49:56.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074282009999999997 2024-09-16T14:49:56.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:56.832 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883323+0000", 2024-09-16T14:49:56.832 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0740000000000002e-05 2024-09-16T14:49:56.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.832 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.832 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884057+0000", 2024-09-16T14:49:56.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00073422299999999999 2024-09-16T14:49:56.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893020+0000", 2024-09-16T14:49:56.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089623210000000005 2024-09-16T14:49:56.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.833 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:56.833 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893403+0000", 2024-09-16T14:49:56.833 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00038375799999999999 2024-09-16T14:49:56.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.833 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.834 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.834 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893499+0000", 2024-09-16T14:49:56.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5284000000000005e-05 2024-09-16T14:49:56.834 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.834 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.834 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.834 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:397 3.9s0 3:93498a0b:::smithi04040151-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:56.834 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:56.834 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.916502248999997, 2024-09-16T14:49:56.834 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.019838090999999999, 2024-09-16T14:49:56.834 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.834 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:56.835 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:56.835 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:56.835 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.835 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 397 2024-09-16T14:49:56.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.835 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:56.835 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.835 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.835 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.835 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:56.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875748+0000", 2024-09-16T14:49:56.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.836 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.836 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875750+0000", 2024-09-16T14:49:56.836 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3939999999999999e-06 2024-09-16T14:49:56.836 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.836 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875755+0000", 2024-09-16T14:49:56.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.31e-06 2024-09-16T14:49:56.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875760+0000", 2024-09-16T14:49:56.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4769999999999997e-06 2024-09-16T14:49:56.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.837 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.837 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.837 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875824+0000", 2024-09-16T14:49:56.837 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3893000000000006e-05 2024-09-16T14:49:56.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:56.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.875829+0000", 2024-09-16T14:49:56.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3489999999999999e-06 2024-09-16T14:49:56.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885652+0000", 2024-09-16T14:49:56.838 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0098232530000000005 2024-09-16T14:49:56.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.838 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.838 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:56.838 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.885748+0000", 2024-09-16T14:49:56.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5366999999999997e-05 2024-09-16T14:49:56.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893876+0000", 2024-09-16T14:49:56.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081279479999999994 2024-09-16T14:49:56.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.839 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:56.839 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.893967+0000", 2024-09-16T14:49:56.839 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0777000000000002e-05 2024-09-16T14:49:56.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.839 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.840 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.840 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895586+0000", 2024-09-16T14:49:56.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0016197029999999999 2024-09-16T14:49:56.840 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.840 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.840 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.840 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.840 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(osd.3.17:4 3.9s0 3:9b721ec3:::smithi04040151-12:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:56.840 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:56.840 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 62.908279954999998, 2024-09-16T14:49:56.840 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011299052, 2024-09-16T14:49:56.840 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.841 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:56.841 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:56.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.841 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883970+0000", 2024-09-16T14:49:56.841 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.841 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.841 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883969+0000", 2024-09-16T14:49:56.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:56.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883976+0000", 2024-09-16T14:49:56.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3730000000000004e-06 2024-09-16T14:49:56.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.842 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.842 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883999+0000", 2024-09-16T14:49:56.842 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3390000000000001e-05 2024-09-16T14:49:56.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.842 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884007+0000", 2024-09-16T14:49:56.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1489999999999997e-06 2024-09-16T14:49:56.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886141+0000", 2024-09-16T14:49:56.843 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0021347240000000002 2024-09-16T14:49:56.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.843 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.843 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:56.843 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.886230+0000", 2024-09-16T14:49:56.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8234999999999998e-05 2024-09-16T14:49:56.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.844 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.844 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895269+0000", 2024-09-16T14:49:56.844 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090399230000000001 2024-09-16T14:49:56.844 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.844 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.844 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.844 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.844 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:440 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:56.844 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:56.845 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.916226633999997, 2024-09-16T14:49:56.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01253772, 2024-09-16T14:49:56.845 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.845 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:56.845 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:56.845 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:56.845 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.845 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 440 2024-09-16T14:49:56.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.845 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.845 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.845 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.845 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:56.845 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876024+0000", 2024-09-16T14:49:56.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.846 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876022+0000", 2024-09-16T14:49:56.846 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:56.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.846 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.846 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876033+0000", 2024-09-16T14:49:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1474000000000001e-05 2024-09-16T14:49:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876051+0000", 2024-09-16T14:49:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7628000000000001e-05 2024-09-16T14:49:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876060+0000", 2024-09-16T14:49:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6339999999999996e-06 2024-09-16T14:49:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.847 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876128+0000", 2024-09-16T14:49:56.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7460999999999997e-05 2024-09-16T14:49:56.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:56.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876138+0000", 2024-09-16T14:49:56.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8649999999999996e-06 2024-09-16T14:49:56.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.848 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.848 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.848 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882141+0000", 2024-09-16T14:49:56.848 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060036550000000001 2024-09-16T14:49:56.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:56.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.882233+0000", 2024-09-16T14:49:56.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1626000000000007e-05 2024-09-16T14:49:56.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.849 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.849 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.888561+0000", 2024-09-16T14:49:56.849 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063284769999999999 2024-09-16T14:49:56.849 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.849 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.849 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.850 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:441 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:56.850 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:56.850 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.916179577000001, 2024-09-16T14:49:56.850 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.013066019999999999, 2024-09-16T14:49:56.850 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.850 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:56.850 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:56.850 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:56.850 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.850 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 441 2024-09-16T14:49:56.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.850 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.850 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:56.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876071+0000", 2024-09-16T14:49:56.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.851 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.851 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.851 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876070+0000", 2024-09-16T14:49:56.851 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876074+0000", 2024-09-16T14:49:56.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1330000000000001e-06 2024-09-16T14:49:56.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.852 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.852 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876083+0000", 2024-09-16T14:49:56.852 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2519999999999999e-06 2024-09-16T14:49:56.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.852 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876088+0000", 2024-09-16T14:49:56.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5509999999999999e-06 2024-09-16T14:49:56.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876235+0000", 2024-09-16T14:49:56.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014688900000000001 2024-09-16T14:49:56.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.853 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.853 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:56.853 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876243+0000", 2024-09-16T14:49:56.853 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5490000000000001e-06 2024-09-16T14:49:56.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.884127+0000", 2024-09-16T14:49:56.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078848489999999993 2024-09-16T14:49:56.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:56.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.884238+0000", 2024-09-16T14:49:56.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000111007 2024-09-16T14:49:56.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.854 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.854 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.854 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889137+0000", 2024-09-16T14:49:56.854 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0048983739999999996 2024-09-16T14:49:56.855 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.855 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.855 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.855 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.855 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:442 3.9s0 3:9b721ec3:::smithi04040151-12:head [read 0~196608,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:56.855 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:56.855 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 58.916140919, 2024-09-16T14:49:56.855 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014710668, 2024-09-16T14:49:56.855 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.855 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:56.855 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:56.855 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:56.856 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.856 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 442 2024-09-16T14:49:56.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.856 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:56.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.856 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.856 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.856 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:56.856 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876109+0000", 2024-09-16T14:49:56.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876113+0000", 2024-09-16T14:49:56.857 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0759999999999996e-06 2024-09-16T14:49:56.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.857 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.857 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.857 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876119+0000", 2024-09-16T14:49:56.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3339999999999998e-06 2024-09-16T14:49:56.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876126+0000", 2024-09-16T14:49:56.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8410000000000002e-06 2024-09-16T14:49:56.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.858 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876273+0000", 2024-09-16T14:49:56.858 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00014714900000000001 2024-09-16T14:49:56.858 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.858 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.858 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:56.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876279+0000", 2024-09-16T14:49:56.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7819999999999999e-06 2024-09-16T14:49:56.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.885397+0000", 2024-09-16T14:49:56.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0091178149999999996 2024-09-16T14:49:56.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.859 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.859 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:56.859 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.885492+0000", 2024-09-16T14:49:56.859 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5384999999999996e-05 2024-09-16T14:49:56.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.860 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.860 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.890820+0000", 2024-09-16T14:49:56.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053278559999999997 2024-09-16T14:49:56.860 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.860 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.860 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.860 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.860 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs3 61/27 ECSubWrite(tid=238, reqid=osd.3.0:238, at_version=61'42, trim_to=0'0, roll_forward_to=60'38))", 2024-09-16T14:49:56.860 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:56.860 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.865382134999997, 2024-09-16T14:49:56.860 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0089143629999999998, 2024-09-16T14:49:56.861 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.861 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:56.861 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:56.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.861 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.861 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.861 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:56.861 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926868+0000", 2024-09-16T14:49:56.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926896+0000", 2024-09-16T14:49:56.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7764999999999999e-05 2024-09-16T14:49:56.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.862 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.862 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.862 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926909+0000", 2024-09-16T14:49:56.862 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3454e-05 2024-09-16T14:49:56.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926918+0000", 2024-09-16T14:49:56.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7099999999999996e-06 2024-09-16T14:49:56.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926968+0000", 2024-09-16T14:49:56.863 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0778000000000001e-05 2024-09-16T14:49:56.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.863 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.863 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.863 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926989+0000", 2024-09-16T14:49:56.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0069999999999999e-05 2024-09-16T14:49:56.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935750+0000", 2024-09-16T14:49:56.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.008761876 2024-09-16T14:49:56.864 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.864 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.864 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.864 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935783+0000", 2024-09-16T14:49:56.864 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2202000000000001e-05 2024-09-16T14:49:56.864 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.864 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.864 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.865 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s6 61/27 ECSubWrite(tid=239, reqid=osd.3.0:239, at_version=61'79, trim_to=0'0, roll_forward_to=60'75))", 2024-09-16T14:49:56.865 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:56.865 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.865319337999999, 2024-09-16T14:49:56.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.009451849, 2024-09-16T14:49:56.865 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.865 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:56.865 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.865 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.865 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.865 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:56.865 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:56.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926931+0000", 2024-09-16T14:49:56.866 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.866 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.866 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.866 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926951+0000", 2024-09-16T14:49:56.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9959999999999999e-05 2024-09-16T14:49:56.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926962+0000", 2024-09-16T14:49:56.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1498000000000001e-05 2024-09-16T14:49:56.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.867 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.926970+0000", 2024-09-16T14:49:56.867 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5410000000000003e-06 2024-09-16T14:49:56.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.867 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.867 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.927857+0000", 2024-09-16T14:49:56.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00088758200000000004 2024-09-16T14:49:56.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.927868+0000", 2024-09-16T14:49:56.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1234e-05 2024-09-16T14:49:56.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.868 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.868 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.868 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936355+0000", 2024-09-16T14:49:56.868 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0084864520000000002 2024-09-16T14:49:56.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.869 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.869 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936383+0000", 2024-09-16T14:49:56.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8068e-05 2024-09-16T14:49:56.869 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.869 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.869 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.869 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.869 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:709 3.9s0 3:96d2b183:::smithi04040151-29:head [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:56.869 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.900189+0000", 2024-09-16T14:49:56.869 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 32.892061863999999, 2024-09-16T14:49:56.869 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011070020999999999, 2024-09-16T14:49:56.869 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.870 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:56.870 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:56.870 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:56.870 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.870 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 709 2024-09-16T14:49:56.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.870 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.870 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900189+0000", 2024-09-16T14:49:56.870 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.870 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900189+0000", 2024-09-16T14:49:56.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900188+0000", 2024-09-16T14:49:56.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.871 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.871 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900192+0000", 2024-09-16T14:49:56.871 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7670000000000001e-06 2024-09-16T14:49:56.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.871 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900196+0000", 2024-09-16T14:49:56.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5480000000000003e-06 2024-09-16T14:49:56.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900201+0000", 2024-09-16T14:49:56.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5310000000000003e-06 2024-09-16T14:49:56.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.872 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.872 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.872 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900321+0000", 2024-09-16T14:49:56.872 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000120096 2024-09-16T14:49:56.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:56.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.900327+0000", 2024-09-16T14:49:56.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3389999999999998e-06 2024-09-16T14:49:56.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.907572+0000", 2024-09-16T14:49:56.873 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0072446180000000004 2024-09-16T14:49:56.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.873 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.873 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:56.873 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.907672+0000", 2024-09-16T14:49:56.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9617000000000005e-05 2024-09-16T14:49:56.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.874 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.874 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.874 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.911259+0000", 2024-09-16T14:49:56.874 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0035870350000000001 2024-09-16T14:49:56.874 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.874 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.874 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.875 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.bs3 81/65 ECSubWrite(tid=337, reqid=osd.3.0:337, at_version=81'65, trim_to=0'0, roll_forward_to=80'61))", 2024-09-16T14:49:56.875 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:27.927767+0000", 2024-09-16T14:49:56.875 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 28.864483848999999, 2024-09-16T14:49:56.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0088756419999999996, 2024-09-16T14:49:56.875 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.875 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:56.875 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.875 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.875 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.875 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927767+0000", 2024-09-16T14:49:56.875 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927767+0000", 2024-09-16T14:49:56.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927766+0000", 2024-09-16T14:49:56.876 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:56.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.876 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.876 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.876 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927792+0000", 2024-09-16T14:49:56.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5914999999999998e-05 2024-09-16T14:49:56.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927809+0000", 2024-09-16T14:49:56.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7926000000000001e-05 2024-09-16T14:49:56.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.877 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.927820+0000", 2024-09-16T14:49:56.877 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0277e-05 2024-09-16T14:49:56.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.877 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.877 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930629+0000", 2024-09-16T14:49:56.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00280919 2024-09-16T14:49:56.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930637+0000", 2024-09-16T14:49:56.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3000000000000002e-06 2024-09-16T14:49:56.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.878 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.878 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.936614+0000", 2024-09-16T14:49:56.878 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059770400000000003 2024-09-16T14:49:56.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.878 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.879 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.879 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.936642+0000", 2024-09-16T14:49:56.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7911000000000001e-05 2024-09-16T14:49:56.879 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.879 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.879 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.879 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.879 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:822 3.19s0 3:9b72282b:::smithi04040151-27:head [copy-from ver 52] snapc 3b=[3b,3a,38,37] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T14:49:56.879 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.944334+0000", 2024-09-16T14:49:56.879 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.847915921999999, 2024-09-16T14:49:56.879 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.073023423, 2024-09-16T14:49:56.879 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.879 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:56.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:56.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:56.880 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.880 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 822 2024-09-16T14:49:56.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.880 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.880 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944334+0000", 2024-09-16T14:49:56.880 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.880 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.880 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944334+0000", 2024-09-16T14:49:56.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944332+0000", 2024-09-16T14:49:56.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:56.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.881 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.881 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944343+0000", 2024-09-16T14:49:56.881 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0324e-05 2024-09-16T14:49:56.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.881 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944361+0000", 2024-09-16T14:49:56.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8354000000000001e-05 2024-09-16T14:49:56.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944372+0000", 2024-09-16T14:49:56.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1123e-05 2024-09-16T14:49:56.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.882 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.882 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.882 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958524+0000", 2024-09-16T14:49:56.882 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014151403999999999 2024-09-16T14:49:56.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:56.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958527+0000", 2024-09-16T14:49:56.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.101e-06 2024-09-16T14:49:56.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.970154+0000", 2024-09-16T14:49:56.883 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0116273200000001 2024-09-16T14:49:56.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.883 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.883 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:56.883 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.970333+0000", 2024-09-16T14:49:56.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00017934799999999999 2024-09-16T14:49:56.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.005943+0000", 2024-09-16T14:49:56.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.035610044 2024-09-16T14:49:56.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.884 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.884 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.014246+0000", 2024-09-16T14:49:56.884 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083026430000000002 2024-09-16T14:49:56.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.884 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:56.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017305+0000", 2024-09-16T14:49:56.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003059291 2024-09-16T14:49:56.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.885 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.885 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017358+0000", 2024-09-16T14:49:56.885 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2552000000000001e-05 2024-09-16T14:49:56.885 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.885 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.885 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.885 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.885 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:823 3.19s0 3:9b72282b:::smithi04040151-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90)", 2024-09-16T14:49:56.886 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:56.886 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.847863925999999, 2024-09-16T14:49:56.886 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0666982119999999, 2024-09-16T14:49:56.886 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.886 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:56.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:56.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:56.886 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:56.886 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 823 2024-09-16T14:49:56.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.886 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.886 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.886 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.886 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944386+0000", 2024-09-16T14:49:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.887 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944389+0000", 2024-09-16T14:49:56.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9960000000000002e-06 2024-09-16T14:49:56.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944399+0000", 2024-09-16T14:49:56.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8179999999999999e-06 2024-09-16T14:49:56.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.888 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.888 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.944407+0000", 2024-09-16T14:49:56.888 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.456e-06 2024-09-16T14:49:56.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.888 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958549+0000", 2024-09-16T14:49:56.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014142050999999999 2024-09-16T14:49:56.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:56.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.958550+0000", 2024-09-16T14:49:56.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1200000000000002e-07 2024-09-16T14:49:56.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.889 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.889 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.889 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971413+0000", 2024-09-16T14:49:56.889 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.012862798 2024-09-16T14:49:56.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:56.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971466+0000", 2024-09-16T14:49:56.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3282999999999998e-05 2024-09-16T14:49:56.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010860+0000", 2024-09-16T14:49:56.890 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.039393862000000002 2024-09-16T14:49:56.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.890 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.890 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:56.890 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010968+0000", 2024-09-16T14:49:56.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000108378 2024-09-16T14:49:56.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.891 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.891 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.011085+0000", 2024-09-16T14:49:56.891 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000116402 2024-09-16T14:49:56.891 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.891 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.891 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.891 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.891 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es1 90/27 ECSubWrite(tid=301, reqid=osd.11.0:301, at_version=90'62, trim_to=0'0, roll_forward_to=83'60))", 2024-09-16T14:49:56.891 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.956242+0000", 2024-09-16T14:49:56.891 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.836008875000001, 2024-09-16T14:49:56.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083679859999999991, 2024-09-16T14:49:56.892 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.892 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:56.892 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956242+0000", 2024-09-16T14:49:56.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.892 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.892 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.892 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956242+0000", 2024-09-16T14:49:56.892 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956240+0000", 2024-09-16T14:49:56.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:56.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956267+0000", 2024-09-16T14:49:56.893 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6101999999999998e-05 2024-09-16T14:49:56.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.893 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.893 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.893 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956279+0000", 2024-09-16T14:49:56.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2118e-05 2024-09-16T14:49:56.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956290+0000", 2024-09-16T14:49:56.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1311000000000001e-05 2024-09-16T14:49:56.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.894 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956492+0000", 2024-09-16T14:49:56.894 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000201795 2024-09-16T14:49:56.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.894 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.894 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.956501+0000", 2024-09-16T14:49:56.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1060000000000006e-06 2024-09-16T14:49:56.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.964580+0000", 2024-09-16T14:49:56.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0080795469999999994 2024-09-16T14:49:56.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.895 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.895 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.895 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.964609+0000", 2024-09-16T14:49:56.895 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9043e-05 2024-09-16T14:49:56.895 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.896 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.896 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.896 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.18s4 99/92 ECSubWrite(tid=441, reqid=client.4500.0:905, at_version=99'119, trim_to=0'0, roll_forward_to=98'116))", 2024-09-16T14:49:56.896 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:45.876953+0000", 2024-09-16T14:49:56.896 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 10.915297216999999, 2024-09-16T14:49:56.896 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083132399999999995, 2024-09-16T14:49:56.896 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:56.896 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:56.896 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:56.896 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.896 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:56.896 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876953+0000", 2024-09-16T14:49:56.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:56.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876953+0000", 2024-09-16T14:49:56.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:56.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:56.897 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876952+0000", 2024-09-16T14:49:56.897 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:56.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.897 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.897 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:56.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876973+0000", 2024-09-16T14:49:56.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0709000000000001e-05 2024-09-16T14:49:56.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:56.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.876995+0000", 2024-09-16T14:49:56.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1634e-05 2024-09-16T14:49:56.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.898 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:56.898 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.877003+0000", 2024-09-16T14:49:56.898 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1270000000000003e-06 2024-09-16T14:49:56.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.898 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:56.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.877447+0000", 2024-09-16T14:49:56.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00044436600000000002 2024-09-16T14:49:56.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:56.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.877459+0000", 2024-09-16T14:49:56.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1367e-05 2024-09-16T14:49:56.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.899 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.899 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:56.899 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.885241+0000", 2024-09-16T14:49:56.899 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007781924 2024-09-16T14:49:56.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:56.900 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:56.900 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:56.900 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.885266+0000", 2024-09-16T14:49:56.900 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5795999999999998e-05 2024-09-16T14:49:56.900 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.900 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.900 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.900 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:56.900 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:56.900 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:56.901 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-16T14:49:57.007 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:57.007 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:57.007 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:57.007 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:57.017 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-16T14:49:57.120 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:57.120 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:57.120 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:57.120 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:57.120 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:57.128 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-16T14:49:57.232 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:57.232 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:57.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:57.232 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:57.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.232 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:49:57.232 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:57.232 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.320042799999996, 2024-09-16T14:49:57.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007542137, 2024-09-16T14:49:57.233 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.233 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.233 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:57.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:57.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909881+0000", 2024-09-16T14:49:57.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909917+0000", 2024-09-16T14:49:57.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6513000000000001e-05 2024-09-16T14:49:57.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:49:57.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5784999999999999e-05 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909941+0000", 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1859999999999992e-06 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910004+0000", 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2706999999999994e-05 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.910012+0000", 2024-09-16T14:49:57.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.157e-06 2024-09-16T14:49:57.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917395+0000", 2024-09-16T14:49:57.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073827659999999998 2024-09-16T14:49:57.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917423+0000", 2024-09-16T14:49:57.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.8568000000000002e-05 2024-09-16T14:49:57.236 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.236 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.236 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.237 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s1 37/27 ECSubWrite(tid=115, reqid=osd.3.0:115, at_version=37'32, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:57.237 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:57.237 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.308932455999994, 2024-09-16T14:49:57.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070991120000000003, 2024-09-16T14:49:57.237 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.237 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.237 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:57.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920992+0000", 2024-09-16T14:49:57.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.920991+0000", 2024-09-16T14:49:57.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921010+0000", 2024-09-16T14:49:57.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8529000000000002e-05 2024-09-16T14:49:57.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921018+0000", 2024-09-16T14:49:57.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4670000000000004e-06 2024-09-16T14:49:57.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921025+0000", 2024-09-16T14:49:57.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9330000000000004e-06 2024-09-16T14:49:57.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921076+0000", 2024-09-16T14:49:57.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.0662999999999999e-05 2024-09-16T14:49:57.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921084+0000", 2024-09-16T14:49:57.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4780000000000008e-06 2024-09-16T14:49:57.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928073+0000", 2024-09-16T14:49:57.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069883940000000002 2024-09-16T14:49:57.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928091+0000", 2024-09-16T14:49:57.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7992999999999999e-05 2024-09-16T14:49:57.241 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.241 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.241 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.242 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:57.242 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:57.242 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.304724085999993, 2024-09-16T14:49:57.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079457759999999999, 2024-09-16T14:49:57.242 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.242 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.242 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:57.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:57.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925200+0000", 2024-09-16T14:49:57.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925222+0000", 2024-09-16T14:49:57.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1871000000000001e-05 2024-09-16T14:49:57.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925238+0000", 2024-09-16T14:49:57.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6719999999999999e-05 2024-09-16T14:49:57.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925244+0000", 2024-09-16T14:49:57.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3630000000000002e-06 2024-09-16T14:49:57.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925291+0000", 2024-09-16T14:49:57.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6944000000000003e-05 2024-09-16T14:49:57.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925299+0000", 2024-09-16T14:49:57.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8130000000000002e-06 2024-09-16T14:49:57.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933129+0000", 2024-09-16T14:49:57.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078297330000000002 2024-09-16T14:49:57.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933146+0000", 2024-09-16T14:49:57.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.664e-05 2024-09-16T14:49:57.246 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.246 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.246 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.246 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es5 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:57.246 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:57.246 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.304672451000002, 2024-09-16T14:49:57.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078358790000000005, 2024-09-16T14:49:57.246 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.246 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.246 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925252+0000", 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925251+0000", 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925265+0000", 2024-09-16T14:49:57.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4078e-05 2024-09-16T14:49:57.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.248 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925273+0000", 2024-09-16T14:49:57.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1069999999999999e-06 2024-09-16T14:49:57.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925279+0000", 2024-09-16T14:49:57.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4409999999999998e-06 2024-09-16T14:49:57.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.249 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.249 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925330+0000", 2024-09-16T14:49:57.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1208999999999997e-05 2024-09-16T14:49:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.925339+0000", 2024-09-16T14:49:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8109999999999992e-06 2024-09-16T14:49:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933064+0000", 2024-09-16T14:49:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077252149999999997 2024-09-16T14:49:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.933088+0000", 2024-09-16T14:49:57.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3317999999999999e-05 2024-09-16T14:49:57.251 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.251 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.251 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.251 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:57.251 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933425+0000", 2024-09-16T14:49:57.251 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.296499273000002, 2024-09-16T14:49:57.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0090001820000000007, 2024-09-16T14:49:57.251 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.251 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.251 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933425+0000", 2024-09-16T14:49:57.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933425+0000", 2024-09-16T14:49:57.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933424+0000", 2024-09-16T14:49:57.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933478+0000", 2024-09-16T14:49:57.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.3735e-05 2024-09-16T14:49:57.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933493+0000", 2024-09-16T14:49:57.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4978e-05 2024-09-16T14:49:57.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933502+0000", 2024-09-16T14:49:57.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6509999999999996e-06 2024-09-16T14:49:57.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933761+0000", 2024-09-16T14:49:57.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025924500000000002 2024-09-16T14:49:57.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933772+0000", 2024-09-16T14:49:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0694e-05 2024-09-16T14:49:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942409+0000", 2024-09-16T14:49:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086371539999999993 2024-09-16T14:49:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942425+0000", 2024-09-16T14:49:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5781e-05 2024-09-16T14:49:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.255 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.cs4 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.296137666999996, 2024-09-16T14:49:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086187320000000005, 2024-09-16T14:49:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:57.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:57.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.257 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933786+0000", 2024-09-16T14:49:57.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933807+0000", 2024-09-16T14:49:57.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1107e-05 2024-09-16T14:49:57.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933817+0000", 2024-09-16T14:49:57.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8870000000000004e-06 2024-09-16T14:49:57.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933826+0000", 2024-09-16T14:49:57.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5319999999999995e-06 2024-09-16T14:49:57.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933899+0000", 2024-09-16T14:49:57.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2294999999999998e-05 2024-09-16T14:49:57.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933908+0000", 2024-09-16T14:49:57.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4110000000000002e-06 2024-09-16T14:49:57.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942383+0000", 2024-09-16T14:49:57.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0084747099999999999 2024-09-16T14:49:57.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942405+0000", 2024-09-16T14:49:57.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2399e-05 2024-09-16T14:49:57.260 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.260 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.261 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.261 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 61/27 ECSubWrite(tid=236, reqid=client.4500.0:535, at_version=61'22, trim_to=0'0, roll_forward_to=60'19))", 2024-09-16T14:49:57.261 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.934497+0000", 2024-09-16T14:49:57.261 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.295427513999996, 2024-09-16T14:49:57.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0078697109999999997, 2024-09-16T14:49:57.261 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.261 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.261 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934497+0000", 2024-09-16T14:49:57.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934497+0000", 2024-09-16T14:49:57.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934496+0000", 2024-09-16T14:49:57.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935463+0000", 2024-09-16T14:49:57.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00096685900000000003 2024-09-16T14:49:57.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935498+0000", 2024-09-16T14:49:57.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.5179000000000003e-05 2024-09-16T14:49:57.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935509+0000", 2024-09-16T14:49:57.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1223000000000001e-05 2024-09-16T14:49:57.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935573+0000", 2024-09-16T14:49:57.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4114000000000003e-05 2024-09-16T14:49:57.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.935583+0000", 2024-09-16T14:49:57.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.008e-05 2024-09-16T14:49:57.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942344+0000", 2024-09-16T14:49:57.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067610760000000004 2024-09-16T14:49:57.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.942366+0000", 2024-09-16T14:49:57.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2059e-05 2024-09-16T14:49:57.265 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.265 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.265 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.265 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 61/27 ECSubWrite(tid=242, reqid=osd.3.0:242, at_version=61'79, trim_to=0'0, roll_forward_to=60'77))", 2024-09-16T14:49:57.265 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.944331+0000", 2024-09-16T14:49:57.265 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.285593384000002, 2024-09-16T14:49:57.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0083214770000000007, 2024-09-16T14:49:57.265 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.266 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.266 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944331+0000", 2024-09-16T14:49:57.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944331+0000", 2024-09-16T14:49:57.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944330+0000", 2024-09-16T14:49:57.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944354+0000", 2024-09-16T14:49:57.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3702000000000001e-05 2024-09-16T14:49:57.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944367+0000", 2024-09-16T14:49:57.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3485999999999999e-05 2024-09-16T14:49:57.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944375+0000", 2024-09-16T14:49:57.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3219999999999993e-06 2024-09-16T14:49:57.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944425+0000", 2024-09-16T14:49:57.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9289000000000002e-05 2024-09-16T14:49:57.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.268 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944436+0000", 2024-09-16T14:49:57.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1381e-05 2024-09-16T14:49:57.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.952629+0000", 2024-09-16T14:49:57.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081932840000000003 2024-09-16T14:49:57.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.952652+0000", 2024-09-16T14:49:57.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.296e-05 2024-09-16T14:49:57.269 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.269 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.269 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.270 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 77/27 ECSubWrite(tid=311, reqid=osd.3.0:311, at_version=77'100, trim_to=0'0, roll_forward_to=76'98))", 2024-09-16T14:49:57.270 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.915451+0000", 2024-09-16T14:49:57.270 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.314473419999999, 2024-09-16T14:49:57.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079661820000000005, 2024-09-16T14:49:57.270 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.270 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.270 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915451+0000", 2024-09-16T14:49:57.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915451+0000", 2024-09-16T14:49:57.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915450+0000", 2024-09-16T14:49:57.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915534+0000", 2024-09-16T14:49:57.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4160999999999996e-05 2024-09-16T14:49:57.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915547+0000", 2024-09-16T14:49:57.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2764e-05 2024-09-16T14:49:57.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915555+0000", 2024-09-16T14:49:57.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2090000000000005e-06 2024-09-16T14:49:57.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915659+0000", 2024-09-16T14:49:57.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000103879 2024-09-16T14:49:57.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.915685+0000", 2024-09-16T14:49:57.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5959999999999999e-05 2024-09-16T14:49:57.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923392+0000", 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077072379999999999 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923417+0000", 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5000000000000001e-05 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.0s1 77/65 ECSubWrite(tid=279, reqid=osd.11.0:279, at_version=77'35, trim_to=0'0, roll_forward_to=74'31))", 2024-09-16T14:49:57.274 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.916327+0000", 2024-09-16T14:49:57.275 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.313597358000003, 2024-09-16T14:49:57.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070483610000000004, 2024-09-16T14:49:57.275 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.275 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.275 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916327+0000", 2024-09-16T14:49:57.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916327+0000", 2024-09-16T14:49:57.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916326+0000", 2024-09-16T14:49:57.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916348+0000", 2024-09-16T14:49:57.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2327000000000002e-05 2024-09-16T14:49:57.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916363+0000", 2024-09-16T14:49:57.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4295e-05 2024-09-16T14:49:57.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916371+0000", 2024-09-16T14:49:57.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9519999999999994e-06 2024-09-16T14:49:57.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916417+0000", 2024-09-16T14:49:57.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.6279000000000002e-05 2024-09-16T14:49:57.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916427+0000", 2024-09-16T14:49:57.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0164e-05 2024-09-16T14:49:57.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.278 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.278 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923355+0000", 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069278569999999999 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.923375+0000", 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0341999999999999e-05 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as5 77/72 ECSubWrite(tid=14, reqid=client.4500.0:702, at_version=77'33, trim_to=0'0, roll_forward_to=77'32))", 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.312233657999997, 2024-09-16T14:49:57.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070951110000000003, 2024-09-16T14:49:57.280 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.280 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.280 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:57.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:57.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.917690+0000", 2024-09-16T14:49:57.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918213+0000", 2024-09-16T14:49:57.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000523721 2024-09-16T14:49:57.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918232+0000", 2024-09-16T14:49:57.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8743999999999999e-05 2024-09-16T14:49:57.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918241+0000", 2024-09-16T14:49:57.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9539999999999993e-06 2024-09-16T14:49:57.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918356+0000", 2024-09-16T14:49:57.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011514999999999999 2024-09-16T14:49:57.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.918367+0000", 2024-09-16T14:49:57.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0933e-05 2024-09-16T14:49:57.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.924758+0000", 2024-09-16T14:49:57.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063913499999999996 2024-09-16T14:49:57.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.283 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.283 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.283 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.924786+0000", 2024-09-16T14:49:57.283 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7129000000000001e-05 2024-09-16T14:49:57.283 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.283 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.283 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.284 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 81/27 ECSubWrite(tid=339, reqid=osd.3.0:339, at_version=81'108, trim_to=0'0, roll_forward_to=80'104))", 2024-09-16T14:49:57.284 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:27.930359+0000", 2024-09-16T14:49:57.284 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 29.299565341000001, 2024-09-16T14:49:57.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0085286319999999995, 2024-09-16T14:49:57.284 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.284 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.284 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.284 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.284 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.284 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930359+0000", 2024-09-16T14:49:57.284 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930359+0000", 2024-09-16T14:49:57.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930358+0000", 2024-09-16T14:49:57.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.285 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.285 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930382+0000", 2024-09-16T14:49:57.285 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4034999999999999e-05 2024-09-16T14:49:57.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930393+0000", 2024-09-16T14:49:57.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1062e-05 2024-09-16T14:49:57.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930403+0000", 2024-09-16T14:49:57.286 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8779999999999993e-06 2024-09-16T14:49:57.286 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.286 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.286 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.286 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930675+0000", 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00027152900000000001 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.930681+0000", 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.229e-06 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.938865+0000", 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0081845249999999998 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.287 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.938887+0000", 2024-09-16T14:49:57.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1946e-05 2024-09-16T14:49:57.288 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.288 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.288 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.288 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s2 81/65 ECSubWrite(tid=83, reqid=osd.1.0:83, at_version=81'66, trim_to=0'0, roll_forward_to=80'64))", 2024-09-16T14:49:57.288 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:27.931996+0000", 2024-09-16T14:49:57.288 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 29.297928585000001, 2024-09-16T14:49:57.288 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074671399999999997, 2024-09-16T14:49:57.288 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.288 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.288 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.288 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.931996+0000", 2024-09-16T14:49:57.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.931996+0000", 2024-09-16T14:49:57.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.289 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.289 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.289 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.931995+0000", 2024-09-16T14:49:57.289 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.932025+0000", 2024-09-16T14:49:57.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9884999999999999e-05 2024-09-16T14:49:57.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.932035+0000", 2024-09-16T14:49:57.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0182e-05 2024-09-16T14:49:57.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.290 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.290 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.290 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.932043+0000", 2024-09-16T14:49:57.290 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0900000000000005e-06 2024-09-16T14:49:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.933159+0000", 2024-09-16T14:49:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011161980000000001 2024-09-16T14:49:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.933170+0000", 2024-09-16T14:49:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0925e-05 2024-09-16T14:49:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.291 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.939441+0000", 2024-09-16T14:49:57.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062703339999999998 2024-09-16T14:49:57.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.292 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.292 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:27.939463+0000", 2024-09-16T14:49:57.292 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2005e-05 2024-09-16T14:49:57.292 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.292 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.292 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.292 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.292 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs4 90/27 ECSubWrite(tid=310, reqid=osd.7.0:310, at_version=90'48, trim_to=0'0, roll_forward_to=89'46))", 2024-09-16T14:49:57.292 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:57.292 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.275691137999999, 2024-09-16T14:49:57.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011963438999999999, 2024-09-16T14:49:57.293 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.293 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.293 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:57.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.293 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.293 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.293 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:57.293 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954233+0000", 2024-09-16T14:49:57.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954255+0000", 2024-09-16T14:49:57.294 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2704000000000001e-05 2024-09-16T14:49:57.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.294 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.294 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.294 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954273+0000", 2024-09-16T14:49:57.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.7609999999999999e-05 2024-09-16T14:49:57.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954283+0000", 2024-09-16T14:49:57.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9539999999999999e-06 2024-09-16T14:49:57.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.295 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954340+0000", 2024-09-16T14:49:57.295 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6968000000000001e-05 2024-09-16T14:49:57.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.295 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.954352+0000", 2024-09-16T14:49:57.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1548e-05 2024-09-16T14:49:57.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.966177+0000", 2024-09-16T14:49:57.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011825293000000001 2024-09-16T14:49:57.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.296 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.296 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.296 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:36.966196+0000", 2024-09-16T14:49:57.296 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9636999999999999e-05 2024-09-16T14:49:57.296 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.297 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.297 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.297 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s5 91/90 ECSubWrite(tid=383, reqid=client.4500.0:825, at_version=91'144, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:57.297 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:57.297 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.253969788999999, 2024-09-16T14:49:57.297 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079391419999999997, 2024-09-16T14:49:57.297 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.297 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.297 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.297 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.297 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.297 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:57.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:57.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.298 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.975954+0000", 2024-09-16T14:49:57.298 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.298 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.298 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976430+0000", 2024-09-16T14:49:57.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000476222 2024-09-16T14:49:57.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976441+0000", 2024-09-16T14:49:57.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1457000000000001e-05 2024-09-16T14:49:57.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.299 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.299 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976449+0000", 2024-09-16T14:49:57.299 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.9719999999999992e-06 2024-09-16T14:49:57.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.299 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976957+0000", 2024-09-16T14:49:57.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00050779500000000001 2024-09-16T14:49:57.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976967+0000", 2024-09-16T14:49:57.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0424e-05 2024-09-16T14:49:57.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.300 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.300 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.300 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.983864+0000", 2024-09-16T14:49:57.300 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068963369999999998 2024-09-16T14:49:57.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.301 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.301 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.983894+0000", 2024-09-16T14:49:57.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.9784000000000001e-05 2024-09-16T14:49:57.301 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.301 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.301 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.301 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.301 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.17s5 91/90 ECSubWrite(tid=394, reqid=osd.3.0:391, at_version=0'0, trim_to=0'0, roll_forward_to=91'144))", 2024-09-16T14:49:57.301 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.006074+0000", 2024-09-16T14:49:57.301 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.223850096, 2024-09-16T14:49:57.301 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010644525, 2024-09-16T14:49:57.301 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.302 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.302 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006074+0000", 2024-09-16T14:49:57.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.302 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006074+0000", 2024-09-16T14:49:57.302 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.302 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.302 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006073+0000", 2024-09-16T14:49:57.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006086+0000", 2024-09-16T14:49:57.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2862e-05 2024-09-16T14:49:57.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.303 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.303 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.303 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006102+0000", 2024-09-16T14:49:57.303 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6331e-05 2024-09-16T14:49:57.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006113+0000", 2024-09-16T14:49:57.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0518e-05 2024-09-16T14:49:57.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.304 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006171+0000", 2024-09-16T14:49:57.304 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8326999999999999e-05 2024-09-16T14:49:57.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.304 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.304 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006182+0000", 2024-09-16T14:49:57.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1134999999999999e-05 2024-09-16T14:49:57.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016684+0000", 2024-09-16T14:49:57.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010501664000000001 2024-09-16T14:49:57.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.305 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.305 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.305 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016719+0000", 2024-09-16T14:49:57.305 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4480000000000002e-05 2024-09-16T14:49:57.305 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.306 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.306 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.306 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.19s3 91/90 ECSubWrite(tid=102, reqid=client.4500.0:822, at_version=91'77, trim_to=0'0, roll_forward_to=91'75))", 2024-09-16T14:49:57.306 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.006461+0000", 2024-09-16T14:49:57.306 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.223462916999999, 2024-09-16T14:49:57.306 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010302113, 2024-09-16T14:49:57.306 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.306 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.306 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.306 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.306 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.306 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006461+0000", 2024-09-16T14:49:57.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006461+0000", 2024-09-16T14:49:57.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.307 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006460+0000", 2024-09-16T14:49:57.307 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.307 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.307 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.307 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007931+0000", 2024-09-16T14:49:57.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0014713079999999999 2024-09-16T14:49:57.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007954+0000", 2024-09-16T14:49:57.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2308999999999999e-05 2024-09-16T14:49:57.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.308 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.308 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007970+0000", 2024-09-16T14:49:57.308 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5880999999999999e-05 2024-09-16T14:49:57.308 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008058+0000", 2024-09-16T14:49:57.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8506999999999998e-05 2024-09-16T14:49:57.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008071+0000", 2024-09-16T14:49:57.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2434e-05 2024-09-16T14:49:57.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.309 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.309 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.309 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016742+0000", 2024-09-16T14:49:57.309 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0086718799999999999 2024-09-16T14:49:57.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.310 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.310 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016763+0000", 2024-09-16T14:49:57.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0890000000000002e-05 2024-09-16T14:49:57.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.310 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.310 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.310 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.17s5 91/90 ECSubWrite(tid=395, reqid=osd.3.0:395, at_version=91'146, trim_to=0'0, roll_forward_to=91'144))", 2024-09-16T14:49:57.310 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.008031+0000", 2024-09-16T14:49:57.310 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.221892728, 2024-09-16T14:49:57.310 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0087059660000000007, 2024-09-16T14:49:57.311 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.311 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.311 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008031+0000", 2024-09-16T14:49:57.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.311 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.311 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008031+0000", 2024-09-16T14:49:57.311 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.311 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008030+0000", 2024-09-16T14:49:57.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:57.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008049+0000", 2024-09-16T14:49:57.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9230999999999999e-05 2024-09-16T14:49:57.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.312 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.312 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.312 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008059+0000", 2024-09-16T14:49:57.312 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7729999999999994e-06 2024-09-16T14:49:57.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008066+0000", 2024-09-16T14:49:57.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4849999999999997e-06 2024-09-16T14:49:57.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008113+0000", 2024-09-16T14:49:57.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7154999999999998e-05 2024-09-16T14:49:57.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.313 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.313 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.313 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008124+0000", 2024-09-16T14:49:57.313 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0933e-05 2024-09-16T14:49:57.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016723+0000", 2024-09-16T14:49:57.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0085989640000000006 2024-09-16T14:49:57.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.314 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.314 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.314 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016737+0000", 2024-09-16T14:49:57.314 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4635e-05 2024-09-16T14:49:57.314 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.314 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.314 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.315 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.14s2 91/90 ECSubWrite(tid=103, reqid=osd.1.0:102, at_version=0'0, trim_to=0'0, roll_forward_to=91'76))", 2024-09-16T14:49:57.315 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.009355+0000", 2024-09-16T14:49:57.315 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.220568651000001, 2024-09-16T14:49:57.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074028089999999998, 2024-09-16T14:49:57.315 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.315 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.315 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.315 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.315 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009355+0000", 2024-09-16T14:49:57.315 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.315 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009355+0000", 2024-09-16T14:49:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009354+0000", 2024-09-16T14:49:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009370+0000", 2024-09-16T14:49:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.6163000000000001e-05 2024-09-16T14:49:57.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009384+0000", 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3655000000000001e-05 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009393+0000", 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1919999999999994e-06 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009453+0000", 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9626000000000001e-05 2024-09-16T14:49:57.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009467+0000", 2024-09-16T14:49:57.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4123e-05 2024-09-16T14:49:57.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016732+0000", 2024-09-16T14:49:57.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007265395 2024-09-16T14:49:57.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.318 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.318 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.318 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016758+0000", 2024-09-16T14:49:57.318 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6157e-05 2024-09-16T14:49:57.319 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.319 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.319 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.319 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.ds2 95/65 ECSubWrite(tid=335, reqid=osd.7.0:333, at_version=0'0, trim_to=0'0, roll_forward_to=95'81))", 2024-09-16T14:49:57.319 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:41.883865+0000", 2024-09-16T14:49:57.319 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 15.346058877000001, 2024-09-16T14:49:57.319 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0072428129999999999, 2024-09-16T14:49:57.319 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.319 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.319 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.319 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.319 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883865+0000", 2024-09-16T14:49:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883865+0000", 2024-09-16T14:49:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883865+0000", 2024-09-16T14:49:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.320 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883874+0000", 2024-09-16T14:49:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7290000000000006e-06 2024-09-16T14:49:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883882+0000", 2024-09-16T14:49:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4859999999999997e-06 2024-09-16T14:49:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883889+0000", 2024-09-16T14:49:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3309999999999998e-06 2024-09-16T14:49:57.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883986+0000", 2024-09-16T14:49:57.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6576999999999994e-05 2024-09-16T14:49:57.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.883994+0000", 2024-09-16T14:49:57.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0139999999999998e-06 2024-09-16T14:49:57.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.322 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.322 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.322 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.891085+0000", 2024-09-16T14:49:57.322 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0070908899999999999 2024-09-16T14:49:57.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.323 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.323 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:41.891108+0000", 2024-09-16T14:49:57.323 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3181999999999999e-05 2024-09-16T14:49:57.323 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.323 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.323 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.323 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.323 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.323 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:57.325 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-16T14:49:57.427 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:57.427 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:57.427 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:57.427 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:57.435 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-16T14:49:57.536 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:57.537 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:57.537 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:57.537 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:57.537 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:57.545 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-16T14:49:57.647 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 34/27 ECSubWrite(tid=81, reqid=client.4500.0:234, at_version=34'22, trim_to=0'0, roll_forward_to=31'19))", 2024-09-16T14:49:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:57.648 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.736216006999996, 2024-09-16T14:49:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0079855010000000008, 2024-09-16T14:49:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.649 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909484+0000", 2024-09-16T14:49:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.650 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909483+0000", 2024-09-16T14:49:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909521+0000", 2024-09-16T14:49:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7185999999999999e-05 2024-09-16T14:49:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909543+0000", 2024-09-16T14:49:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2802999999999998e-05 2024-09-16T14:49:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.651 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909551+0000", 2024-09-16T14:49:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5039999999999996e-06 2024-09-16T14:49:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909927+0000", 2024-09-16T14:49:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00037637400000000001 2024-09-16T14:49:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.652 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.909933+0000", 2024-09-16T14:49:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2040000000000002e-06 2024-09-16T14:49:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917442+0000", 2024-09-16T14:49:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007508659 2024-09-16T14:49:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917469+0000", 2024-09-16T14:49:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7197999999999999e-05 2024-09-16T14:49:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.653 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 34/27 ECSubWrite(tid=168, reqid=osd.7.0:168, at_version=34'8, trim_to=0'0, roll_forward_to=33'6))", 2024-09-16T14:49:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.732143660999995, 2024-09-16T14:49:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067701159999999996, 2024-09-16T14:49:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.654 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913556+0000", 2024-09-16T14:49:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913579+0000", 2024-09-16T14:49:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2829000000000001e-05 2024-09-16T14:49:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913591+0000", 2024-09-16T14:49:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2264e-05 2024-09-16T14:49:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.656 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913600+0000", 2024-09-16T14:49:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9959999999999999e-06 2024-09-16T14:49:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913648+0000", 2024-09-16T14:49:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7759000000000001e-05 2024-09-16T14:49:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.913657+0000", 2024-09-16T14:49:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6709999999999994e-06 2024-09-16T14:49:57.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920296+0000", 2024-09-16T14:49:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066385120000000001 2024-09-16T14:49:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920326+0000", 2024-09-16T14:49:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.0405000000000001e-05 2024-09-16T14:49:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.658 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 34/27 ECSubWrite(tid=74, reqid=client.4500.0:229, at_version=34'6, trim_to=0'0, roll_forward_to=30'4))", 2024-09-16T14:49:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.731327703000005, 2024-09-16T14:49:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060056090000000003, 2024-09-16T14:49:57.659 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.660 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914372+0000", 2024-09-16T14:49:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914468+0000", 2024-09-16T14:49:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6398999999999996e-05 2024-09-16T14:49:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914483+0000", 2024-09-16T14:49:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5238e-05 2024-09-16T14:49:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.662 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.663 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.663 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914494+0000", 2024-09-16T14:49:57.663 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0192e-05 2024-09-16T14:49:57.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.663 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914622+0000", 2024-09-16T14:49:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00012855599999999999 2024-09-16T14:49:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.914635+0000", 2024-09-16T14:49:57.664 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2510000000000001e-05 2024-09-16T14:49:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920339+0000", 2024-09-16T14:49:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057041549999999998 2024-09-16T14:49:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920378+0000", 2024-09-16T14:49:57.665 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.8918999999999998e-05 2024-09-16T14:49:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 34/27 ECSubWrite(tid=26, reqid=osd.1.0:26, at_version=34'16, trim_to=0'0, roll_forward_to=32'14))", 2024-09-16T14:49:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 76.729314927000004, 2024-09-16T14:49:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069011319999999999, 2024-09-16T14:49:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.666 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.667 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916385+0000", 2024-09-16T14:49:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:49:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916410+0000", 2024-09-16T14:49:57.668 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5343000000000001e-05 2024-09-16T14:49:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916418+0000", 2024-09-16T14:49:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2360000000000004e-06 2024-09-16T14:49:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916428+0000", 2024-09-16T14:49:57.669 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6779999999999995e-06 2024-09-16T14:49:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916492+0000", 2024-09-16T14:49:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4219000000000003e-05 2024-09-16T14:49:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.670 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.916502+0000", 2024-09-16T14:49:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6690000000000001e-06 2024-09-16T14:49:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923254+0000", 2024-09-16T14:49:57.671 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0067518070000000003 2024-09-16T14:49:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.923286+0000", 2024-09-16T14:49:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2367999999999999e-05 2024-09-16T14:49:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.672 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.3s7 37/27 ECSubWrite(tid=16, reqid=osd.4.0:16, at_version=37'14, trim_to=0'0, roll_forward_to=35'12))", 2024-09-16T14:49:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.724010543000006, 2024-09-16T14:49:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006972692, 2024-09-16T14:49:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.673 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.674 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.675 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921689+0000", 2024-09-16T14:49:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.676 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921709+0000", 2024-09-16T14:49:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0411000000000002e-05 2024-09-16T14:49:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921719+0000", 2024-09-16T14:49:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2650000000000002e-06 2024-09-16T14:49:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.677 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921726+0000", 2024-09-16T14:49:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8770000000000004e-06 2024-09-16T14:49:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921773+0000", 2024-09-16T14:49:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7435999999999998e-05 2024-09-16T14:49:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.678 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.921782+0000", 2024-09-16T14:49:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.6039999999999993e-06 2024-09-16T14:49:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928635+0000", 2024-09-16T14:49:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068533129999999998 2024-09-16T14:49:57.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928662+0000", 2024-09-16T14:49:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7089000000000001e-05 2024-09-16T14:49:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.680 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.2s7 37/27 ECSubWrite(tid=117, reqid=osd.3.0:117, at_version=37'34, trim_to=0'0, roll_forward_to=35'30))", 2024-09-16T14:49:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.723514054999995, 2024-09-16T14:49:57.681 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064783530000000001, 2024-09-16T14:49:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.682 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922186+0000", 2024-09-16T14:49:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922185+0000", 2024-09-16T14:49:57.683 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922205+0000", 2024-09-16T14:49:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9871e-05 2024-09-16T14:49:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.684 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922219+0000", 2024-09-16T14:49:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4266e-05 2024-09-16T14:49:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922229+0000", 2024-09-16T14:49:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5619999999999993e-06 2024-09-16T14:49:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.685 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922270+0000", 2024-09-16T14:49:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0917000000000001e-05 2024-09-16T14:49:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.922278+0000", 2024-09-16T14:49:57.686 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3890000000000005e-06 2024-09-16T14:49:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.687 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928630+0000", 2024-09-16T14:49:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063524439999999996 2024-09-16T14:49:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.928664+0000", 2024-09-16T14:49:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3651e-05 2024-09-16T14:49:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s5 37/27 ECSubWrite(tid=24, reqid=osd.0.0:23, at_version=0'0, trim_to=0'0, roll_forward_to=37'17))", 2024-09-16T14:49:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.722065650999994, 2024-09-16T14:49:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069615850000000002, 2024-09-16T14:49:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.689 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923634+0000", 2024-09-16T14:49:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.690 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923633+0000", 2024-09-16T14:49:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923644+0000", 2024-09-16T14:49:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.08e-05 2024-09-16T14:49:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923657+0000", 2024-09-16T14:49:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2836999999999999e-05 2024-09-16T14:49:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.691 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923664+0000", 2024-09-16T14:49:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.1470000000000004e-06 2024-09-16T14:49:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923885+0000", 2024-09-16T14:49:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00022155800000000001 2024-09-16T14:49:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.692 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.923894+0000", 2024-09-16T14:49:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9260000000000006e-06 2024-09-16T14:49:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930571+0000", 2024-09-16T14:49:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066770420000000002 2024-09-16T14:49:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930596+0000", 2024-09-16T14:49:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4383e-05 2024-09-16T14:49:57.693 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.as3 37/27 ECSubWrite(tid=104, reqid=osd.11.0:104, at_version=37'10, trim_to=0'0, roll_forward_to=35'8))", 2024-09-16T14:49:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.721319414999996, 2024-09-16T14:49:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062471519999999997, 2024-09-16T14:49:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.694 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924380+0000", 2024-09-16T14:49:57.695 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924405+0000", 2024-09-16T14:49:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5147000000000002e-05 2024-09-16T14:49:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924414+0000", 2024-09-16T14:49:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4810000000000006e-06 2024-09-16T14:49:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.696 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924421+0000", 2024-09-16T14:49:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0899999999999999e-06 2024-09-16T14:49:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924468+0000", 2024-09-16T14:49:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.727e-05 2024-09-16T14:49:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.697 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924477+0000", 2024-09-16T14:49:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7360000000000007e-06 2024-09-16T14:49:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930601+0000", 2024-09-16T14:49:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0061242529999999996 2024-09-16T14:49:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.698 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930628+0000", 2024-09-16T14:49:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6517e-05 2024-09-16T14:49:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 37/27 ECSubWrite(tid=105, reqid=osd.11.0:105, at_version=37'11, trim_to=0'0, roll_forward_to=35'9))", 2024-09-16T14:49:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.721069211, 2024-09-16T14:49:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006019328, 2024-09-16T14:49:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.699 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:57.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.700 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.700 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.700 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924631+0000", 2024-09-16T14:49:57.700 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924630+0000", 2024-09-16T14:49:57.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.701 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924650+0000", 2024-09-16T14:49:57.701 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0024e-05 2024-09-16T14:49:57.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.701 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.701 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924660+0000", 2024-09-16T14:49:57.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9219999999999992e-06 2024-09-16T14:49:57.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924668+0000", 2024-09-16T14:49:57.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6820000000000003e-06 2024-09-16T14:49:57.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.702 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.702 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924711+0000", 2024-09-16T14:49:57.702 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.3009000000000001e-05 2024-09-16T14:49:57.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.702 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.924720+0000", 2024-09-16T14:49:57.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9290000000000004e-06 2024-09-16T14:49:57.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.703 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.703 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.703 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930634+0000", 2024-09-16T14:49:57.703 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059144810000000001 2024-09-16T14:49:57.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.704 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.704 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:43.930650+0000", 2024-09-16T14:49:57.704 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5688999999999998e-05 2024-09-16T14:49:57.704 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.704 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.704 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.704 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.704 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s4 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:57.704 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:57.705 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.712700091999999, 2024-09-16T14:49:57.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062137989999999999, 2024-09-16T14:49:57.705 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.705 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.705 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.705 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.705 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:57.705 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933000+0000", 2024-09-16T14:49:57.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.706 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.932999+0000", 2024-09-16T14:49:57.706 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.706 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.706 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933022+0000", 2024-09-16T14:49:57.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3518000000000001e-05 2024-09-16T14:49:57.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.707 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.707 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.707 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933035+0000", 2024-09-16T14:49:57.707 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2565000000000001e-05 2024-09-16T14:49:57.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933044+0000", 2024-09-16T14:49:57.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3009999999999995e-06 2024-09-16T14:49:57.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.708 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.708 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.708 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934016+0000", 2024-09-16T14:49:57.708 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00097159700000000004 2024-09-16T14:49:57.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.709 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.934022+0000", 2024-09-16T14:49:57.709 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4869999999999999e-06 2024-09-16T14:49:57.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.709 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939202+0000", 2024-09-16T14:49:57.710 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051795820000000003 2024-09-16T14:49:57.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.710 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.710 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.710 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939214+0000", 2024-09-16T14:49:57.711 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1663e-05 2024-09-16T14:49:57.711 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.711 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.711 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.711 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.712 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.cs3 61/27 ECSubWrite(tid=95, reqid=osd.5.0:95, at_version=61'33, trim_to=0'0, roll_forward_to=60'29))", 2024-09-16T14:49:57.712 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:57.712 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 49.712233556999998, 2024-09-16T14:49:57.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0057306709999999997, 2024-09-16T14:49:57.712 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.712 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.712 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.712 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.712 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:57.712 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933466+0000", 2024-09-16T14:49:57.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.713 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.713 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933465+0000", 2024-09-16T14:49:57.713 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.713 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933489+0000", 2024-09-16T14:49:57.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3940000000000001e-05 2024-09-16T14:49:57.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.714 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.714 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933501+0000", 2024-09-16T14:49:57.714 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1932e-05 2024-09-16T14:49:57.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.714 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933510+0000", 2024-09-16T14:49:57.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.5820000000000002e-06 2024-09-16T14:49:57.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936680+0000", 2024-09-16T14:49:57.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003169893 2024-09-16T14:49:57.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.715 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.715 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.715 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.936689+0000", 2024-09-16T14:49:57.715 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.5810000000000003e-06 2024-09-16T14:49:57.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939176+0000", 2024-09-16T14:49:57.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002486238 2024-09-16T14:49:57.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.716 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.716 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.939197+0000", 2024-09-16T14:49:57.716 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1325e-05 2024-09-16T14:49:57.716 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.717 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.717 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.717 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.717 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 62/27 ECSubWrite(tid=244, reqid=osd.7.0:244, at_version=62'27, trim_to=0'0, roll_forward_to=61'25))", 2024-09-16T14:49:57.717 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:57.717 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.70287055, 2024-09-16T14:49:57.717 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.007111081, 2024-09-16T14:49:57.717 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.717 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.717 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:57.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.718 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.718 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:57.718 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942829+0000", 2024-09-16T14:49:57.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942855+0000", 2024-09-16T14:49:57.719 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5999999999999998e-05 2024-09-16T14:49:57.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.719 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.719 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.719 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942867+0000", 2024-09-16T14:49:57.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2481e-05 2024-09-16T14:49:57.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.720 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942879+0000", 2024-09-16T14:49:57.720 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1838000000000001e-05 2024-09-16T14:49:57.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.720 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.720 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942938+0000", 2024-09-16T14:49:57.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8282000000000002e-05 2024-09-16T14:49:57.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.721 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.942948+0000", 2024-09-16T14:49:57.721 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0497e-05 2024-09-16T14:49:57.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.721 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.721 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949905+0000", 2024-09-16T14:49:57.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0069572940000000002 2024-09-16T14:49:57.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.722 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.722 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.722 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949940+0000", 2024-09-16T14:49:57.722 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.4965999999999997e-05 2024-09-16T14:49:57.722 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.722 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.723 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.723 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 62/27 ECSubWrite(tid=234, reqid=osd.11.0:234, at_version=62'39, trim_to=0'0, roll_forward_to=61'37))", 2024-09-16T14:49:57.723 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:57.723 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 48.702437066000002, 2024-09-16T14:49:57.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066952440000000004, 2024-09-16T14:49:57.723 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.723 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.723 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.723 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.723 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.723 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:57.723 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943263+0000", 2024-09-16T14:49:57.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.724 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.724 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943262+0000", 2024-09-16T14:49:57.724 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943285+0000", 2024-09-16T14:49:57.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2650000000000002e-05 2024-09-16T14:49:57.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.725 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.725 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.725 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943301+0000", 2024-09-16T14:49:57.725 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5398000000000001e-05 2024-09-16T14:49:57.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943309+0000", 2024-09-16T14:49:57.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9269999999999994e-06 2024-09-16T14:49:57.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.726 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943373+0000", 2024-09-16T14:49:57.726 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3194999999999994e-05 2024-09-16T14:49:57.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.726 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.943383+0000", 2024-09-16T14:49:57.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0778000000000001e-05 2024-09-16T14:49:57.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949934+0000", 2024-09-16T14:49:57.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065509790000000002 2024-09-16T14:49:57.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.727 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.727 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.727 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:08.949958+0000", 2024-09-16T14:49:57.727 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3598999999999999e-05 2024-09-16T14:49:57.727 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.727 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.728 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.728 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.728 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 76/27 ECSubWrite(tid=303, reqid=client.4500.0:699, at_version=76'98, trim_to=0'0, roll_forward_to=76'95))", 2024-09-16T14:49:57.728 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:57.728 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 34.735666520999999, 2024-09-16T14:49:57.728 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006028032, 2024-09-16T14:49:57.728 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.728 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.729 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:57.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910033+0000", 2024-09-16T14:49:57.729 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.729 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.729 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.729 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910032+0000", 2024-09-16T14:49:57.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910053+0000", 2024-09-16T14:49:57.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0665e-05 2024-09-16T14:49:57.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.730 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910067+0000", 2024-09-16T14:49:57.730 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4416000000000001e-05 2024-09-16T14:49:57.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.730 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.730 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910077+0000", 2024-09-16T14:49:57.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4509999999999998e-06 2024-09-16T14:49:57.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910367+0000", 2024-09-16T14:49:57.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00029006799999999999 2024-09-16T14:49:57.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.731 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.731 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.910378+0000", 2024-09-16T14:49:57.731 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0628000000000001e-05 2024-09-16T14:49:57.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.731 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.916036+0000", 2024-09-16T14:49:57.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0056580270000000004 2024-09-16T14:49:57.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.732 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.732 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:22.916061+0000", 2024-09-16T14:49:57.732 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.5658000000000001e-05 2024-09-16T14:49:57.732 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.732 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.732 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.732 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.733 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.es7 77/27 ECSubWrite(tid=277, reqid=osd.11.0:277, at_version=77'54, trim_to=0'0, roll_forward_to=75'52))", 2024-09-16T14:49:57.733 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:57.733 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 33.735179152999997, 2024-09-16T14:49:57.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064553950000000001, 2024-09-16T14:49:57.733 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.733 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.733 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.733 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.733 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:57.733 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.733 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910521+0000", 2024-09-16T14:49:57.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.734 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910520+0000", 2024-09-16T14:49:57.734 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967296 2024-09-16T14:49:57.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.734 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.734 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910554+0000", 2024-09-16T14:49:57.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.3111999999999997e-05 2024-09-16T14:49:57.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910578+0000", 2024-09-16T14:49:57.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4757999999999999e-05 2024-09-16T14:49:57.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.735 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.735 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.735 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910590+0000", 2024-09-16T14:49:57.735 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2156e-05 2024-09-16T14:49:57.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910646+0000", 2024-09-16T14:49:57.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5979000000000001e-05 2024-09-16T14:49:57.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.910656+0000", 2024-09-16T14:49:57.736 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9410000000000002e-06 2024-09-16T14:49:57.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.736 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.736 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.736 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916943+0000", 2024-09-16T14:49:57.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062870260000000002 2024-09-16T14:49:57.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.737 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.737 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:23.916976+0000", 2024-09-16T14:49:57.737 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2641999999999997e-05 2024-09-16T14:49:57.737 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.737 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.737 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.737 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.737 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.fs5 91/27 ECSubWrite(tid=316, reqid=osd.7.0:315, at_version=0'0, trim_to=0'0, roll_forward_to=91'50))", 2024-09-16T14:49:57.737 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.963325+0000", 2024-09-16T14:49:57.737 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.682375061999998, 2024-09-16T14:49:57.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0060303650000000002, 2024-09-16T14:49:57.738 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.738 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.738 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963325+0000", 2024-09-16T14:49:57.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.738 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.738 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.738 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963325+0000", 2024-09-16T14:49:57.738 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.739 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963324+0000", 2024-09-16T14:49:57.739 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.739 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.739 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963338+0000", 2024-09-16T14:49:57.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3813e-05 2024-09-16T14:49:57.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963350+0000", 2024-09-16T14:49:57.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2512e-05 2024-09-16T14:49:57.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.740 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.740 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.740 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963360+0000", 2024-09-16T14:49:57.740 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0141e-05 2024-09-16T14:49:57.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963425+0000", 2024-09-16T14:49:57.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4579999999999995e-05 2024-09-16T14:49:57.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.741 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.741 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.741 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.963437+0000", 2024-09-16T14:49:57.741 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1780000000000001e-05 2024-09-16T14:49:57.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.969335+0000", 2024-09-16T14:49:57.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0058980500000000002 2024-09-16T14:49:57.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.742 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.742 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.969355+0000", 2024-09-16T14:49:57.742 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0489e-05 2024-09-16T14:49:57.742 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.742 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.742 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.743 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.7s4 91/90 ECSubWrite(tid=383, reqid=client.4500.0:825, at_version=91'144, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:57.743 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.976070+0000", 2024-09-16T14:49:57.743 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.669629854, 2024-09-16T14:49:57.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066451510000000002, 2024-09-16T14:49:57.743 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.743 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.743 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.743 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.743 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.743 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976070+0000", 2024-09-16T14:49:57.743 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976070+0000", 2024-09-16T14:49:57.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.744 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976069+0000", 2024-09-16T14:49:57.744 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:57.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.744 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.744 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.976953+0000", 2024-09-16T14:49:57.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00088349499999999998 2024-09-16T14:49:57.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.745 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.745 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.745 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977007+0000", 2024-09-16T14:49:57.745 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4676000000000003e-05 2024-09-16T14:49:57.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977083+0000", 2024-09-16T14:49:57.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5831e-05 2024-09-16T14:49:57.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.746 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.746 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977120+0000", 2024-09-16T14:49:57.746 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.6887000000000001e-05 2024-09-16T14:49:57.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.746 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.977129+0000", 2024-09-16T14:49:57.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9120000000000004e-06 2024-09-16T14:49:57.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.982689+0000", 2024-09-16T14:49:57.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0055597590000000001 2024-09-16T14:49:57.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.747 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.747 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.747 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.982715+0000", 2024-09-16T14:49:57.747 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6478999999999998e-05 2024-09-16T14:49:57.747 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.748 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.748 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.748 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.19s7 91/90 ECSubWrite(tid=102, reqid=client.4500.0:822, at_version=91'77, trim_to=0'0, roll_forward_to=91'75))", 2024-09-16T14:49:57.748 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.007185+0000", 2024-09-16T14:49:57.748 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.638515120000001, 2024-09-16T14:49:57.748 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0068803600000000003, 2024-09-16T14:49:57.748 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.748 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.748 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007185+0000", 2024-09-16T14:49:57.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.749 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.749 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007185+0000", 2024-09-16T14:49:57.749 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.749 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007183+0000", 2024-09-16T14:49:57.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:57.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008457+0000", 2024-09-16T14:49:57.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001274025 2024-09-16T14:49:57.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.750 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.750 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.750 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008473+0000", 2024-09-16T14:49:57.750 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5605999999999999e-05 2024-09-16T14:49:57.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008483+0000", 2024-09-16T14:49:57.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.8430000000000005e-06 2024-09-16T14:49:57.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008541+0000", 2024-09-16T14:49:57.751 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7911999999999997e-05 2024-09-16T14:49:57.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.751 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.751 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.008551+0000", 2024-09-16T14:49:57.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0353999999999999e-05 2024-09-16T14:49:57.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.014044+0000", 2024-09-16T14:49:57.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0054927439999999999 2024-09-16T14:49:57.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.752 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.752 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.752 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.014065+0000", 2024-09-16T14:49:57.752 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1124999999999998e-05 2024-09-16T14:49:57.752 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.753 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.753 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.753 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.753 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.17s4 91/90 ECSubWrite(tid=395, reqid=osd.3.0:395, at_version=91'146, trim_to=0'0, roll_forward_to=91'144))", 2024-09-16T14:49:57.753 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:57.753 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 19.636245502000001, 2024-09-16T14:49:57.753 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0076634049999999999, 2024-09-16T14:49:57.753 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.753 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.754 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.754 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:57.754 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.754 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.756 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:57.756 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009454+0000", 2024-09-16T14:49:57.757 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.757 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.757 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.757 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009478+0000", 2024-09-16T14:49:57.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4235000000000001e-05 2024-09-16T14:49:57.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009486+0000", 2024-09-16T14:49:57.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7940000000000003e-06 2024-09-16T14:49:57.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.758 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.758 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.758 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.009492+0000", 2024-09-16T14:49:57.758 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7980000000000002e-06 2024-09-16T14:49:57.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010420+0000", 2024-09-16T14:49:57.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00092807100000000004 2024-09-16T14:49:57.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.010429+0000", 2024-09-16T14:49:57.759 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8389999999999996e-06 2024-09-16T14:49:57.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.759 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.759 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017094+0000", 2024-09-16T14:49:57.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0066650879999999996 2024-09-16T14:49:57.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.760 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.760 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.760 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.017118+0000", 2024-09-16T14:49:57.760 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.3941e-05 2024-09-16T14:49:57.760 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.760 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.760 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.761 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.6s5 104/90 ECSubWrite(tid=128, reqid=client.4500.0:971, at_version=104'50, trim_to=0'0, roll_forward_to=91'48))", 2024-09-16T14:49:57.761 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:50.871724+0000", 2024-09-16T14:49:57.761 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.7739755170000002, 2024-09-16T14:49:57.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0074684510000000001, 2024-09-16T14:49:57.761 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:57.761 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:57.761 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:57.761 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.761 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:57.761 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.871724+0000", 2024-09-16T14:49:57.761 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:57.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.871724+0000", 2024-09-16T14:49:57.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:57.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.762 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.762 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:57.762 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.871724+0000", 2024-09-16T14:49:57.762 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:57.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:57.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.872414+0000", 2024-09-16T14:49:57.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00068969599999999995 2024-09-16T14:49:57.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:57.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.872429+0000", 2024-09-16T14:49:57.763 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5865000000000001e-05 2024-09-16T14:49:57.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.763 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.763 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:57.763 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.872440+0000", 2024-09-16T14:49:57.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0771000000000001e-05 2024-09-16T14:49:57.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:57.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.872501+0000", 2024-09-16T14:49:57.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1230999999999997e-05 2024-09-16T14:49:57.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.764 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:57.764 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.872515+0000", 2024-09-16T14:49:57.764 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3448000000000001e-05 2024-09-16T14:49:57.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.764 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:57.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.879160+0000", 2024-09-16T14:49:57.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.006645349 2024-09-16T14:49:57.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:57.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:57.765 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:57.765 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:50.879193+0000", 2024-09-16T14:49:57.765 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.2490000000000002e-05 2024-09-16T14:49:57.765 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.765 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.765 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.766 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:57.766 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:57.766 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:57.767 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-16T14:49:57.892 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:57.892 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:57.892 INFO:teuthology.orchestra.run.smithi114.stdout: "num_ops": 0 2024-09-16T14:49:57.893 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:57.901 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-16T14:49:58.010 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:58.010 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [], 2024-09-16T14:49:58.010 INFO:teuthology.orchestra.run.smithi114.stdout: "complaint_time": 30, 2024-09-16T14:49:58.010 INFO:teuthology.orchestra.run.smithi114.stdout: "num_blocked_ops": 0 2024-09-16T14:49:58.010 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:58.018 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-16T14:49:58.074 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T14:49:58.125 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-16T14:49:58.125 INFO:teuthology.orchestra.run.smithi114.stdout: "size": 20, 2024-09-16T14:49:58.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 600, 2024-09-16T14:49:58.125 INFO:teuthology.orchestra.run.smithi114.stdout: "ops": [ 2024-09-16T14:49:58.125 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.125 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:229 3.es0 3:7ecfd044:::smithi04040151-35:head [copy-from ver 4] snapc 4=[4,2] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-16T14:49:58.125 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:58.125 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 77.218412146999995, 2024-09-16T14:49:58.125 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016506461, 2024-09-16T14:49:58.125 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.125 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.126 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.126 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 229 2024-09-16T14:49:58.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.126 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:58.126 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.126 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.126 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.126 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904170+0000", 2024-09-16T14:49:58.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904168+0000", 2024-09-16T14:49:58.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:58.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.127 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904187+0000", 2024-09-16T14:49:58.127 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9449999999999998e-05 2024-09-16T14:49:58.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.127 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.127 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904206+0000", 2024-09-16T14:49:58.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.8698e-05 2024-09-16T14:49:58.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904215+0000", 2024-09-16T14:49:58.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.7779999999999996e-06 2024-09-16T14:49:58.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.128 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.128 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904276+0000", 2024-09-16T14:49:58.128 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1130999999999995e-05 2024-09-16T14:49:58.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:58.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.904284+0000", 2024-09-16T14:49:58.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0609999999999996e-06 2024-09-16T14:49:58.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.129 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.129 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.129 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906272+0000", 2024-09-16T14:49:58.129 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0019884210000000002 2024-09-16T14:49:58.129 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.906391+0000", 2024-09-16T14:49:58.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000118843 2024-09-16T14:49:58.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.130 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.912595+0000", 2024-09-16T14:49:58.130 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0062038190000000002 2024-09-16T14:49:58.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.130 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.130 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.917584+0000", 2024-09-16T14:49:58.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0049889069999999999 2024-09-16T14:49:58.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:58.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920592+0000", 2024-09-16T14:49:58.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030081639999999998 2024-09-16T14:49:58.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.131 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.131 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.131 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:40.920676+0000", 2024-09-16T14:49:58.131 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.3974999999999995e-05 2024-09-16T14:49:58.131 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.131 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.132 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.132 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.132 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:298 3.es0 3:7ecfd044:::smithi04040151-35:head [append 0~720896] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:58.132 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:58.132 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.193748331999998, 2024-09-16T14:49:58.132 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014187808, 2024-09-16T14:49:58.132 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.132 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.132 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.132 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.132 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.132 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 298 2024-09-16T14:49:58.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.133 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:58.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.133 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.133 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.133 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928834+0000", 2024-09-16T14:49:58.133 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.928832+0000", 2024-09-16T14:49:58.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:58.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.134 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929746+0000", 2024-09-16T14:49:58.134 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00091434199999999995 2024-09-16T14:49:58.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.134 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.134 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929760+0000", 2024-09-16T14:49:58.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.3519e-05 2024-09-16T14:49:58.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929767+0000", 2024-09-16T14:49:58.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3180000000000001e-06 2024-09-16T14:49:58.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.135 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.135 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929849+0000", 2024-09-16T14:49:58.135 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2020000000000004e-05 2024-09-16T14:49:58.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.135 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:58.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.929854+0000", 2024-09-16T14:49:58.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1209999999999998e-06 2024-09-16T14:49:58.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931714+0000", 2024-09-16T14:49:58.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001859989 2024-09-16T14:49:58.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.136 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.136 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.136 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931845+0000", 2024-09-16T14:49:58.136 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000130901 2024-09-16T14:49:58.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933089+0000", 2024-09-16T14:49:58.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001243535 2024-09-16T14:49:58.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.137 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936370+0000", 2024-09-16T14:49:58.137 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032817660000000002 2024-09-16T14:49:58.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.137 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:58.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.942945+0000", 2024-09-16T14:49:58.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065741560000000003 2024-09-16T14:49:58.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.138 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.138 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943021+0000", 2024-09-16T14:49:58.138 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6734000000000001e-05 2024-09-16T14:49:58.138 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.138 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.138 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.138 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.138 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:299 3.es0 3:7ecfd044:::smithi04040151-35:head [append 0~376832] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:58.139 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:58.139 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.191458531999999, 2024-09-16T14:49:58.139 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.012149627, 2024-09-16T14:49:58.139 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.139 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.139 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.139 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.139 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.139 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 299 2024-09-16T14:49:58.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.139 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.139 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.139 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.139 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:58.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.140 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.140 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931123+0000", 2024-09-16T14:49:58.140 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.140 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931122+0000", 2024-09-16T14:49:58.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:58.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931722+0000", 2024-09-16T14:49:58.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00059944100000000004 2024-09-16T14:49:58.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.141 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.141 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.141 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931732+0000", 2024-09-16T14:49:58.141 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9369999999999999e-06 2024-09-16T14:49:58.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931739+0000", 2024-09-16T14:49:58.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2819999999999999e-06 2024-09-16T14:49:58.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.142 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933835+0000", 2024-09-16T14:49:58.142 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020961140000000001 2024-09-16T14:49:58.142 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.142 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.142 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.933929+0000", 2024-09-16T14:49:58.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.3683000000000004e-05 2024-09-16T14:49:58.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.934917+0000", 2024-09-16T14:49:58.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00098803200000000002 2024-09-16T14:49:58.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.143 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.143 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.143 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.939558+0000", 2024-09-16T14:49:58.143 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00464175 2024-09-16T14:49:58.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:58.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943205+0000", 2024-09-16T14:49:58.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0036465820000000002 2024-09-16T14:49:58.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.144 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.144 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.144 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943273+0000", 2024-09-16T14:49:58.144 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.7917999999999996e-05 2024-09-16T14:49:58.144 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.144 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.144 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.145 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.145 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:300 3.es0 3:7ecfd044:::smithi04040151-35:head [setxattr _header (56) in=7b] snapc 7=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:58.145 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:58.145 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.190832438000001, 2024-09-16T14:49:58.145 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011740911, 2024-09-16T14:49:58.145 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.145 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.145 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.145 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 300 2024-09-16T14:49:58.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.145 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:58.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:58.146 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.146 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.146 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.146 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931749+0000", 2024-09-16T14:49:58.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:58.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931753+0000", 2024-09-16T14:49:58.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0380000000000001e-06 2024-09-16T14:49:58.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.147 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931759+0000", 2024-09-16T14:49:58.147 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.3430000000000001e-06 2024-09-16T14:49:58.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.147 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931765+0000", 2024-09-16T14:49:58.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.6269999999999997e-06 2024-09-16T14:49:58.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.935623+0000", 2024-09-16T14:49:58.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0038583049999999998 2024-09-16T14:49:58.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.148 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.148 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.148 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.935717+0000", 2024-09-16T14:49:58.148 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.4587999999999996e-05 2024-09-16T14:49:58.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936150+0000", 2024-09-16T14:49:58.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043270599999999999 2024-09-16T14:49:58.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.939576+0000", 2024-09-16T14:49:58.149 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0034257620000000002 2024-09-16T14:49:58.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.149 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.149 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:58.149 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943399+0000", 2024-09-16T14:49:58.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003823145 2024-09-16T14:49:58.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.150 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.150 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943490+0000", 2024-09-16T14:49:58.150 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1241000000000003e-05 2024-09-16T14:49:58.150 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.150 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.150 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.150 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.150 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.150 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:301 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-16T14:49:58.150 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:58.150 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 73.190811138000001, 2024-09-16T14:49:58.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014215955000000001, 2024-09-16T14:49:58.151 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.151 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:58.151 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.151 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.151 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.151 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 301 2024-09-16T14:49:58.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.151 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.151 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.151 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.151 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:58.151 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931771+0000", 2024-09-16T14:49:58.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931770+0000", 2024-09-16T14:49:58.152 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.152 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.152 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.152 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931775+0000", 2024-09-16T14:49:58.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4530000000000004e-06 2024-09-16T14:49:58.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931782+0000", 2024-09-16T14:49:58.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.0550000000000002e-06 2024-09-16T14:49:58.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.153 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.153 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.931787+0000", 2024-09-16T14:49:58.153 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2310000000000004e-06 2024-09-16T14:49:58.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.153 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936401+0000", 2024-09-16T14:49:58.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0046143419999999996 2024-09-16T14:49:58.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:58.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.936482+0000", 2024-09-16T14:49:58.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.0407000000000005e-05 2024-09-16T14:49:58.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.154 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.154 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.154 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943851+0000", 2024-09-16T14:49:58.154 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0073695000000000002 2024-09-16T14:49:58.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.943951+0000", 2024-09-16T14:49:58.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9327000000000006e-05 2024-09-16T14:49:58.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.155 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.155 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.155 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:44.945987+0000", 2024-09-16T14:49:58.155 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.002036057 2024-09-16T14:49:58.155 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.155 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.155 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.156 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.156 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:369 3.8s0 3:1f96bdfc:::smithi04040151-37:head [append 0~475136] snapc d=[d,c] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-16T14:49:58.156 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:58.156 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 69.164869144999997, 2024-09-16T14:49:58.156 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010979078999999999, 2024-09-16T14:49:58.156 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.156 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.156 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.156 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 369 2024-09-16T14:49:58.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.157 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.157 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:58.157 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.157 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.157 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957713+0000", 2024-09-16T14:49:58.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.957711+0000", 2024-09-16T14:49:58.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:58.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.158 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.158 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.158 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958323+0000", 2024-09-16T14:49:58.158 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000612439 2024-09-16T14:49:58.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958339+0000", 2024-09-16T14:49:58.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.5985999999999999e-05 2024-09-16T14:49:58.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958348+0000", 2024-09-16T14:49:58.159 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8699999999999998e-06 2024-09-16T14:49:58.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.159 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.159 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.159 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958393+0000", 2024-09-16T14:49:58.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4981000000000002e-05 2024-09-16T14:49:58.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.958478+0000", 2024-09-16T14:49:58.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4845999999999994e-05 2024-09-16T14:49:58.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.160 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.160 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.959659+0000", 2024-09-16T14:49:58.160 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0011808999999999999 2024-09-16T14:49:58.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.160 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.962687+0000", 2024-09-16T14:49:58.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0030278330000000002 2024-09-16T14:49:58.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:58.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968651+0000", 2024-09-16T14:49:58.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005964764 2024-09-16T14:49:58.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.161 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.161 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.161 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:48.968692+0000", 2024-09-16T14:49:58.161 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.0374999999999999e-05 2024-09-16T14:49:58.161 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.162 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.162 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.162 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.162 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.162 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:398 3.as0 3:572aff57:::smithi04040151-20:head [append 0~507904] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:58.162 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:58.162 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.243504379000001, 2024-09-16T14:49:58.162 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010447988, 2024-09-16T14:49:58.162 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.162 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.162 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.162 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.163 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.163 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 398 2024-09-16T14:49:58.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.163 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:58.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.163 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.163 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.163 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879077+0000", 2024-09-16T14:49:58.163 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.879076+0000", 2024-09-16T14:49:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881851+0000", 2024-09-16T14:49:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0027753420000000001 2024-09-16T14:49:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.164 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881863+0000", 2024-09-16T14:49:58.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.2203e-05 2024-09-16T14:49:58.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881871+0000", 2024-09-16T14:49:58.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.4059999999999999e-06 2024-09-16T14:49:58.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.165 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.881924+0000", 2024-09-16T14:49:58.165 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.2932000000000001e-05 2024-09-16T14:49:58.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.165 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.165 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.882118+0000", 2024-09-16T14:49:58.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00019413500000000001 2024-09-16T14:49:58.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.884114+0000", 2024-09-16T14:49:58.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001996154 2024-09-16T14:49:58.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.166 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.166 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889295+0000", 2024-09-16T14:49:58.166 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0051809899999999999 2024-09-16T14:49:58.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.166 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:58.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889480+0000", 2024-09-16T14:49:58.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00018519700000000001 2024-09-16T14:49:58.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.167 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.167 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889525+0000", 2024-09-16T14:49:58.167 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.5399999999999999e-05 2024-09-16T14:49:58.167 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.167 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.167 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.167 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.167 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.167 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:400 3.as0 3:572aff57:::smithi04040151-20:head [append 0~262144] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:58.168 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:58.168 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.238995349000007, 2024-09-16T14:49:58.168 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010619327, 2024-09-16T14:49:58.168 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.168 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.168 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.168 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.168 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.168 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 400 2024-09-16T14:49:58.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.168 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.168 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.168 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:58.169 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.169 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.169 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.169 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883587+0000", 2024-09-16T14:49:58.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.170 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.170 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.170 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883585+0000", 2024-09-16T14:49:58.170 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:58.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.171 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.171 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883868+0000", 2024-09-16T14:49:58.171 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00028385599999999997 2024-09-16T14:49:58.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.171 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883880+0000", 2024-09-16T14:49:58.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1181e-05 2024-09-16T14:49:58.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.172 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.172 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.172 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883888+0000", 2024-09-16T14:49:58.172 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.926e-06 2024-09-16T14:49:58.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.173 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.173 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887739+0000", 2024-09-16T14:49:58.173 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003851658 2024-09-16T14:49:58.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.174 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.887849+0000", 2024-09-16T14:49:58.174 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000109616 2024-09-16T14:49:58.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.174 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.174 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.888684+0000", 2024-09-16T14:49:58.175 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00083518000000000004 2024-09-16T14:49:58.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.175 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.175 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.175 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894010+0000", 2024-09-16T14:49:58.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0053260670000000003 2024-09-16T14:49:58.176 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.176 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.176 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:58.176 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894143+0000", 2024-09-16T14:49:58.176 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000133077 2024-09-16T14:49:58.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.177 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.177 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.177 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894206+0000", 2024-09-16T14:49:58.177 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.2658999999999998e-05 2024-09-16T14:49:58.177 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.178 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:401 3.as0 3:572aff57:::smithi04040151-20:head [setxattr _header (57) in=7b] snapc 12=[12,10,c] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.238685425, 2024-09-16T14:49:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010524774000000001, 2024-09-16T14:49:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.179 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 401 2024-09-16T14:49:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.180 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.181 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.182 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.182 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:58.182 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.182 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.183 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883896+0000", 2024-09-16T14:49:58.183 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.183 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.183 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883900+0000", 2024-09-16T14:49:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1400000000000002e-06 2024-09-16T14:49:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.184 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883908+0000", 2024-09-16T14:49:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2689999999999999e-06 2024-09-16T14:49:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.185 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883913+0000", 2024-09-16T14:49:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.9320000000000004e-06 2024-09-16T14:49:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889560+0000", 2024-09-16T14:49:58.186 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.005647165 2024-09-16T14:49:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889627+0000", 2024-09-16T14:49:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6951000000000003e-05 2024-09-16T14:49:58.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.188 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.188 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.889898+0000", 2024-09-16T14:49:58.188 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000270709 2024-09-16T14:49:58.188 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.188 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.189 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.189 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894012+0000", 2024-09-16T14:49:58.189 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0041137810000000004 2024-09-16T14:49:58.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.190 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.190 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:58.190 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894344+0000", 2024-09-16T14:49:58.190 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000332121 2024-09-16T14:49:58.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.191 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.191 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.191 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894421+0000", 2024-09-16T14:49:58.191 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7406999999999999e-05 2024-09-16T14:49:58.191 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.191 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.192 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.192 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.192 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:402 3.as0 3:572aff57:::smithi04040151-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-16T14:49:58.192 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:58.193 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 64.238663548999995, 2024-09-16T14:49:58.193 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011864743000000001, 2024-09-16T14:49:58.193 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.193 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:58.193 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.194 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.194 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.194 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 402 2024-09-16T14:49:58.194 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.194 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.194 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.195 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.195 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:58.195 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.196 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.196 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.196 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:58.196 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.197 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.197 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.197 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883918+0000", 2024-09-16T14:49:58.197 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.198 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.198 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883921+0000", 2024-09-16T14:49:58.198 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7520000000000001e-06 2024-09-16T14:49:58.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.198 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.199 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.199 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883927+0000", 2024-09-16T14:49:58.199 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4679999999999997e-06 2024-09-16T14:49:58.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.199 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.200 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.883932+0000", 2024-09-16T14:49:58.200 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.1440000000000002e-06 2024-09-16T14:49:58.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.200 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.201 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890477+0000", 2024-09-16T14:49:58.201 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0065446430000000002 2024-09-16T14:49:58.201 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.201 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.201 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:58.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.890586+0000", 2024-09-16T14:49:58.202 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010909 2024-09-16T14:49:58.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.202 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.202 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.202 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894750+0000", 2024-09-16T14:49:58.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00416359 2024-09-16T14:49:58.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.203 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.203 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.203 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.894834+0000", 2024-09-16T14:49:58.203 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4596000000000001e-05 2024-09-16T14:49:58.204 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.204 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.204 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.204 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:53.895783+0000", 2024-09-16T14:49:58.205 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00094892000000000002 2024-09-16T14:49:58.205 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.205 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.205 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.206 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.206 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:443 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:58.206 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:58.206 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 60.246452976, 2024-09-16T14:49:58.206 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.013766494000000001, 2024-09-16T14:49:58.206 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.207 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:58.207 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.207 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.207 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.207 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 443 2024-09-16T14:49:58.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.208 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.208 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.208 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.208 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:58.208 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.209 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.209 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.209 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876129+0000", 2024-09-16T14:49:58.209 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.210 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.210 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.210 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876127+0000", 2024-09-16T14:49:58.210 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:58.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.211 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.211 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.211 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876138+0000", 2024-09-16T14:49:58.211 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.1344e-05 2024-09-16T14:49:58.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.212 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.212 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.212 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876159+0000", 2024-09-16T14:49:58.212 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1157000000000001e-05 2024-09-16T14:49:58.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.213 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.213 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.213 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876170+0000", 2024-09-16T14:49:58.213 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0651e-05 2024-09-16T14:49:58.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.214 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.214 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876234+0000", 2024-09-16T14:49:58.214 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.4114000000000003e-05 2024-09-16T14:49:58.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.214 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.215 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "op must wait for map", 2024-09-16T14:49:58.215 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876243+0000", 2024-09-16T14:49:58.215 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0049999999999993e-06 2024-09-16T14:49:58.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.215 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.216 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879445+0000", 2024-09-16T14:49:58.216 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0032019269999999998 2024-09-16T14:49:58.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.216 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.216 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.879550+0000", 2024-09-16T14:49:58.217 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00010536500000000001 2024-09-16T14:49:58.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.217 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.217 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.217 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.889895+0000", 2024-09-16T14:49:58.218 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010345043 2024-09-16T14:49:58.218 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.218 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.218 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.219 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:444 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:58.219 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:58.219 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 60.246363269, 2024-09-16T14:49:58.219 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.014716241, 2024-09-16T14:49:58.219 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.219 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:58.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.220 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.220 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 444 2024-09-16T14:49:58.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.220 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.221 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.221 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.221 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:58.221 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.221 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.222 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.222 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876219+0000", 2024-09-16T14:49:58.222 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.222 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.223 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876218+0000", 2024-09-16T14:49:58.223 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:58.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.223 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.223 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876224+0000", 2024-09-16T14:49:58.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.6089999999999997e-06 2024-09-16T14:49:58.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.224 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.224 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.224 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876235+0000", 2024-09-16T14:49:58.224 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.025e-05 2024-09-16T14:49:58.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.225 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876242+0000", 2024-09-16T14:49:58.225 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.8790000000000003e-06 2024-09-16T14:49:58.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.225 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.225 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876315+0000", 2024-09-16T14:49:58.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.2260000000000003e-05 2024-09-16T14:49:58.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.226 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.226 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:58.226 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876320+0000", 2024-09-16T14:49:58.226 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.5690000000000004e-06 2024-09-16T14:49:58.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880078+0000", 2024-09-16T14:49:58.227 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0037576430000000002 2024-09-16T14:49:58.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.227 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.227 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.880168+0000", 2024-09-16T14:49:58.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.0353999999999996e-05 2024-09-16T14:49:58.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.228 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.228 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.228 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.890935+0000", 2024-09-16T14:49:58.228 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010766556 2024-09-16T14:49:58.228 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.229 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.229 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.229 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.229 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:445 3.es0 3:7ecfd044:::smithi04040151-35:head [read 0~1458176,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-16T14:49:58.229 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:58.230 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 60.246319286999999, 2024-09-16T14:49:58.230 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016529716999999999, 2024-09-16T14:49:58.230 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.230 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:58.230 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.230 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.230 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.230 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 445 2024-09-16T14:49:58.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.231 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.231 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.231 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:58.231 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.231 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876263+0000", 2024-09-16T14:49:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876262+0000", 2024-09-16T14:49:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876269+0000", 2024-09-16T14:49:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.9469999999999997e-06 2024-09-16T14:49:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.233 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876276+0000", 2024-09-16T14:49:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.5950000000000004e-06 2024-09-16T14:49:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.234 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.235 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876281+0000", 2024-09-16T14:49:58.235 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.7509999999999997e-06 2024-09-16T14:49:58.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.235 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876343+0000", 2024-09-16T14:49:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.1685000000000003e-05 2024-09-16T14:49:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.236 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.876348+0000", 2024-09-16T14:49:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.7269999999999998e-06 2024-09-16T14:49:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.237 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881091+0000", 2024-09-16T14:49:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0047434599999999997 2024-09-16T14:49:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.881181+0000", 2024-09-16T14:49:58.238 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.9838000000000003e-05 2024-09-16T14:49:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:48:57.892792+0000", 2024-09-16T14:49:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011611316 2024-09-16T14:49:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.239 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.4s1 61/27 ECSubWrite(tid=64, reqid=osd.1.0:64, at_version=61'52, trim_to=0'0, roll_forward_to=60'48))", 2024-09-16T14:49:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 50.189509037000001, 2024-09-16T14:49:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.011783765999999999, 2024-09-16T14:49:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.240 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933073+0000", 2024-09-16T14:49:58.241 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933072+0000", 2024-09-16T14:49:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:58.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933092+0000", 2024-09-16T14:49:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0000000000000002e-05 2024-09-16T14:49:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.243 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933119+0000", 2024-09-16T14:49:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6877e-05 2024-09-16T14:49:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.244 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.933127+0000", 2024-09-16T14:49:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.4600000000000003e-06 2024-09-16T14:49:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.937087+0000", 2024-09-16T14:49:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0039599129999999998 2024-09-16T14:49:58.245 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.937096+0000", 2024-09-16T14:49:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.8910000000000001e-06 2024-09-16T14:49:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.246 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944834+0000", 2024-09-16T14:49:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0077376809999999997 2024-09-16T14:49:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.247 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:07.944857+0000", 2024-09-16T14:49:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.2785999999999998e-05 2024-09-16T14:49:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.248 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:591 3.0s0 3:060eef9f:::smithi04040151-16:head [copy-from ver 27] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.146710028999998, 2024-09-16T14:49:58.249 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0279670919999999, 2024-09-16T14:49:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 591 2024-09-16T14:49:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975872+0000", 2024-09-16T14:49:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975870+0000", 2024-09-16T14:49:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.251 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975878+0000", 2024-09-16T14:49:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.2619999999999998e-06 2024-09-16T14:49:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975902+0000", 2024-09-16T14:49:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.4036000000000002e-05 2024-09-16T14:49:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975912+0000", 2024-09-16T14:49:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.9639999999999998e-06 2024-09-16T14:49:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.252 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975986+0000", 2024-09-16T14:49:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3548999999999997e-05 2024-09-16T14:49:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975990+0000", 2024-09-16T14:49:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 3.7670000000000001e-06 2024-09-16T14:49:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.975658+0000", 2024-09-16T14:49:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.99966882800000001 2024-09-16T14:49:58.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for missing object", 2024-09-16T14:49:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.975885+0000", 2024-09-16T14:49:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000226682 2024-09-16T14:49:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.982217+0000", 2024-09-16T14:49:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0063321530000000001 2024-09-16T14:49:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.254 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.982350+0000", 2024-09-16T14:49:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00013316700000000001 2024-09-16T14:49:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.999798+0000", 2024-09-16T14:49:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.017447608 2024-09-16T14:49:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001884+0000", 2024-09-16T14:49:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0020859889999999999 2024-09-16T14:49:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.255 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.003629+0000", 2024-09-16T14:49:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.001745106 2024-09-16T14:49:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.003839+0000", 2024-09-16T14:49:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00020980099999999999 2024-09-16T14:49:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.256 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:592 3.0s0 3:060eef9f:::smithi04040151-16:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e65)", 2024-09-16T14:49:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 46.146653104999999, 2024-09-16T14:49:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0256284010000001, 2024-09-16T14:49:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 592 2024-09-16T14:49:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.257 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975929+0000", 2024-09-16T14:49:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975928+0000", 2024-09-16T14:49:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.258 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975936+0000", 2024-09-16T14:49:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.6090000000000003e-06 2024-09-16T14:49:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975943+0000", 2024-09-16T14:49:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.7849999999999992e-06 2024-09-16T14:49:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.975949+0000", 2024-09-16T14:49:58.259 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.8300000000000001e-06 2024-09-16T14:49:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.976030+0000", 2024-09-16T14:49:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 8.1267000000000006e-05 2024-09-16T14:49:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for peered", 2024-09-16T14:49:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:11.976033+0000", 2024-09-16T14:49:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7590000000000002e-06 2024-09-16T14:49:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.260 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.977041+0000", 2024-09-16T14:49:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0010081639999999 2024-09-16T14:49:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for missing object", 2024-09-16T14:49:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.977086+0000", 2024-09-16T14:49:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.4879999999999997e-05 2024-09-16T14:49:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.983015+0000", 2024-09-16T14:49:58.261 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0059287480000000002 2024-09-16T14:49:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:12.983090+0000", 2024-09-16T14:49:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5081999999999997e-05 2024-09-16T14:49:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001193+0000", 2024-09-16T14:49:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.018102713999999999 2024-09-16T14:49:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.262 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001306+0000", 2024-09-16T14:49:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00011330799999999999 2024-09-16T14:49:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:13.001557+0000", 2024-09-16T14:49:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00025100900000000001 2024-09-16T14:49:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.263 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(osd.8.17:4 3.es0 3:7ecfd044:::smithi04040151-35:head [assert-version v52,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-16T14:49:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:37.971941+0000", 2024-09-16T14:49:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.150640729999999, 2024-09-16T14:49:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.016070661, 2024-09-16T14:49:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971941+0000", 2024-09-16T14:49:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.264 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971941+0000", 2024-09-16T14:49:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971939+0000", 2024-09-16T14:49:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971949+0000", 2024-09-16T14:49:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.2420000000000006e-06 2024-09-16T14:49:58.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971959+0000", 2024-09-16T14:49:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.0827e-05 2024-09-16T14:49:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.971969+0000", 2024-09-16T14:49:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.6579999999999997e-06 2024-09-16T14:49:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.972028+0000", 2024-09-16T14:49:58.266 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.9457999999999999e-05 2024-09-16T14:49:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.972120+0000", 2024-09-16T14:49:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.1174999999999994e-05 2024-09-16T14:49:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:37.988012+0000", 2024-09-16T14:49:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.015892125 2024-09-16T14:49:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.267 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "MOSDECSubOpWrite(3.19s1 91/90 ECSubWrite(tid=102, reqid=client.4500.0:822, at_version=91'77, trim_to=0'0, roll_forward_to=91'75))", 2024-09-16T14:49:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:38.006520+0000", 2024-09-16T14:49:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 20.116062021000001, 2024-09-16T14:49:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01031002, 2024-09-16T14:49:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "reached pg", 2024-09-16T14:49:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.268 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006520+0000", 2024-09-16T14:49:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006520+0000", 2024-09-16T14:49:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006519+0000", 2024-09-16T14:49:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.999999 2024-09-16T14:49:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.269 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006954+0000", 2024-09-16T14:49:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00043463699999999998 2024-09-16T14:49:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006973+0000", 2024-09-16T14:49:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.9389e-05 2024-09-16T14:49:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.006994+0000", 2024-09-16T14:49:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.1229999999999998e-05 2024-09-16T14:49:58.270 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007063+0000", 2024-09-16T14:49:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 6.8979999999999993e-05 2024-09-16T14:49:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.007077+0000", 2024-09-16T14:49:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 1.4119e-05 2024-09-16T14:49:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016803+0000", 2024-09-16T14:49:58.271 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0097257460000000004 2024-09-16T14:49:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:38.016830+0000", 2024-09-16T14:49:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.6835999999999999e-05 2024-09-16T14:49:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:927 3.es0 3:7b667bcb:::smithi04040151-2:head [append 0~573440] snapc 46=[46] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-16T14:49:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:45.975604+0000", 2024-09-16T14:49:58.272 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 12.146977726999999, 2024-09-16T14:49:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.010575327000000001, 2024-09-16T14:49:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 927 2024-09-16T14:49:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.975604+0000", 2024-09-16T14:49:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.975604+0000", 2024-09-16T14:49:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.975602+0000", 2024-09-16T14:49:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.976305+0000", 2024-09-16T14:49:58.274 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00070255000000000003 2024-09-16T14:49:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.976332+0000", 2024-09-16T14:49:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.7155e-05 2024-09-16T14:49:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.976337+0000", 2024-09-16T14:49:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4790000000000004e-06 2024-09-16T14:49:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.275 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.976392+0000", 2024-09-16T14:49:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 5.4843999999999999e-05 2024-09-16T14:49:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.976512+0000", 2024-09-16T14:49:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.000119578 2024-09-16T14:49:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.977488+0000", 2024-09-16T14:49:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.00097599399999999999 2024-09-16T14:49:58.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.978514+0000", 2024-09-16T14:49:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0010261109999999999 2024-09-16T14:49:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "commit_sent", 2024-09-16T14:49:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.986106+0000", 2024-09-16T14:49:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0075921909999999999 2024-09-16T14:49:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.277 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:45.986179+0000", 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.3362999999999995e-05 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "description": "osd_op(client.4500.0:1005 3.0s0 3:074dfb2a:::smithi04040151-7:4a [read 0~3129344,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "initiated_at": "2024-09-16T14:49:51.908094+0000", 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 6.2144879990000002, 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.01043324, 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "type_data": { 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "flag_point": "started", 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "client_info": { 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "client": "client.4500", 2024-09-16T14:49:58.278 INFO:teuthology.orchestra.run.smithi114.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "tid": 1005 2024-09-16T14:49:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "events": [ 2024-09-16T14:49:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "initiated", 2024-09-16T14:49:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:51.908094+0000", 2024-09-16T14:49:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "throttled", 2024-09-16T14:49:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:51.908094+0000", 2024-09-16T14:49:58.279 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0 2024-09-16T14:49:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "header_read", 2024-09-16T14:49:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:51.908094+0000", 2024-09-16T14:49:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "all_read", 2024-09-16T14:49:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:51.908096+0000", 2024-09-16T14:49:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 2.0660000000000002e-06 2024-09-16T14:49:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "dispatched", 2024-09-16T14:49:58.280 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:51.908103+0000", 2024-09-16T14:49:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 7.5120000000000002e-06 2024-09-16T14:49:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:51.908107+0000", 2024-09-16T14:49:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 4.1309999999999999e-06 2024-09-16T14:49:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "reached_pg", 2024-09-16T14:49:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:51.912026+0000", 2024-09-16T14:49:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.003918497 2024-09-16T14:49:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.281 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "started", 2024-09-16T14:49:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:51.912123+0000", 2024-09-16T14:49:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 9.7688000000000004e-05 2024-09-16T14:49:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-16T14:49:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-16T14:49:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: "event": "done", 2024-09-16T14:49:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: "time": "2024-09-16T14:49:51.918527+0000", 2024-09-16T14:49:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: "duration": 0.0064036149999999997 2024-09-16T14:49:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-16T14:49:58.282 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-16T14:49:58.282 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-16T14:49:58.284 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T14:49:58.399 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: "num_ops": 0 2024-09-16T14:49:58.399 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:58.408 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T14:49:58.511 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:58.511 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [], 2024-09-16T14:49:58.511 INFO:teuthology.orchestra.run.smithi111.stdout: "complaint_time": 30, 2024-09-16T14:49:58.511 INFO:teuthology.orchestra.run.smithi111.stdout: "num_blocked_ops": 0 2024-09-16T14:49:58.511 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:58.519 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T14:49:58.624 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-16T14:49:58.625 INFO:teuthology.orchestra.run.smithi111.stdout: "size": 20, 2024-09-16T14:49:58.625 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 600, 2024-09-16T14:49:58.625 INFO:teuthology.orchestra.run.smithi111.stdout: "ops": [ 2024-09-16T14:49:58.625 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.625 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.3s3 66/65 ECSubWrite(tid=39, reqid=client.4500.0:597, at_version=66'26, trim_to=0'0, roll_forward_to=66'23))", 2024-09-16T14:49:58.625 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:58.625 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 45.610160633, 2024-09-16T14:49:58.625 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0060634030000000002, 2024-09-16T14:49:58.625 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.625 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:58.625 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.625 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.626 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.626 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:58.626 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.626 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.626 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.626 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.626 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012389+0000", 2024-09-16T14:49:58.626 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.626 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.626 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.626 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.626 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012388+0000", 2024-09-16T14:49:58.626 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:58.627 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.627 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.627 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.627 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012433+0000", 2024-09-16T14:49:58.627 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4588000000000001e-05 2024-09-16T14:49:58.627 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.627 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.627 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.627 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012604+0000", 2024-09-16T14:49:58.627 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00017141300000000001 2024-09-16T14:49:58.627 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.627 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.627 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.627 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012617+0000", 2024-09-16T14:49:58.628 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2792999999999999e-05 2024-09-16T14:49:58.628 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.628 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.628 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.628 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012666+0000", 2024-09-16T14:49:58.628 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.9079000000000002e-05 2024-09-16T14:49:58.628 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.628 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.628 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.628 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.012678+0000", 2024-09-16T14:49:58.628 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1405e-05 2024-09-16T14:49:58.628 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.628 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.629 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.629 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018433+0000", 2024-09-16T14:49:58.629 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005755146 2024-09-16T14:49:58.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.629 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.629 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.629 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:13.018453+0000", 2024-09-16T14:49:58.629 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9780999999999999e-05 2024-09-16T14:49:58.629 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.629 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.629 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.629 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.630 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:702 3.as0 3:572aff57:::smithi04040151-20:head [copy-from ver 4] snapc 2f=[2f,2e,29,28,25] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:58.630 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:58.630 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 34.722472351, 2024-09-16T14:49:58.630 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.032342711000000003, 2024-09-16T14:49:58.630 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.630 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.630 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:58.630 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:58.630 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.630 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 702 2024-09-16T14:49:58.630 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.630 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.630 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.631 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.631 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:58.631 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.631 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.631 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.631 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.631 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900078+0000", 2024-09-16T14:49:58.631 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.631 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.631 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.631 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.631 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900076+0000", 2024-09-16T14:49:58.631 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:58.631 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.632 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.632 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.632 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900082+0000", 2024-09-16T14:49:58.632 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.3640000000000004e-06 2024-09-16T14:49:58.632 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.632 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.632 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.632 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900094+0000", 2024-09-16T14:49:58.632 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1208e-05 2024-09-16T14:49:58.632 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.632 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.632 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.632 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900105+0000", 2024-09-16T14:49:58.633 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1782e-05 2024-09-16T14:49:58.633 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.633 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.633 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.633 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900161+0000", 2024-09-16T14:49:58.633 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5244999999999997e-05 2024-09-16T14:49:58.633 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.633 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.633 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "op must wait for map", 2024-09-16T14:49:58.633 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900170+0000", 2024-09-16T14:49:58.633 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.4409999999999999e-06 2024-09-16T14:49:58.633 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.633 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.633 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.634 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.901930+0000", 2024-09-16T14:49:58.634 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001760246 2024-09-16T14:49:58.634 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.634 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.634 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:58.634 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902046+0000", 2024-09-16T14:49:58.634 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000115882 2024-09-16T14:49:58.634 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.634 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.634 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.634 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.917553+0000", 2024-09-16T14:49:58.634 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015506967999999999 2024-09-16T14:49:58.634 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.634 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.635 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.635 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918732+0000", 2024-09-16T14:49:58.635 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001178733 2024-09-16T14:49:58.635 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.635 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.635 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:58.635 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932379+0000", 2024-09-16T14:49:58.635 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.013647276999999999 2024-09-16T14:49:58.635 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.635 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.635 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.635 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.932420+0000", 2024-09-16T14:49:58.635 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.1066999999999998e-05 2024-09-16T14:49:58.635 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:703 3.as0 3:572aff57:::smithi04040151-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e77)", 2024-09-16T14:49:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 34.722430953, 2024-09-16T14:49:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.018566787000000001, 2024-09-16T14:49:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 703 2024-09-16T14:49:58.637 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.637 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.637 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.637 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.637 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:58.637 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.637 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.637 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.637 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.637 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900119+0000", 2024-09-16T14:49:58.637 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.637 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.637 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.637 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.638 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900118+0000", 2024-09-16T14:49:58.638 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.638 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.638 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.638 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.638 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900122+0000", 2024-09-16T14:49:58.638 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.5669999999999999e-06 2024-09-16T14:49:58.638 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.638 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.638 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.638 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900129+0000", 2024-09-16T14:49:58.638 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0360000000000001e-06 2024-09-16T14:49:58.638 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.638 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.639 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.639 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900134+0000", 2024-09-16T14:49:58.639 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.916e-06 2024-09-16T14:49:58.639 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.639 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.639 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.639 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900217+0000", 2024-09-16T14:49:58.639 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.3189000000000006e-05 2024-09-16T14:49:58.639 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.639 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.639 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting_for_map not empty", 2024-09-16T14:49:58.639 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.900222+0000", 2024-09-16T14:49:58.639 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7430000000000002e-06 2024-09-16T14:49:58.640 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.640 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.640 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.640 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902209+0000", 2024-09-16T14:49:58.640 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0019871440000000001 2024-09-16T14:49:58.640 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.640 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.640 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:58.640 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.902254+0000", 2024-09-16T14:49:58.640 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.4969000000000003e-05 2024-09-16T14:49:58.640 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.640 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.640 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.640 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918411+0000", 2024-09-16T14:49:58.641 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.016157207999999999 2024-09-16T14:49:58.641 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.641 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.641 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:58.641 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918534+0000", 2024-09-16T14:49:58.641 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000123146 2024-09-16T14:49:58.641 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.641 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.641 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.641 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:23.918686+0000", 2024-09-16T14:49:58.641 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00015134799999999999 2024-09-16T14:49:58.641 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.641 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.642 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.642 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.642 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.642 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:767 3.as0 3:572aff57:::smithi04040151-20:head [copy-from ver 63] snapc 31=[2e,28] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:49:58.642 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:29.077215+0000", 2024-09-16T14:49:58.642 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.545334641, 2024-09-16T14:49:58.642 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.021272019999999999, 2024-09-16T14:49:58.642 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.642 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.642 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:58.642 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:58.642 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.642 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 767 2024-09-16T14:49:58.643 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.643 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.643 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.643 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.643 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077215+0000", 2024-09-16T14:49:58.643 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.643 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.643 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.643 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.643 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077215+0000", 2024-09-16T14:49:58.643 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.643 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.643 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.644 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.644 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077213+0000", 2024-09-16T14:49:58.644 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999981 2024-09-16T14:49:58.644 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.644 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.644 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.644 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077223+0000", 2024-09-16T14:49:58.644 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.9890000000000004e-06 2024-09-16T14:49:58.644 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.644 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.644 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.644 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077242+0000", 2024-09-16T14:49:58.644 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.8286000000000001e-05 2024-09-16T14:49:58.645 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.645 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.645 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.645 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077253+0000", 2024-09-16T14:49:58.645 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1807999999999999e-05 2024-09-16T14:49:58.645 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.645 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.645 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.645 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077321+0000", 2024-09-16T14:49:58.645 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.7131999999999994e-05 2024-09-16T14:49:58.645 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.645 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.645 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:58.646 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077431+0000", 2024-09-16T14:49:58.646 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0001102 2024-09-16T14:49:58.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.646 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.646 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.646 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.090085+0000", 2024-09-16T14:49:58.646 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.012654349 2024-09-16T14:49:58.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.646 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.646 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.646 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092962+0000", 2024-09-16T14:49:58.646 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002877133 2024-09-16T14:49:58.646 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.646 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.647 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:58.647 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.098442+0000", 2024-09-16T14:49:58.647 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0054794149999999996 2024-09-16T14:49:58.647 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.647 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.647 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.647 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.098487+0000", 2024-09-16T14:49:58.647 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.5587000000000002e-05 2024-09-16T14:49:58.647 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.647 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.647 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.647 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.648 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.648 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:768 3.as0 3:572aff57:::smithi04040151-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-09-16T14:49:58.648 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:58.648 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 29.545283501, 2024-09-16T14:49:58.648 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.015338218000000001, 2024-09-16T14:49:58.648 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.648 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:58.648 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:58.648 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:58.648 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.648 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 768 2024-09-16T14:49:58.648 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.648 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.649 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.649 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.649 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:58.649 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.649 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.649 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.649 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.649 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:58.649 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.649 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.649 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.649 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.649 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077266+0000", 2024-09-16T14:49:58.649 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.650 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.650 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.650 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.650 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077268+0000", 2024-09-16T14:49:58.650 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.266e-06 2024-09-16T14:49:58.650 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.650 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.650 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.650 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077275+0000", 2024-09-16T14:49:58.650 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.6200000000000001e-06 2024-09-16T14:49:58.650 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.650 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.650 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.651 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077278+0000", 2024-09-16T14:49:58.651 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.055e-06 2024-09-16T14:49:58.651 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.651 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.651 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.651 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077612+0000", 2024-09-16T14:49:58.651 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000334321 2024-09-16T14:49:58.651 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.651 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.651 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for blocked object", 2024-09-16T14:49:58.651 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.077660+0000", 2024-09-16T14:49:58.651 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.8161999999999997e-05 2024-09-16T14:49:58.651 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.651 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.652 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.652 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092440+0000", 2024-09-16T14:49:58.652 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014779825999999999 2024-09-16T14:49:58.652 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.652 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.652 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:58.652 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092530+0000", 2024-09-16T14:49:58.652 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9232000000000002e-05 2024-09-16T14:49:58.652 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.652 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.652 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.652 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:29.092605+0000", 2024-09-16T14:49:58.653 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5086999999999994e-05 2024-09-16T14:49:58.653 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.653 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.653 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.653 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.653 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.653 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds3 86/65 ECSubWrite(tid=302, reqid=client.4500.0:792, at_version=86'71, trim_to=0'0, roll_forward_to=83'69))", 2024-09-16T14:49:58.653 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:32.881497+0000", 2024-09-16T14:49:58.653 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 25.74105321, 2024-09-16T14:49:58.653 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0055907470000000001, 2024-09-16T14:49:58.653 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.653 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:58.653 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.654 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.654 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.654 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881497+0000", 2024-09-16T14:49:58.654 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.654 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.654 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.654 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.654 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881497+0000", 2024-09-16T14:49:58.654 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.654 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.654 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.654 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.655 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881496+0000", 2024-09-16T14:49:58.655 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.655 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.655 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.655 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.655 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881544+0000", 2024-09-16T14:49:58.655 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.7973000000000003e-05 2024-09-16T14:49:58.655 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.655 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.655 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.655 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881555+0000", 2024-09-16T14:49:58.655 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1405e-05 2024-09-16T14:49:58.655 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.656 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.656 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.656 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881564+0000", 2024-09-16T14:49:58.656 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7329999999999992e-06 2024-09-16T14:49:58.656 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.656 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.656 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.656 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881635+0000", 2024-09-16T14:49:58.656 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.1067000000000002e-05 2024-09-16T14:49:58.656 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.656 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.656 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.656 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.881647+0000", 2024-09-16T14:49:58.657 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1239e-05 2024-09-16T14:49:58.657 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.657 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.657 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.657 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.887060+0000", 2024-09-16T14:49:58.657 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.005413691 2024-09-16T14:49:58.657 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.657 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.657 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.657 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:32.887087+0000", 2024-09-16T14:49:58.657 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.7234e-05 2024-09-16T14:49:58.657 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.657 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.657 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.658 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.658 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.658 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.cs2 90/65 ECSubWrite(tid=189, reqid=osd.5.0:189, at_version=90'73, trim_to=0'0, roll_forward_to=89'71))", 2024-09-16T14:49:58.658 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.947178+0000", 2024-09-16T14:49:58.658 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 21.675371906999999, 2024-09-16T14:49:58.658 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014311760999999999, 2024-09-16T14:49:58.658 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.658 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:58.658 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.658 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.658 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.658 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947178+0000", 2024-09-16T14:49:58.658 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.659 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.659 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.659 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.659 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947178+0000", 2024-09-16T14:49:58.659 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.659 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.659 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.659 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.659 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947177+0000", 2024-09-16T14:49:58.659 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.659 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.659 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.659 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.660 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947210+0000", 2024-09-16T14:49:58.660 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2759999999999998e-05 2024-09-16T14:49:58.660 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.660 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.660 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.660 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947224+0000", 2024-09-16T14:49:58.660 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3577000000000001e-05 2024-09-16T14:49:58.660 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.660 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.660 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.660 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947233+0000", 2024-09-16T14:49:58.660 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.8169999999999994e-06 2024-09-16T14:49:58.660 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.661 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.661 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.661 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947304+0000", 2024-09-16T14:49:58.661 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0809000000000006e-05 2024-09-16T14:49:58.661 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.661 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.661 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.661 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947317+0000", 2024-09-16T14:49:58.661 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.292e-05 2024-09-16T14:49:58.661 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.661 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.661 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.661 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.961464+0000", 2024-09-16T14:49:58.662 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014146532 2024-09-16T14:49:58.662 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.662 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.662 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.662 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.961490+0000", 2024-09-16T14:49:58.662 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6012000000000001e-05 2024-09-16T14:49:58.662 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.662 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.662 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.662 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.662 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.662 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs7 90/65 ECSubWrite(tid=379, reqid=osd.3.0:379, at_version=90'75, trim_to=0'0, roll_forward_to=83'73))", 2024-09-16T14:49:58.662 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.947459+0000", 2024-09-16T14:49:58.663 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 21.675091359, 2024-09-16T14:49:58.663 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0096387420000000005, 2024-09-16T14:49:58.663 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.663 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:58.663 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.663 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.663 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.663 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947459+0000", 2024-09-16T14:49:58.663 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.663 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.663 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.663 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.664 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947459+0000", 2024-09-16T14:49:58.664 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.664 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.664 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.664 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.664 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947458+0000", 2024-09-16T14:49:58.664 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.664 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.664 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.664 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.664 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947486+0000", 2024-09-16T14:49:58.664 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.8123000000000002e-05 2024-09-16T14:49:58.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.665 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.665 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.665 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947496+0000", 2024-09-16T14:49:58.665 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7650000000000005e-06 2024-09-16T14:49:58.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.665 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.665 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.665 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947504+0000", 2024-09-16T14:49:58.665 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9719999999999992e-06 2024-09-16T14:49:58.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.665 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.665 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.665 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947574+0000", 2024-09-16T14:49:58.666 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0259999999999995e-05 2024-09-16T14:49:58.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.666 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.666 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.666 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.947584+0000", 2024-09-16T14:49:58.666 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.7440000000000002e-06 2024-09-16T14:49:58.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.666 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.666 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.666 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957064+0000", 2024-09-16T14:49:58.666 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0094797530000000005 2024-09-16T14:49:58.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.666 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.667 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.667 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957097+0000", 2024-09-16T14:49:58.667 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.3615999999999998e-05 2024-09-16T14:49:58.667 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.667 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.667 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.667 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.667 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.667 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.bs7 90/65 ECSubWrite(tid=380, reqid=osd.3.0:380, at_version=90'77, trim_to=0'0, roll_forward_to=83'73))", 2024-09-16T14:49:58.667 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.948118+0000", 2024-09-16T14:49:58.667 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 21.674431513000002, 2024-09-16T14:49:58.667 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014633293, 2024-09-16T14:49:58.667 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.667 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:58.668 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.668 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.668 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.668 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948118+0000", 2024-09-16T14:49:58.668 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.668 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.668 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.668 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.668 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948118+0000", 2024-09-16T14:49:58.668 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.668 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.668 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.668 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.668 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948118+0000", 2024-09-16T14:49:58.669 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:58.669 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.669 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.669 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.669 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948141+0000", 2024-09-16T14:49:58.669 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3300999999999999e-05 2024-09-16T14:49:58.669 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.669 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.669 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.669 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948158+0000", 2024-09-16T14:49:58.669 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6999000000000001e-05 2024-09-16T14:49:58.669 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.669 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948166+0000", 2024-09-16T14:49:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.551e-06 2024-09-16T14:49:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948701+0000", 2024-09-16T14:49:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00053460599999999997 2024-09-16T14:49:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.948711+0000", 2024-09-16T14:49:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6889999999999999e-06 2024-09-16T14:49:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.671 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.671 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.671 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.962726+0000", 2024-09-16T14:49:58.671 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.014015272 2024-09-16T14:49:58.671 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.671 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.671 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.671 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.962752+0000", 2024-09-16T14:49:58.671 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5701999999999999e-05 2024-09-16T14:49:58.671 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.671 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.671 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.671 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.671 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.672 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.cs2 90/65 ECSubWrite(tid=190, reqid=osd.5.0:190, at_version=90'75, trim_to=0'0, roll_forward_to=89'71))", 2024-09-16T14:49:58.672 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.957165+0000", 2024-09-16T14:49:58.672 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 21.665385378, 2024-09-16T14:49:58.672 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0094526560000000003, 2024-09-16T14:49:58.672 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.672 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:58.672 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.672 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.672 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.672 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957165+0000", 2024-09-16T14:49:58.672 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.672 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.672 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.672 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.673 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957165+0000", 2024-09-16T14:49:58.673 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.673 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.673 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.673 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.673 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957164+0000", 2024-09-16T14:49:58.673 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:58.673 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.673 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.673 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.673 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957190+0000", 2024-09-16T14:49:58.673 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.6451000000000001e-05 2024-09-16T14:49:58.673 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.674 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.674 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.674 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957203+0000", 2024-09-16T14:49:58.674 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.237e-05 2024-09-16T14:49:58.674 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.674 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.674 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.674 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.957211+0000", 2024-09-16T14:49:58.674 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8789999999999992e-06 2024-09-16T14:49:58.674 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.674 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.674 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.674 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.961640+0000", 2024-09-16T14:49:58.674 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0044288629999999999 2024-09-16T14:49:58.675 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.675 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.675 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.675 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.961649+0000", 2024-09-16T14:49:58.675 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.9660000000000002e-06 2024-09-16T14:49:58.675 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.675 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.675 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.675 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.966600+0000", 2024-09-16T14:49:58.675 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0049509059999999997 2024-09-16T14:49:58.675 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.675 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.675 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.675 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.966617+0000", 2024-09-16T14:49:58.676 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.7010000000000001e-05 2024-09-16T14:49:58.676 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.676 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.676 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.676 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.676 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.676 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.ds3 90/65 ECSubWrite(tid=311, reqid=osd.7.0:311, at_version=90'75, trim_to=0'0, roll_forward_to=89'73))", 2024-09-16T14:49:58.676 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:36.958625+0000", 2024-09-16T14:49:58.676 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 21.663924643000001, 2024-09-16T14:49:58.676 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0061203280000000004, 2024-09-16T14:49:58.676 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.676 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:58.676 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.677 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.677 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.677 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958625+0000", 2024-09-16T14:49:58.677 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.677 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.677 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.677 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.677 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958625+0000", 2024-09-16T14:49:58.677 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.677 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.677 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.677 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.677 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958625+0000", 2024-09-16T14:49:58.677 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967296 2024-09-16T14:49:58.678 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.678 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.678 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.678 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958647+0000", 2024-09-16T14:49:58.678 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.2073999999999999e-05 2024-09-16T14:49:58.678 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.678 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.678 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.678 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958660+0000", 2024-09-16T14:49:58.678 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.2831e-05 2024-09-16T14:49:58.678 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.678 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.678 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.678 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958673+0000", 2024-09-16T14:49:58.679 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3169999999999999e-05 2024-09-16T14:49:58.679 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.679 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.679 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.679 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958738+0000", 2024-09-16T14:49:58.679 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.4768000000000001e-05 2024-09-16T14:49:58.679 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.679 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.679 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.679 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.958754+0000", 2024-09-16T14:49:58.679 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.6388999999999998e-05 2024-09-16T14:49:58.679 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.679 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.679 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.680 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.964722+0000", 2024-09-16T14:49:58.680 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0059672579999999996 2024-09-16T14:49:58.680 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.680 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.680 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.680 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:36.964746+0000", 2024-09-16T14:49:58.680 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.4070000000000002e-05 2024-09-16T14:49:58.680 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.680 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.680 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.680 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.680 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.680 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.13s3 91/90 ECSubWrite(tid=54, reqid=osd.4.0:54, at_version=91'39, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:58.680 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:37.982933+0000", 2024-09-16T14:49:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 20.639616822000001, 2024-09-16T14:49:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0063814340000000001, 2024-09-16T14:49:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.982933+0000", 2024-09-16T14:49:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.982933+0000", 2024-09-16T14:49:58.681 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.682 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.682 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.682 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.682 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.982932+0000", 2024-09-16T14:49:58.682 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.682 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.682 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.682 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.682 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.982958+0000", 2024-09-16T14:49:58.682 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.5984999999999999e-05 2024-09-16T14:49:58.682 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.682 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.682 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.682 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.982972+0000", 2024-09-16T14:49:58.683 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3397000000000001e-05 2024-09-16T14:49:58.683 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.683 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.683 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.683 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.982988+0000", 2024-09-16T14:49:58.683 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.5905000000000001e-05 2024-09-16T14:49:58.683 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.683 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.683 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.683 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.983040+0000", 2024-09-16T14:49:58.683 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.2039000000000001e-05 2024-09-16T14:49:58.683 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.684 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.684 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.684 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.983050+0000", 2024-09-16T14:49:58.684 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0132999999999999e-05 2024-09-16T14:49:58.684 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.684 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.684 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.684 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.989291+0000", 2024-09-16T14:49:58.684 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0062410520000000004 2024-09-16T14:49:58.684 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.684 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.684 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.684 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:37.989315+0000", 2024-09-16T14:49:58.685 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3567e-05 2024-09-16T14:49:58.685 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.685 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.685 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.685 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.685 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.685 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:870 3.as0 3:53e83d93:::smithi04040151-22:head [append 0~622592] snapc 40=[3b] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:58.685 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.990043+0000", 2024-09-16T14:49:58.685 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.632506628000002, 2024-09-16T14:49:58.685 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0057099860000000002, 2024-09-16T14:49:58.685 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.685 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.685 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:58.685 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:58.686 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.686 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 870 2024-09-16T14:49:58.686 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.686 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.686 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.686 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.686 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990043+0000", 2024-09-16T14:49:58.686 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.686 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.686 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.686 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.686 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990043+0000", 2024-09-16T14:49:58.686 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.686 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.687 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.687 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.687 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990042+0000", 2024-09-16T14:49:58.687 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999986 2024-09-16T14:49:58.687 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.687 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.687 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.687 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990616+0000", 2024-09-16T14:49:58.687 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00057397299999999997 2024-09-16T14:49:58.687 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.687 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.687 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.687 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990626+0000", 2024-09-16T14:49:58.688 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.0217000000000001e-05 2024-09-16T14:49:58.688 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.688 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.688 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.688 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990633+0000", 2024-09-16T14:49:58.688 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.1740000000000003e-06 2024-09-16T14:49:58.688 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.688 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.688 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.688 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990867+0000", 2024-09-16T14:49:58.688 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00023375300000000001 2024-09-16T14:49:58.688 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.688 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.689 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:58.689 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.990973+0000", 2024-09-16T14:49:58.689 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010624 2024-09-16T14:49:58.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.689 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.689 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.689 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.991742+0000", 2024-09-16T14:49:58.689 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000768701 2024-09-16T14:49:58.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.689 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.689 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.689 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.994341+0000", 2024-09-16T14:49:58.689 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0025990919999999999 2024-09-16T14:49:58.690 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.690 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.690 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:58.690 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.995696+0000", 2024-09-16T14:49:58.690 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001355532 2024-09-16T14:49:58.690 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.690 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.690 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.690 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.995753+0000", 2024-09-16T14:49:58.690 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.6799000000000003e-05 2024-09-16T14:49:58.690 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.690 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.690 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.690 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.691 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.691 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:871 3.as0 3:53e83d93:::smithi04040151-22:head [append 0~311296] snapc 40=[3b] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:58.691 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.991954+0000", 2024-09-16T14:49:58.691 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.630595434, 2024-09-16T14:49:58.691 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0064516110000000003, 2024-09-16T14:49:58.691 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.691 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.691 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:58.691 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:58.691 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.691 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 871 2024-09-16T14:49:58.691 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.692 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.692 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.692 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.692 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.991954+0000", 2024-09-16T14:49:58.692 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.692 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.692 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.692 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.692 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.991954+0000", 2024-09-16T14:49:58.692 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.692 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.692 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.692 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.693 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.991954+0000", 2024-09-16T14:49:58.693 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.693 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.693 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.693 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.693 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992242+0000", 2024-09-16T14:49:58.693 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00028799899999999999 2024-09-16T14:49:58.693 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.693 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.693 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.693 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992254+0000", 2024-09-16T14:49:58.693 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.1741e-05 2024-09-16T14:49:58.693 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.693 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.694 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.694 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992262+0000", 2024-09-16T14:49:58.694 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6449999999999994e-06 2024-09-16T14:49:58.694 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.694 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.694 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.694 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992458+0000", 2024-09-16T14:49:58.694 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000195737 2024-09-16T14:49:58.694 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.694 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.694 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:58.694 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992566+0000", 2024-09-16T14:49:58.694 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00010817199999999999 2024-09-16T14:49:58.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.695 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.695 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.695 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.993258+0000", 2024-09-16T14:49:58.695 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000691744 2024-09-16T14:49:58.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.695 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.695 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.695 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.994449+0000", 2024-09-16T14:49:58.695 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00119136 2024-09-16T14:49:58.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.695 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.695 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:58.695 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.998368+0000", 2024-09-16T14:49:58.696 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0039188540000000003 2024-09-16T14:49:58.696 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.696 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.696 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.696 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.998406+0000", 2024-09-16T14:49:58.696 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.7953000000000001e-05 2024-09-16T14:49:58.696 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.696 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.696 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.696 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.696 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.696 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:872 3.as0 3:53e83d93:::smithi04040151-22:head [setxattr _header (57) in=7b] snapc 40=[3b] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:58.696 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.992270+0000", 2024-09-16T14:49:58.696 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.630279623, 2024-09-16T14:49:58.697 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0062624170000000002, 2024-09-16T14:49:58.697 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.697 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T14:49:58.697 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:58.697 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:58.697 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.697 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 872 2024-09-16T14:49:58.697 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.697 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.697 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.697 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.697 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992270+0000", 2024-09-16T14:49:58.697 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.698 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.698 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.698 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.698 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992270+0000", 2024-09-16T14:49:58.698 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.698 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.698 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.698 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.698 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992270+0000", 2024-09-16T14:49:58.698 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.698 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.698 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.698 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.698 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992273+0000", 2024-09-16T14:49:58.699 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.72e-06 2024-09-16T14:49:58.699 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.699 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.699 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.699 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992281+0000", 2024-09-16T14:49:58.699 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.2910000000000002e-06 2024-09-16T14:49:58.699 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.699 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.699 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.699 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992286+0000", 2024-09-16T14:49:58.699 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.5820000000000001e-06 2024-09-16T14:49:58.699 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.699 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.700 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.700 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.993679+0000", 2024-09-16T14:49:58.700 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001392627 2024-09-16T14:49:58.700 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.700 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.700 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:58.700 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.993772+0000", 2024-09-16T14:49:58.700 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.2819999999999996e-05 2024-09-16T14:49:58.700 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.700 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.700 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.700 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.994153+0000", 2024-09-16T14:49:58.700 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000380949 2024-09-16T14:49:58.701 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.701 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.701 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.701 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.995398+0000", 2024-09-16T14:49:58.701 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.001245803 2024-09-16T14:49:58.701 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.701 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.701 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "commit_sent", 2024-09-16T14:49:58.701 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.998475+0000", 2024-09-16T14:49:58.701 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0030767239999999999 2024-09-16T14:49:58.701 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.701 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.701 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.701 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.998533+0000", 2024-09-16T14:49:58.702 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 5.7595e-05 2024-09-16T14:49:58.702 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.702 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.702 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.702 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.702 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.702 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "osd_op(client.4500.0:873 3.as0 3:53e83d93:::smithi04040151-22:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e95)", 2024-09-16T14:49:58.702 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:41.992307+0000", 2024-09-16T14:49:58.702 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 16.630243146000002, 2024-09-16T14:49:58.702 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0074946509999999997, 2024-09-16T14:49:58.702 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.702 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "started", 2024-09-16T14:49:58.702 INFO:teuthology.orchestra.run.smithi111.stdout: "client_info": { 2024-09-16T14:49:58.703 INFO:teuthology.orchestra.run.smithi111.stdout: "client": "client.4500", 2024-09-16T14:49:58.703 INFO:teuthology.orchestra.run.smithi111.stdout: "client_addr": "172.21.15.40:0/3896173645", 2024-09-16T14:49:58.703 INFO:teuthology.orchestra.run.smithi111.stdout: "tid": 873 2024-09-16T14:49:58.703 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.703 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.703 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.703 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.703 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992307+0000", 2024-09-16T14:49:58.703 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.703 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.703 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.703 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.703 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992307+0000", 2024-09-16T14:49:58.704 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.704 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.704 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.704 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.704 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992306+0000", 2024-09-16T14:49:58.704 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.704 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.704 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.704 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.704 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992309+0000", 2024-09-16T14:49:58.704 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.2059999999999999e-06 2024-09-16T14:49:58.704 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.704 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.704 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.705 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992318+0000", 2024-09-16T14:49:58.705 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.0189999999999995e-06 2024-09-16T14:49:58.705 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.705 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.705 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.705 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.992326+0000", 2024-09-16T14:49:58.705 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.0550000000000002e-06 2024-09-16T14:49:58.705 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.705 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.705 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.705 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.994733+0000", 2024-09-16T14:49:58.705 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.002407345 2024-09-16T14:49:58.705 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.706 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.706 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "waiting for rw locks", 2024-09-16T14:49:58.706 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.994843+0000", 2024-09-16T14:49:58.706 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.000110263 2024-09-16T14:49:58.706 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.706 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.706 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.706 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.998807+0000", 2024-09-16T14:49:58.706 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0039642460000000003 2024-09-16T14:49:58.706 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.706 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.706 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "started", 2024-09-16T14:49:58.706 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.998890+0000", 2024-09-16T14:49:58.706 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2894999999999998e-05 2024-09-16T14:49:58.707 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.707 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.707 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.707 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:41.999801+0000", 2024-09-16T14:49:58.707 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.00091116000000000005 2024-09-16T14:49:58.707 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.707 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.707 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.707 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.707 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.707 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.17s5 96/95 ECSubWrite(tid=105, reqid=osd.0.0:105, at_version=96'152, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:58.707 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:42.873248+0000", 2024-09-16T14:49:58.707 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 15.749301715, 2024-09-16T14:49:58.707 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0071842269999999996, 2024-09-16T14:49:58.708 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.708 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:58.708 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.708 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.708 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.708 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873248+0000", 2024-09-16T14:49:58.708 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.708 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.708 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.708 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.708 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873248+0000", 2024-09-16T14:49:58.708 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.709 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.709 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.709 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.709 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873248+0000", 2024-09-16T14:49:58.709 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.709 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.709 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.709 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.709 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873268+0000", 2024-09-16T14:49:58.709 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0279e-05 2024-09-16T14:49:58.709 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.709 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.709 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.709 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873281+0000", 2024-09-16T14:49:58.710 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.3149999999999999e-05 2024-09-16T14:49:58.710 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.710 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.710 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.710 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873290+0000", 2024-09-16T14:49:58.710 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.8699999999999998e-06 2024-09-16T14:49:58.710 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.710 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.710 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.710 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873321+0000", 2024-09-16T14:49:58.710 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1430999999999998e-05 2024-09-16T14:49:58.710 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.710 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.710 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.711 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.873329+0000", 2024-09-16T14:49:58.711 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.7700000000000001e-06 2024-09-16T14:49:58.711 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.711 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.711 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.711 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.880401+0000", 2024-09-16T14:49:58.711 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0070722299999999997 2024-09-16T14:49:58.711 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.711 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.711 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.711 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.880432+0000", 2024-09-16T14:49:58.711 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.1173000000000001e-05 2024-09-16T14:49:58.711 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.711 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.712 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.712 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.712 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.712 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.12s6 96/95 ECSubWrite(tid=106, reqid=osd.0.0:106, at_version=96'106, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:58.712 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:42.874711+0000", 2024-09-16T14:49:58.712 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 15.747838613000001, 2024-09-16T14:49:58.712 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0089858890000000004, 2024-09-16T14:49:58.712 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.712 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:58.712 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.712 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.712 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.712 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874711+0000", 2024-09-16T14:49:58.713 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.713 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.713 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.713 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.713 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874711+0000", 2024-09-16T14:49:58.713 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.713 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.713 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.713 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.713 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874711+0000", 2024-09-16T14:49:58.713 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.9999995 2024-09-16T14:49:58.713 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.713 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.714 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.714 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874731+0000", 2024-09-16T14:49:58.714 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.0307000000000001e-05 2024-09-16T14:49:58.714 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.714 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.714 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.714 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874746+0000", 2024-09-16T14:49:58.714 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.4701e-05 2024-09-16T14:49:58.714 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.714 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.714 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.714 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874754+0000", 2024-09-16T14:49:58.714 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.14e-06 2024-09-16T14:49:58.715 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.715 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.715 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.715 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874796+0000", 2024-09-16T14:49:58.715 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4.2651999999999998e-05 2024-09-16T14:49:58.715 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.715 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.715 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.715 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.874805+0000", 2024-09-16T14:49:58.715 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.7900000000000005e-06 2024-09-16T14:49:58.715 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.715 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.715 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.715 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.883673+0000", 2024-09-16T14:49:58.716 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0088682129999999998 2024-09-16T14:49:58.716 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.716 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.716 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.716 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.883697+0000", 2024-09-16T14:49:58.716 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3787000000000001e-05 2024-09-16T14:49:58.716 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.716 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.716 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.716 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.716 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.716 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.17s5 96/95 ECSubWrite(tid=107, reqid=osd.0.0:107, at_version=96'154, trim_to=0'0, roll_forward_to=0'0))", 2024-09-16T14:49:58.716 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:42.875143+0000", 2024-09-16T14:49:58.717 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 15.747406931, 2024-09-16T14:49:58.717 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0084637489999999996, 2024-09-16T14:49:58.717 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.717 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:58.717 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.717 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.717 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.717 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875143+0000", 2024-09-16T14:49:58.717 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.717 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.717 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.717 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.717 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875143+0000", 2024-09-16T14:49:58.717 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.718 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.718 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.718 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.718 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875142+0000", 2024-09-16T14:49:58.718 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:58.718 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.718 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.718 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.718 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875164+0000", 2024-09-16T14:49:58.719 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.1545000000000001e-05 2024-09-16T14:49:58.719 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.719 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.719 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.719 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875173+0000", 2024-09-16T14:49:58.719 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 9.6320000000000003e-06 2024-09-16T14:49:58.719 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.719 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.719 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.719 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875181+0000", 2024-09-16T14:49:58.719 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.5830000000000001e-06 2024-09-16T14:49:58.719 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.720 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.720 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.720 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875241+0000", 2024-09-16T14:49:58.720 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 6.0374999999999997e-05 2024-09-16T14:49:58.720 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.720 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.720 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.720 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.875250+0000", 2024-09-16T14:49:58.720 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.6000000000000007e-06 2024-09-16T14:49:58.720 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.720 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.720 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.720 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.883584+0000", 2024-09-16T14:49:58.720 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0083335600000000003 2024-09-16T14:49:58.721 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.721 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.721 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.721 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.883607+0000", 2024-09-16T14:49:58.721 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 2.3198999999999999e-05 2024-09-16T14:49:58.721 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.721 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.721 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.721 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.721 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.721 INFO:teuthology.orchestra.run.smithi111.stdout: "description": "MOSDECSubOpWrite(3.17s5 96/95 ECSubWrite(tid=108, reqid=client.4500.0:866, at_version=96'156, trim_to=0'0, roll_forward_to=96'152))", 2024-09-16T14:49:58.721 INFO:teuthology.orchestra.run.smithi111.stdout: "initiated_at": "2024-09-16T14:49:42.894666+0000", 2024-09-16T14:49:58.721 INFO:teuthology.orchestra.run.smithi111.stdout: "age": 15.727883947, 2024-09-16T14:49:58.722 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0078174909999999993, 2024-09-16T14:49:58.722 INFO:teuthology.orchestra.run.smithi111.stdout: "type_data": { 2024-09-16T14:49:58.722 INFO:teuthology.orchestra.run.smithi111.stdout: "flag_point": "reached pg", 2024-09-16T14:49:58.722 INFO:teuthology.orchestra.run.smithi111.stdout: "events": [ 2024-09-16T14:49:58.722 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.722 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "initiated", 2024-09-16T14:49:58.722 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.894666+0000", 2024-09-16T14:49:58.722 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.722 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.722 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.722 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "throttled", 2024-09-16T14:49:58.722 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.894666+0000", 2024-09-16T14:49:58.722 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0 2024-09-16T14:49:58.722 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.723 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.723 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "header_read", 2024-09-16T14:49:58.723 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.894665+0000", 2024-09-16T14:49:58.723 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 4294967295.999999 2024-09-16T14:49:58.723 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.723 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.723 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "all_read", 2024-09-16T14:49:58.723 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.899248+0000", 2024-09-16T14:49:58.723 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0045826900000000004 2024-09-16T14:49:58.723 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.723 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.723 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "dispatched", 2024-09-16T14:49:58.723 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.899267+0000", 2024-09-16T14:49:58.723 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9718999999999999e-05 2024-09-16T14:49:58.724 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.724 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.724 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "queued_for_pg", 2024-09-16T14:49:58.724 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.899275+0000", 2024-09-16T14:49:58.724 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 7.9500000000000001e-06 2024-09-16T14:49:58.724 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.724 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.724 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "reached_pg", 2024-09-16T14:49:58.724 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.899315+0000", 2024-09-16T14:49:58.724 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 3.9246999999999999e-05 2024-09-16T14:49:58.724 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_started", 2024-09-16T14:49:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.899323+0000", 2024-09-16T14:49:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 8.2320000000000001e-06 2024-09-16T14:49:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "sub_op_committed", 2024-09-16T14:49:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.902464+0000", 2024-09-16T14:49:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 0.0031410779999999998 2024-09-16T14:49:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-16T14:49:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-16T14:49:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: "event": "done", 2024-09-16T14:49:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: "time": "2024-09-16T14:49:42.902483+0000", 2024-09-16T14:49:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: "duration": 1.9491999999999999e-05 2024-09-16T14:49:58.726 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.726 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.726 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.726 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-16T14:49:58.726 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-16T14:49:58.726 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-16T14:49:59.106 INFO:teuthology.orchestra.run.smithi040.stderr:noscrub is unset 2024-09-16T14:49:59.123 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T14:50:00.055 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-16T14:50:00.116 INFO:teuthology.orchestra.run.smithi040.stderr:nodeep-scrub is unset 2024-09-16T14:50:00.189 INFO:teuthology.orchestra.run.smithi040.stdout:mon.a: {} 2024-09-16T14:50:00.189 INFO:teuthology.orchestra.run.smithi040.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T14:50:00.199 INFO:teuthology.orchestra.run.smithi040.stdout:mon.b: {} 2024-09-16T14:50:00.199 INFO:teuthology.orchestra.run.smithi040.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T14:50:00.411 INFO:teuthology.orchestra.run.smithi040.stdout:mon.c: {} 2024-09-16T14:50:00.411 INFO:teuthology.orchestra.run.smithi040.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T14:50:00.422 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T14:50:00.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:50:00.731+0000 7fbe06ffd640 0 --2- 172.21.15.40:0/491540164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf00039c0 0x7fbdf0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T14:50:01.122 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T14:50:01.136 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-16T14:50:02.128 INFO:teuthology.orchestra.run.smithi040.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-16T14:50:02.145 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T14:50:03.133 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T14:50:03.146 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-16T14:50:04.141 INFO:teuthology.orchestra.run.smithi040.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-16T14:50:04.155 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T14:50:05.143 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T14:50:05.160 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-16T14:50:06.151 INFO:teuthology.orchestra.run.smithi040.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-16T14:50:06.167 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T14:50:07.153 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T14:50:07.170 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-16T14:50:08.162 INFO:teuthology.orchestra.run.smithi040.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-16T14:50:08.178 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T14:50:09.167 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T14:50:09.183 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-16T14:50:10.174 INFO:teuthology.orchestra.run.smithi040.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-16T14:50:10.191 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T14:50:11.184 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T14:50:11.201 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-16T14:50:12.192 INFO:teuthology.orchestra.run.smithi040.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-16T14:50:12.207 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T14:50:13.198 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T14:50:13.213 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-16T14:50:14.203 INFO:teuthology.orchestra.run.smithi040.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-16T14:50:14.217 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-09-16T14:50:15.209 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.8 to 1 (10000) 2024-09-16T14:50:15.225 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 8 1 2024-09-16T14:50:16.214 INFO:teuthology.orchestra.run.smithi040.stderr:set osd.8 primary-affinity to 1 (8655362) 2024-09-16T14:50:16.230 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-09-16T14:50:17.223 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.9 to 1 (10000) 2024-09-16T14:50:17.240 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 9 1 2024-09-16T14:50:18.231 INFO:teuthology.orchestra.run.smithi040.stderr:set osd.9 primary-affinity to 1 (8655362) 2024-09-16T14:50:18.247 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-09-16T14:50:19.239 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.10 to 1 (10000) 2024-09-16T14:50:19.255 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 10 1 2024-09-16T14:50:20.251 INFO:teuthology.orchestra.run.smithi040.stderr:set osd.10 primary-affinity to 1 (8655362) 2024-09-16T14:50:20.266 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-09-16T14:50:20.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-16T14:50:20.576+0000 7f4673fff640 0 --2- 172.21.15.40:0/1181150676 >> v2:172.21.15.111:3300/0 conn(0x7f467406a140 0x7f4674145d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-16T14:50:21.250 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.11 to 1 (10000) 2024-09-16T14:50:21.266 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 11 1 2024-09-16T14:50:22.259 INFO:teuthology.orchestra.run.smithi040.stderr:set osd.11 primary-affinity to 1 (8655362) 2024-09-16T14:50:22.275 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T14:50:23.261 INFO:teuthology.orchestra.run.smithi040.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T14:50:23.277 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-16T14:50:24.265 INFO:teuthology.orchestra.run.smithi040.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-16T14:50:24.282 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-16T14:50:24.282 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-16T14:50:24.282 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-16T14:50:24.282 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-16T14:50:24.282 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T14:50:24.282 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:50:24.577 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:50:24.577 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":134,"fsid":"aaaa1c0f-de8b-48c8-9412-815c8b70b7b8","created":"2024-09-16T14:48:10.063465+0000","modified":"2024-09-16T14:50:24.261612+0000","last_up_change":"2024-09-16T14:48:47.933230+0000","last_in_change":"2024-09-16T14:49:11.939833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:48:20.279244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T14:48:22.580854+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":0.75,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffcfdeb7-eb75-4b3a-bc2a-f1834271654c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":725262},{"type":"v1","addr":"172.21.15.40:6801","nonce":725262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":725262},{"type":"v1","addr":"172.21.15.40:6803","nonce":725262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":725262},{"type":"v1","addr":"172.21.15.40:6807","nonce":725262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":725262},{"type":"v1","addr":"172.21.15.40:6805","nonce":725262}]},"public_addr":"172.21.15.40:6801/725262","cluster_addr":"172.21.15.40:6803/725262","heartbeat_back_addr":"172.21.15.40:6807/725262","heartbeat_front_addr":"172.21.15.40:6805/725262","state":["exists","up"]},{"osd":1,"uuid":"6fb8c93f-5e3c-492f-bd0c-6d97764467f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6821","nonce":501335559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6825","nonce":501335559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6831","nonce":501335559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6829","nonce":501335559}]},"public_addr":"172.21.15.40:6821/501335559","cluster_addr":"172.21.15.40:6825/501335559","heartbeat_back_addr":"172.21.15.40:6831/501335559","heartbeat_front_addr":"172.21.15.40:6829/501335559","state":["exists","up"]},{"osd":2,"uuid":"500457a9-0d59-451c-b2ff-28aa363056bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6809","nonce":1490162929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6811","nonce":1490162929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6815","nonce":1490162929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6813","nonce":1490162929}]},"public_addr":"172.21.15.40:6809/1490162929","cluster_addr":"172.21.15.40:6811/1490162929","heartbeat_back_addr":"172.21.15.40:6815/1490162929","heartbeat_front_addr":"172.21.15.40:6813/1490162929","state":["exists","up"]},{"osd":3,"uuid":"f6e48d98-5d87-4cd5-8bd8-6e4c9dc56a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6817","nonce":682463486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6820","nonce":682463486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6828","nonce":682463486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6824","nonce":682463486}]},"public_addr":"172.21.15.40:6817/682463486","cluster_addr":"172.21.15.40:6820/682463486","heartbeat_back_addr":"172.21.15.40:6828/682463486","heartbeat_front_addr":"172.21.15.40:6824/682463486","state":["exists","up"]},{"osd":4,"uuid":"474dd40c-2b73-44ea-b6a9-48605b78125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6818","nonce":1550824417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6819","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6821","nonce":1550824417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6828","nonce":1550824417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6824","nonce":1550824417}]},"public_addr":"172.21.15.111:6818/1550824417","cluster_addr":"172.21.15.111:6821/1550824417","heartbeat_back_addr":"172.21.15.111:6828/1550824417","heartbeat_front_addr":"172.21.15.111:6824/1550824417","state":["exists","up"]},{"osd":5,"uuid":"479bfe9b-00f6-4763-9740-9c9fd11e2587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6801","nonce":3785556374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6803","nonce":3785556374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6807","nonce":3785556374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6805","nonce":3785556374}]},"public_addr":"172.21.15.111:6801/3785556374","cluster_addr":"172.21.15.111:6803/3785556374","heartbeat_back_addr":"172.21.15.111:6807/3785556374","heartbeat_front_addr":"172.21.15.111:6805/3785556374","state":["exists","up"]},{"osd":6,"uuid":"6ac36041-045e-4813-ad35-ca4e224811fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":23,"up_from":41,"up_thru":92,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6820","nonce":722937064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6825","nonce":722937064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6831","nonce":722937064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6827","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6829","nonce":722937064}]},"public_addr":"172.21.15.111:6820/722937064","cluster_addr":"172.21.15.111:6825/722937064","heartbeat_back_addr":"172.21.15.111:6831/722937064","heartbeat_front_addr":"172.21.15.111:6829/722937064","state":["exists","up"]},{"osd":7,"uuid":"129006cf-b82b-4e11-a007-63deb355d007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6809","nonce":3591930331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6811","nonce":3591930331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6815","nonce":3591930331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6813","nonce":3591930331}]},"public_addr":"172.21.15.111:6809/3591930331","cluster_addr":"172.21.15.111:6811/3591930331","heartbeat_back_addr":"172.21.15.111:6815/3591930331","heartbeat_front_addr":"172.21.15.111:6813/3591930331","state":["exists","up"]},{"osd":8,"uuid":"27e746ee-f67a-4a73-8976-86d012469657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6817","nonce":2473986485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6819","nonce":2473986485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6824","nonce":2473986485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6821","nonce":2473986485}]},"public_addr":"172.21.15.114:6817/2473986485","cluster_addr":"172.21.15.114:6819/2473986485","heartbeat_back_addr":"172.21.15.114:6824/2473986485","heartbeat_front_addr":"172.21.15.114:6821/2473986485","state":["exists","up"]},{"osd":9,"uuid":"90975d4f-24b3-4c84-916e-3138bc52a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6801","nonce":2290892207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6803","nonce":2290892207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6807","nonce":2290892207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6805","nonce":2290892207}]},"public_addr":"172.21.15.114:6801/2290892207","cluster_addr":"172.21.15.114:6803/2290892207","heartbeat_back_addr":"172.21.15.114:6807/2290892207","heartbeat_front_addr":"172.21.15.114:6805/2290892207","state":["exists","up"]},{"osd":10,"uuid":"89c8157b-52ce-4c1c-9df7-e214df30a827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6809","nonce":1795470140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6811","nonce":1795470140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6815","nonce":1795470140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6813","nonce":1795470140}]},"public_addr":"172.21.15.114:6809/1795470140","cluster_addr":"172.21.15.114:6811/1795470140","heartbeat_back_addr":"172.21.15.114:6815/1795470140","heartbeat_front_addr":"172.21.15.114:6813/1795470140","state":["exists","up"]},{"osd":11,"uuid":"0dfbd657-c9f0-42f9-9b43-323c45d89c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6825","nonce":1823608150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6827","nonce":1823608150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6831","nonce":1823608150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6829","nonce":1823608150}]},"public_addr":"172.21.15.114:6825/1823608150","cluster_addr":"172.21.15.114:6827/1823608150","heartbeat_back_addr":"172.21.15.114:6831/1823608150","heartbeat_front_addr":"172.21.15.114:6829/1823608150","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:21.058050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:29.713159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:46.059619+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:45.409424+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:19.879455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:39.305292+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:32.379251+0000","dead_epoch":25},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:25.580810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:32.215102+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:19.285409+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:36.675606+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:32.320290+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"lrcprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","l":"3","m":"2","name":"lrcprofile","plugin":"lrc"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":70,"length":1},{"begin":76,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:50:24.590 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T14:50:24.590 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:50:24.874 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:50:24.874 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":134,"fsid":"aaaa1c0f-de8b-48c8-9412-815c8b70b7b8","created":"2024-09-16T14:48:10.063465+0000","modified":"2024-09-16T14:50:24.261612+0000","last_up_change":"2024-09-16T14:48:47.933230+0000","last_in_change":"2024-09-16T14:49:11.939833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:48:20.279244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T14:48:22.580854+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":0.75,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffcfdeb7-eb75-4b3a-bc2a-f1834271654c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":725262},{"type":"v1","addr":"172.21.15.40:6801","nonce":725262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":725262},{"type":"v1","addr":"172.21.15.40:6803","nonce":725262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":725262},{"type":"v1","addr":"172.21.15.40:6807","nonce":725262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":725262},{"type":"v1","addr":"172.21.15.40:6805","nonce":725262}]},"public_addr":"172.21.15.40:6801/725262","cluster_addr":"172.21.15.40:6803/725262","heartbeat_back_addr":"172.21.15.40:6807/725262","heartbeat_front_addr":"172.21.15.40:6805/725262","state":["exists","up"]},{"osd":1,"uuid":"6fb8c93f-5e3c-492f-bd0c-6d97764467f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6821","nonce":501335559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6825","nonce":501335559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6831","nonce":501335559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6829","nonce":501335559}]},"public_addr":"172.21.15.40:6821/501335559","cluster_addr":"172.21.15.40:6825/501335559","heartbeat_back_addr":"172.21.15.40:6831/501335559","heartbeat_front_addr":"172.21.15.40:6829/501335559","state":["exists","up"]},{"osd":2,"uuid":"500457a9-0d59-451c-b2ff-28aa363056bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6809","nonce":1490162929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6811","nonce":1490162929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6815","nonce":1490162929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6813","nonce":1490162929}]},"public_addr":"172.21.15.40:6809/1490162929","cluster_addr":"172.21.15.40:6811/1490162929","heartbeat_back_addr":"172.21.15.40:6815/1490162929","heartbeat_front_addr":"172.21.15.40:6813/1490162929","state":["exists","up"]},{"osd":3,"uuid":"f6e48d98-5d87-4cd5-8bd8-6e4c9dc56a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6817","nonce":682463486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6820","nonce":682463486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6828","nonce":682463486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6824","nonce":682463486}]},"public_addr":"172.21.15.40:6817/682463486","cluster_addr":"172.21.15.40:6820/682463486","heartbeat_back_addr":"172.21.15.40:6828/682463486","heartbeat_front_addr":"172.21.15.40:6824/682463486","state":["exists","up"]},{"osd":4,"uuid":"474dd40c-2b73-44ea-b6a9-48605b78125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6818","nonce":1550824417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6819","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6821","nonce":1550824417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6828","nonce":1550824417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6824","nonce":1550824417}]},"public_addr":"172.21.15.111:6818/1550824417","cluster_addr":"172.21.15.111:6821/1550824417","heartbeat_back_addr":"172.21.15.111:6828/1550824417","heartbeat_front_addr":"172.21.15.111:6824/1550824417","state":["exists","up"]},{"osd":5,"uuid":"479bfe9b-00f6-4763-9740-9c9fd11e2587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6801","nonce":3785556374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6803","nonce":3785556374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6807","nonce":3785556374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6805","nonce":3785556374}]},"public_addr":"172.21.15.111:6801/3785556374","cluster_addr":"172.21.15.111:6803/3785556374","heartbeat_back_addr":"172.21.15.111:6807/3785556374","heartbeat_front_addr":"172.21.15.111:6805/3785556374","state":["exists","up"]},{"osd":6,"uuid":"6ac36041-045e-4813-ad35-ca4e224811fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":23,"up_from":41,"up_thru":92,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6820","nonce":722937064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6825","nonce":722937064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6831","nonce":722937064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6827","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6829","nonce":722937064}]},"public_addr":"172.21.15.111:6820/722937064","cluster_addr":"172.21.15.111:6825/722937064","heartbeat_back_addr":"172.21.15.111:6831/722937064","heartbeat_front_addr":"172.21.15.111:6829/722937064","state":["exists","up"]},{"osd":7,"uuid":"129006cf-b82b-4e11-a007-63deb355d007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6809","nonce":3591930331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6811","nonce":3591930331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6815","nonce":3591930331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6813","nonce":3591930331}]},"public_addr":"172.21.15.111:6809/3591930331","cluster_addr":"172.21.15.111:6811/3591930331","heartbeat_back_addr":"172.21.15.111:6815/3591930331","heartbeat_front_addr":"172.21.15.111:6813/3591930331","state":["exists","up"]},{"osd":8,"uuid":"27e746ee-f67a-4a73-8976-86d012469657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6817","nonce":2473986485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6819","nonce":2473986485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6824","nonce":2473986485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6821","nonce":2473986485}]},"public_addr":"172.21.15.114:6817/2473986485","cluster_addr":"172.21.15.114:6819/2473986485","heartbeat_back_addr":"172.21.15.114:6824/2473986485","heartbeat_front_addr":"172.21.15.114:6821/2473986485","state":["exists","up"]},{"osd":9,"uuid":"90975d4f-24b3-4c84-916e-3138bc52a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6801","nonce":2290892207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6803","nonce":2290892207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6807","nonce":2290892207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6805","nonce":2290892207}]},"public_addr":"172.21.15.114:6801/2290892207","cluster_addr":"172.21.15.114:6803/2290892207","heartbeat_back_addr":"172.21.15.114:6807/2290892207","heartbeat_front_addr":"172.21.15.114:6805/2290892207","state":["exists","up"]},{"osd":10,"uuid":"89c8157b-52ce-4c1c-9df7-e214df30a827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6809","nonce":1795470140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6811","nonce":1795470140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6815","nonce":1795470140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6813","nonce":1795470140}]},"public_addr":"172.21.15.114:6809/1795470140","cluster_addr":"172.21.15.114:6811/1795470140","heartbeat_back_addr":"172.21.15.114:6815/1795470140","heartbeat_front_addr":"172.21.15.114:6813/1795470140","state":["exists","up"]},{"osd":11,"uuid":"0dfbd657-c9f0-42f9-9b43-323c45d89c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6825","nonce":1823608150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6827","nonce":1823608150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6831","nonce":1823608150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6829","nonce":1823608150}]},"public_addr":"172.21.15.114:6825/1823608150","cluster_addr":"172.21.15.114:6827/1823608150","heartbeat_back_addr":"172.21.15.114:6831/1823608150","heartbeat_front_addr":"172.21.15.114:6829/1823608150","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:21.058050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:29.713159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:46.059619+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:45.409424+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:19.879455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:39.305292+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:32.379251+0000","dead_epoch":25},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:25.580810+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:32.215102+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:19.285409+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:36.675606+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:32.320290+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"lrcprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","l":"3","m":"2","name":"lrcprofile","plugin":"lrc"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":70,"length":1},{"begin":76,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:50:24.887 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T14:50:24.887 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T14:50:24.888 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T14:50:24.888 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T14:50:24.888 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T14:50:24.889 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T14:50:24.889 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T14:50:24.890 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T14:50:24.890 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-16T14:50:24.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-16T14:50:24.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-16T14:50:24.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-16T14:50:25.219 INFO:teuthology.orchestra.run.smithi040.stdout:73014444060 2024-09-16T14:50:25.220 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T14:50:25.230 INFO:teuthology.orchestra.run.smithi040.stdout:73014444060 2024-09-16T14:50:25.231 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-16T14:50:25.233 INFO:teuthology.orchestra.run.smithi040.stdout:77309411357 2024-09-16T14:50:25.234 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T14:50:25.242 INFO:teuthology.orchestra.run.smithi040.stdout:73014444059 2024-09-16T14:50:25.243 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T14:50:25.250 INFO:teuthology.orchestra.run.smithi040.stdout:77309411356 2024-09-16T14:50:25.251 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T14:50:25.251 INFO:teuthology.orchestra.run.smithi040.stdout:73014444061 2024-09-16T14:50:25.252 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T14:50:25.257 INFO:teuthology.orchestra.run.smithi040.stdout:73014444060 2024-09-16T14:50:25.258 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T14:50:25.260 INFO:teuthology.orchestra.run.smithi040.stdout:77309411355 2024-09-16T14:50:25.261 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T14:50:25.265 INFO:teuthology.orchestra.run.smithi040.stdout:77309411357 2024-09-16T14:50:25.266 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-16T14:50:25.270 INFO:teuthology.orchestra.run.smithi040.stdout:73014444060 2024-09-16T14:50:25.270 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-16T14:50:25.285 INFO:teuthology.orchestra.run.smithi040.stdout:176093659158 2024-09-16T14:50:25.285 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T14:50:25.313 INFO:teuthology.orchestra.run.smithi040.stdout:77309411355 2024-09-16T14:50:25.314 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-16T14:50:25.946 INFO:teuthology.orchestra.run.smithi040.stdout:73014444059 2024-09-16T14:50:25.947 INFO:teuthology.orchestra.run.smithi040.stdout:77309411356 2024-09-16T14:50:25.980 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444060 got 73014444059 for osd.9 2024-09-16T14:50:25.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411357 got 77309411356 for osd.1 2024-09-16T14:50:25.997 INFO:teuthology.orchestra.run.smithi040.stdout:73014444060 2024-09-16T14:50:26.013 INFO:teuthology.orchestra.run.smithi040.stdout:73014444058 2024-09-16T14:50:26.023 INFO:teuthology.orchestra.run.smithi040.stdout:73014444059 2024-09-16T14:50:26.025 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444061 got 73014444060 for osd.3 2024-09-16T14:50:26.052 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444060 got 73014444059 for osd.2 2024-09-16T14:50:26.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444059 got 73014444058 for osd.4 2024-09-16T14:50:26.073 INFO:teuthology.orchestra.run.smithi040.stdout:77309411355 2024-09-16T14:50:26.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411356 got 77309411355 for osd.5 2024-09-16T14:50:26.109 INFO:teuthology.orchestra.run.smithi040.stdout:77309411356 2024-09-16T14:50:26.124 INFO:teuthology.orchestra.run.smithi040.stdout:77309411354 2024-09-16T14:50:26.128 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411357 got 77309411356 for osd.10 2024-09-16T14:50:26.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411355 got 77309411354 for osd.0 2024-09-16T14:50:26.149 INFO:teuthology.orchestra.run.smithi040.stdout:73014444059 2024-09-16T14:50:26.154 INFO:teuthology.orchestra.run.smithi040.stdout:176093659157 2024-09-16T14:50:26.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444060 got 73014444059 for osd.7 2024-09-16T14:50:26.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 176093659158 got 176093659157 for osd.6 2024-09-16T14:50:26.192 INFO:teuthology.orchestra.run.smithi040.stdout:73014444059 2024-09-16T14:50:26.205 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444060 got 73014444059 for osd.8 2024-09-16T14:50:26.208 INFO:teuthology.orchestra.run.smithi040.stdout:77309411354 2024-09-16T14:50:26.220 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411355 got 77309411354 for osd.11 2024-09-16T14:50:26.981 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-16T14:50:26.982 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T14:50:27.026 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T14:50:27.053 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T14:50:27.054 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T14:50:27.094 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T14:50:27.129 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-16T14:50:27.138 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T14:50:27.163 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T14:50:27.168 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T14:50:27.206 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-16T14:50:27.221 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-16T14:50:27.558 INFO:teuthology.orchestra.run.smithi040.stdout:73014444060 2024-09-16T14:50:27.618 INFO:teuthology.orchestra.run.smithi040.stdout:73014444061 2024-09-16T14:50:27.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444060 got 73014444060 for osd.9 2024-09-16T14:50:27.628 DEBUG:teuthology.parallel:result is None 2024-09-16T14:50:27.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444061 got 73014444061 for osd.3 2024-09-16T14:50:27.639 DEBUG:teuthology.parallel:result is None 2024-09-16T14:50:27.700 INFO:teuthology.orchestra.run.smithi040.stdout:77309411357 2024-09-16T14:50:27.702 INFO:teuthology.orchestra.run.smithi040.stdout:73014444059 2024-09-16T14:50:27.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444059 got 73014444059 for osd.4 2024-09-16T14:50:27.722 DEBUG:teuthology.parallel:result is None 2024-09-16T14:50:27.757 INFO:teuthology.orchestra.run.smithi040.stdout:73014444060 2024-09-16T14:50:27.765 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411357 got 77309411357 for osd.1 2024-09-16T14:50:27.765 DEBUG:teuthology.parallel:result is None 2024-09-16T14:50:27.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444060 got 73014444060 for osd.2 2024-09-16T14:50:27.782 DEBUG:teuthology.parallel:result is None 2024-09-16T14:50:27.828 INFO:teuthology.orchestra.run.smithi040.stdout:77309411357 2024-09-16T14:50:27.841 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411357 got 77309411357 for osd.10 2024-09-16T14:50:27.841 DEBUG:teuthology.parallel:result is None 2024-09-16T14:50:27.842 INFO:teuthology.orchestra.run.smithi040.stdout:77309411356 2024-09-16T14:50:27.850 INFO:teuthology.orchestra.run.smithi040.stdout:77309411355 2024-09-16T14:50:27.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411356 got 77309411356 for osd.5 2024-09-16T14:50:27.861 DEBUG:teuthology.parallel:result is None 2024-09-16T14:50:27.869 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411355 got 77309411355 for osd.0 2024-09-16T14:50:27.870 DEBUG:teuthology.parallel:result is None 2024-09-16T14:50:27.926 INFO:teuthology.orchestra.run.smithi040.stdout:176093659158 2024-09-16T14:50:27.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 176093659158 got 176093659158 for osd.6 2024-09-16T14:50:27.944 DEBUG:teuthology.parallel:result is None 2024-09-16T14:50:27.947 INFO:teuthology.orchestra.run.smithi040.stdout:77309411355 2024-09-16T14:50:27.956 INFO:teuthology.orchestra.run.smithi040.stdout:73014444060 2024-09-16T14:50:27.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411355 got 77309411355 for osd.11 2024-09-16T14:50:27.962 INFO:teuthology.orchestra.run.smithi040.stdout:73014444060 2024-09-16T14:50:27.962 DEBUG:teuthology.parallel:result is None 2024-09-16T14:50:27.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444060 got 73014444060 for osd.8 2024-09-16T14:50:27.971 DEBUG:teuthology.parallel:result is None 2024-09-16T14:50:27.973 INFO:tasks.ceph.ceph_manager.ceph:need seq 73014444060 got 73014444060 for osd.7 2024-09-16T14:50:27.973 DEBUG:teuthology.parallel:result is None 2024-09-16T14:50:27.974 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T14:50:27.974 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:50:28.299 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:50:28.299 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:50:28.311 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-16T14:50:28.054349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":183,"num_write_kb":2940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":599724,"kb_used_data":19716,"kb_used_omap":19,"kb_used_meta":579820,"kb_avail":1131862356,"statfs":{"total":1159641169920,"available":1159027052544,"internally_reserved":0,"allocated":20189184,"data_stored":12998546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19502,"internal_metadata":593736658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.797108"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":87,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.398990+0000","last_change":"2024-09-16T14:50:07.667627+0000","last_active":"2024-09-16T14:50:24.398990+0000","last_peered":"2024-09-16T14:50:24.398990+0000","last_clean":"2024-09-16T14:50:24.398990+0000","last_became_active":"2024-09-16T14:49:12.961697+0000","last_became_peered":"2024-09-16T14:49:12.961697+0000","last_unstale":"2024-09-16T14:50:24.398990+0000","last_undegraded":"2024-09-16T14:50:24.398990+0000","last_fullsized":"2024-09-16T14:50:24.398990+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:50:07.667489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:50:07.667489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:24.377449+0000","scrub_duration":0.002219033,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.332431+0000","last_change":"2024-09-16T14:49:54.393780+0000","last_active":"2024-09-16T14:50:24.332431+0000","last_peered":"2024-09-16T14:50:24.332431+0000","last_clean":"2024-09-16T14:50:24.332431+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:50:24.332431+0000","last_undegraded":"2024-09-16T14:50:24.332431+0000","last_fullsized":"2024-09-16T14:50:24.332431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:54.393679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:54.393679+0000","last_clean_scrub_stamp":"2024-09-16T14:49:54.393679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:09.244075+0000","scrub_duration":0.0015851929999999999,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":135,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260080+0000","last_change":"2024-09-16T14:49:43.127050+0000","last_active":"2024-09-16T14:50:23.260080+0000","last_peered":"2024-09-16T14:50:23.260080+0000","last_clean":"2024-09-16T14:50:23.260080+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:50:23.260080+0000","last_undegraded":"2024-09-16T14:50:23.260080+0000","last_fullsized":"2024-09-16T14:50:23.260080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_clean_scrub_stamp":"2024-09-16T14:49:43.126944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:52.930143+0000","scrub_duration":0.0012658890000000001,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.332481+0000","last_change":"2024-09-16T14:49:55.441559+0000","last_active":"2024-09-16T14:50:24.332481+0000","last_peered":"2024-09-16T14:50:24.332481+0000","last_clean":"2024-09-16T14:50:24.332481+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:50:24.332481+0000","last_undegraded":"2024-09-16T14:50:24.332481+0000","last_fullsized":"2024-09-16T14:50:24.332481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:55.441436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:55.441436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:00.621776+0000","scrub_duration":0.0011147150000000001,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":138,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260124+0000","last_change":"2024-09-16T14:49:28.351058+0000","last_active":"2024-09-16T14:50:23.260124+0000","last_peered":"2024-09-16T14:50:23.260124+0000","last_clean":"2024-09-16T14:50:23.260124+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:50:23.260124+0000","last_undegraded":"2024-09-16T14:50:23.260124+0000","last_fullsized":"2024-09-16T14:50:23.260124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-16T14:49:28.350938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:28.350938+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:42.292353+0000","scrub_duration":0.0018950309999999999,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.245674+0000","last_change":"2024-09-16T14:49:27.155463+0000","last_active":"2024-09-16T14:50:24.245674+0000","last_peered":"2024-09-16T14:50:24.245674+0000","last_clean":"2024-09-16T14:50:24.245674+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:50:24.245674+0000","last_undegraded":"2024-09-16T14:50:24.245674+0000","last_fullsized":"2024-09-16T14:50:24.245674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:27.155307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:27.155307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:32.986356+0000","scrub_duration":0.001601157,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":137,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.833368+0000","last_change":"2024-09-16T14:49:53.408437+0000","last_active":"2024-09-16T14:50:24.833368+0000","last_peered":"2024-09-16T14:50:24.833368+0000","last_clean":"2024-09-16T14:50:24.833368+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:50:24.833368+0000","last_undegraded":"2024-09-16T14:50:24.833368+0000","last_fullsized":"2024-09-16T14:50:24.833368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:53.408348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:53.408348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:16.057283+0000","scrub_duration":0.0016182830000000001,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":137,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260124+0000","last_change":"2024-09-16T14:49:40.527258+0000","last_active":"2024-09-16T14:50:23.260124+0000","last_peered":"2024-09-16T14:50:23.260124+0000","last_clean":"2024-09-16T14:50:23.260124+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:50:23.260124+0000","last_undegraded":"2024-09-16T14:50:23.260124+0000","last_fullsized":"2024-09-16T14:50:23.260124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2024-09-16T14:49:40.527164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.527164+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:46.029615+0000","scrub_duration":0.0012857120000000001,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":306,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.833372+0000","last_change":"2024-09-16T14:49:59.398994+0000","last_active":"2024-09-16T14:50:24.833372+0000","last_peered":"2024-09-16T14:50:24.833372+0000","last_clean":"2024-09-16T14:50:24.833372+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:50:24.833372+0000","last_undegraded":"2024-09-16T14:50:24.833372+0000","last_fullsized":"2024-09-16T14:50:24.833372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'137","last_scrub_stamp":"2024-09-16T14:49:59.398890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:49:59.398890+0000","objects_scrubbed":2,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:10.229835+0000","scrub_duration":0.001723186,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411356,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55336,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316504,"statfs":{"total":96636764160,"available":96580100096,"internally_reserved":0,"allocated":2125824,"data_stored":1524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.80900000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.83699999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79500000000000004,"5min":1.044,"15min":1.044},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.75800000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94199999999999995,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81999999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.81999999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.85199999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.66100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.72399999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.622},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68500000000000005,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.86699999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.377,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.629,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.56799999999999995}]},{"osd":9,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.77900000000000003}]},{"osd":10,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.5,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.67200000000000004}]}]},{"osd":10,"up_from":18,"seq":77309411358,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1589248,"data_stored":994178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.73199999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.73899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.753},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.872}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.79200000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71299999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72299999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.66400000000000003}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71899999999999997}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.64600000000000002}]}]},{"osd":6,"up_from":41,"seq":176093659159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":30974,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1965,"internal_metadata":31717459},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.66500000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67500000000000004}]},{"osd":3,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.54600000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48999999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.314},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.39600000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65300000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.624}]},{"osd":10,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.50600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":1,"up_from":18,"seq":77309411358,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72872,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298968,"statfs":{"total":96636764160,"available":96562143232,"internally_reserved":0,"allocated":1597440,"data_stored":995153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.54600000000000004,"15min":0.57199999999999995},"min":{"1min":0.26400000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70499999999999996,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.439,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.81999999999999995,"5min":1.806,"15min":1.806},"last":0.61899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.26000000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.872},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.58999999999999997,"15min":0.61299999999999999},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92700000000000005,"5min":4.0590000000000002,"15min":4.0590000000000002},"last":0.92700000000000005}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.435}]},{"osd":6,"last update":"Mon Sep 16 14:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46600000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91000000000000003,"5min":1.831,"15min":1.831},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.77500000000000002,"15min":0.79600000000000004},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.73199999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.55500000000000005,"5min":0.371,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.85799999999999998,"15min":0.875},"min":{"1min":0.54500000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.069,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.81000000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73599999999999999,"15min":0.73499999999999999},"min":{"1min":0.54100000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1299999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.63400000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.71999999999999997}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.89800000000000002,"15min":0.92300000000000004},"min":{"1min":0.61099999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1000000000000001,"5min":4.6589999999999998,"15min":4.6589999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.091,"5min":1.855,"15min":1.855},"last":0.749}]}]},{"osd":0,"up_from":18,"seq":77309411356,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55600000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.32200000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.39500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39900000000000002}]},{"osd":6,"last update":"Mon Sep 16 14:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.499},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53000000000000003}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66500000000000004,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.51100000000000001}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.55100000000000005}]},{"osd":10,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.60899999999999999}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.623},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56599999999999995}]}]},{"osd":5,"up_from":18,"seq":77309411357,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50724,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321116,"statfs":{"total":96636764160,"available":96584822784,"internally_reserved":0,"allocated":1597440,"data_stored":995153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.433}]},{"osd":1,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.505},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.57899999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68400000000000005,"5min":0.877,"15min":0.877},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.66000000000000003,"5min":0.746,"15min":0.746},"last":0.45500000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.621}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.31,"5min":0.252,"15min":0.252},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58799999999999997}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.753}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.72799999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91100000000000003,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.877}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85699999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.204,"5min":1.502,"15min":1.502},"last":0.91800000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78300000000000003,"15min":0.77600000000000002},"min":{"1min":0.53500000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2270000000000001,"5min":1.456,"15min":1.456},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.746,"15min":0.752},"min":{"1min":0.501,"5min":0.499,"15min":0.499},"max":{"1min":0.96699999999999997,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.83099999999999996}]}]},{"osd":9,"up_from":17,"seq":73014444061,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.436},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54200000000000004}]},{"osd":1,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.626,"15min":0.61299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.35699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":1.601,"15min":1.601},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52500000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":3.0019999999999998,"5min":3.0019999999999998,"15min":3.0019999999999998},"last":0.69699999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.65300000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98599999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.40400000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.93000000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85399999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.39400000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0620000000000001,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.81000000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68500000000000005,"15min":0.69699999999999995},"min":{"1min":0.38900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.876,"5min":1.47,"15min":1.47},"last":0.72999999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81299999999999994,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.35599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95799999999999996,"5min":1.014,"15min":1.014},"last":0.71799999999999997}]},{"osd":10,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84199999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.503},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.623}]},{"osd":11,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.99299999999999999,"5min":1.518,"15min":1.518},"last":0.74299999999999999}]}]},{"osd":8,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.38100000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.47099999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.34200000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.54200000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85299999999999998,"5min":1.006,"15min":1.006},"last":0.85299999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.78700000000000003}]},{"osd":6,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.88300000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0680000000000001,"5min":1.202,"15min":1.202},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.89300000000000002,"5min":1.097,"15min":1.097},"last":0.91200000000000003}]},{"osd":9,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82999999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79700000000000004,"5min":1.29,"15min":1.29},"last":0.71699999999999997}]},{"osd":10,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83199999999999996,"5min":1.22,"15min":1.22},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.48799999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94399999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.89200000000000002}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.42999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.496,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":1.254,"15min":1.254},"last":0.63900000000000001}]}]},{"osd":7,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51240,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320600,"statfs":{"total":96636764160,"available":96584294400,"internally_reserved":0,"allocated":2125824,"data_stored":1524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":50330060},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5620000000000003,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77100000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68300000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56299999999999994}]},{"osd":3,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83799999999999997,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.33300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.871,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.67300000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.76500000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:49:52 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.73799999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.44,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67300000000000004,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96499999999999997,"5min":1.163,"15min":1.163},"last":0.68600000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.93200000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.90800000000000003,"5min":1.026,"15min":1.026},"last":0.75600000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90700000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.70399999999999996},"min":{"1min":0.48899999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98399999999999999,"5min":1.079,"15min":1.079},"last":0.80000000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.52600000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0009999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0129999999999999,"5min":1.04,"15min":1.04},"last":0.77100000000000002}]}]},{"osd":4,"up_from":17,"seq":73014444060,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.68500000000000005}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.82699999999999996}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.501}]},{"osd":6,"last update":"Mon Sep 16 14:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.42699999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.373}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.57699999999999996}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64500000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.749}]},{"osd":11,"last update":"Mon Sep 16 14:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.72199999999999998}]}]},{"osd":3,"up_from":17,"seq":73014444062,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50724,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321116,"statfs":{"total":96636764160,"available":96584822784,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.28699999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.29899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.56000000000000005}]},{"osd":4,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.65900000000000003}]},{"osd":5,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94399999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.504}]},{"osd":6,"last update":"Mon Sep 16 14:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.625},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.72899999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.57999999999999996}]},{"osd":8,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71499999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.67900000000000005,"15min":0.66700000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0469999999999999,"5min":1.079,"15min":1.079},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75800000000000001,"15min":0.754},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.79200000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.306,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.81499999999999995}]}]},{"osd":2,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":1.6990000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":1.6739999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.36599999999999999,"15min":0.35399999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.34599999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.95699999999999996}]},{"osd":3,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.44900000000000001,"15min":0.432},"min":{"1min":0.30099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.498},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.6479999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.54200000000000004,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.83799999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.58499999999999996,"15min":0.56299999999999994},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.534},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.47799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.41499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.876}]},{"osd":8,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69699999999999995,"15min":0.68600000000000005},"min":{"1min":0.53100000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.5660000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.66700000000000004},"min":{"1min":0.51700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.79600000000000004}]},{"osd":9,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.5940000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.454},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.93300000000000005}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.74299999999999999,"15min":0.72999999999999998},"min":{"1min":0.56799999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.6220000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.77200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:50:28.314 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:50:28.599 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:50:28.600 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:50:28.612 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-16T14:50:28.054349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":183,"num_write_kb":2940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":599724,"kb_used_data":19716,"kb_used_omap":19,"kb_used_meta":579820,"kb_avail":1131862356,"statfs":{"total":1159641169920,"available":1159027052544,"internally_reserved":0,"allocated":20189184,"data_stored":12998546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19502,"internal_metadata":593736658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.797108"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":87,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.398990+0000","last_change":"2024-09-16T14:50:07.667627+0000","last_active":"2024-09-16T14:50:24.398990+0000","last_peered":"2024-09-16T14:50:24.398990+0000","last_clean":"2024-09-16T14:50:24.398990+0000","last_became_active":"2024-09-16T14:49:12.961697+0000","last_became_peered":"2024-09-16T14:49:12.961697+0000","last_unstale":"2024-09-16T14:50:24.398990+0000","last_undegraded":"2024-09-16T14:50:24.398990+0000","last_fullsized":"2024-09-16T14:50:24.398990+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:50:07.667489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:50:07.667489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:24.377449+0000","scrub_duration":0.002219033,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.332431+0000","last_change":"2024-09-16T14:49:54.393780+0000","last_active":"2024-09-16T14:50:24.332431+0000","last_peered":"2024-09-16T14:50:24.332431+0000","last_clean":"2024-09-16T14:50:24.332431+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:50:24.332431+0000","last_undegraded":"2024-09-16T14:50:24.332431+0000","last_fullsized":"2024-09-16T14:50:24.332431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:54.393679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:54.393679+0000","last_clean_scrub_stamp":"2024-09-16T14:49:54.393679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:09.244075+0000","scrub_duration":0.0015851929999999999,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":135,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260080+0000","last_change":"2024-09-16T14:49:43.127050+0000","last_active":"2024-09-16T14:50:23.260080+0000","last_peered":"2024-09-16T14:50:23.260080+0000","last_clean":"2024-09-16T14:50:23.260080+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:50:23.260080+0000","last_undegraded":"2024-09-16T14:50:23.260080+0000","last_fullsized":"2024-09-16T14:50:23.260080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_clean_scrub_stamp":"2024-09-16T14:49:43.126944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:52.930143+0000","scrub_duration":0.0012658890000000001,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.332481+0000","last_change":"2024-09-16T14:49:55.441559+0000","last_active":"2024-09-16T14:50:24.332481+0000","last_peered":"2024-09-16T14:50:24.332481+0000","last_clean":"2024-09-16T14:50:24.332481+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:50:24.332481+0000","last_undegraded":"2024-09-16T14:50:24.332481+0000","last_fullsized":"2024-09-16T14:50:24.332481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:55.441436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:55.441436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:00.621776+0000","scrub_duration":0.0011147150000000001,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":138,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260124+0000","last_change":"2024-09-16T14:49:28.351058+0000","last_active":"2024-09-16T14:50:23.260124+0000","last_peered":"2024-09-16T14:50:23.260124+0000","last_clean":"2024-09-16T14:50:23.260124+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:50:23.260124+0000","last_undegraded":"2024-09-16T14:50:23.260124+0000","last_fullsized":"2024-09-16T14:50:23.260124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-16T14:49:28.350938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:28.350938+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:42.292353+0000","scrub_duration":0.0018950309999999999,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.245674+0000","last_change":"2024-09-16T14:49:27.155463+0000","last_active":"2024-09-16T14:50:24.245674+0000","last_peered":"2024-09-16T14:50:24.245674+0000","last_clean":"2024-09-16T14:50:24.245674+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:50:24.245674+0000","last_undegraded":"2024-09-16T14:50:24.245674+0000","last_fullsized":"2024-09-16T14:50:24.245674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:27.155307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:27.155307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:32.986356+0000","scrub_duration":0.001601157,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":137,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.833368+0000","last_change":"2024-09-16T14:49:53.408437+0000","last_active":"2024-09-16T14:50:24.833368+0000","last_peered":"2024-09-16T14:50:24.833368+0000","last_clean":"2024-09-16T14:50:24.833368+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:50:24.833368+0000","last_undegraded":"2024-09-16T14:50:24.833368+0000","last_fullsized":"2024-09-16T14:50:24.833368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:53.408348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:53.408348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:16.057283+0000","scrub_duration":0.0016182830000000001,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":137,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260124+0000","last_change":"2024-09-16T14:49:40.527258+0000","last_active":"2024-09-16T14:50:23.260124+0000","last_peered":"2024-09-16T14:50:23.260124+0000","last_clean":"2024-09-16T14:50:23.260124+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:50:23.260124+0000","last_undegraded":"2024-09-16T14:50:23.260124+0000","last_fullsized":"2024-09-16T14:50:23.260124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2024-09-16T14:49:40.527164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.527164+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:46.029615+0000","scrub_duration":0.0012857120000000001,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":306,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.833372+0000","last_change":"2024-09-16T14:49:59.398994+0000","last_active":"2024-09-16T14:50:24.833372+0000","last_peered":"2024-09-16T14:50:24.833372+0000","last_clean":"2024-09-16T14:50:24.833372+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:50:24.833372+0000","last_undegraded":"2024-09-16T14:50:24.833372+0000","last_fullsized":"2024-09-16T14:50:24.833372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'137","last_scrub_stamp":"2024-09-16T14:49:59.398890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:49:59.398890+0000","objects_scrubbed":2,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:10.229835+0000","scrub_duration":0.001723186,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411356,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55336,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316504,"statfs":{"total":96636764160,"available":96580100096,"internally_reserved":0,"allocated":2125824,"data_stored":1524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.80900000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.83699999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79500000000000004,"5min":1.044,"15min":1.044},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.75800000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94199999999999995,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81999999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.81999999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.85199999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.66100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.72399999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.622},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68500000000000005,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.86699999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.377,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.629,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.56799999999999995}]},{"osd":9,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.77900000000000003}]},{"osd":10,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.5,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.67200000000000004}]}]},{"osd":10,"up_from":18,"seq":77309411358,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1589248,"data_stored":994178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.73199999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.73899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.753},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.872}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.79200000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71299999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72299999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.66400000000000003}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71899999999999997}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.64600000000000002}]}]},{"osd":6,"up_from":41,"seq":176093659159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":30974,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1965,"internal_metadata":31717459},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.66500000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67500000000000004}]},{"osd":3,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.54600000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48999999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.314},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.39600000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65300000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.624}]},{"osd":10,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.50600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":1,"up_from":18,"seq":77309411358,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72872,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298968,"statfs":{"total":96636764160,"available":96562143232,"internally_reserved":0,"allocated":1597440,"data_stored":995153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.54600000000000004,"15min":0.57199999999999995},"min":{"1min":0.26400000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70499999999999996,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.439,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.81999999999999995,"5min":1.806,"15min":1.806},"last":0.61899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.26000000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.872},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.58999999999999997,"15min":0.61299999999999999},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92700000000000005,"5min":4.0590000000000002,"15min":4.0590000000000002},"last":0.92700000000000005}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.435}]},{"osd":6,"last update":"Mon Sep 16 14:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46600000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91000000000000003,"5min":1.831,"15min":1.831},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.77500000000000002,"15min":0.79600000000000004},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.73199999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.55500000000000005,"5min":0.371,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.85799999999999998,"15min":0.875},"min":{"1min":0.54500000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.069,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.81000000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73599999999999999,"15min":0.73499999999999999},"min":{"1min":0.54100000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1299999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.63400000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.71999999999999997}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.89800000000000002,"15min":0.92300000000000004},"min":{"1min":0.61099999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1000000000000001,"5min":4.6589999999999998,"15min":4.6589999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.091,"5min":1.855,"15min":1.855},"last":0.749}]}]},{"osd":0,"up_from":18,"seq":77309411356,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55600000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.32200000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.39500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39900000000000002}]},{"osd":6,"last update":"Mon Sep 16 14:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.499},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53000000000000003}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66500000000000004,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.51100000000000001}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.55100000000000005}]},{"osd":10,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.60899999999999999}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.623},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56599999999999995}]}]},{"osd":5,"up_from":18,"seq":77309411357,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50724,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321116,"statfs":{"total":96636764160,"available":96584822784,"internally_reserved":0,"allocated":1597440,"data_stored":995153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.433}]},{"osd":1,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.505},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.57899999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68400000000000005,"5min":0.877,"15min":0.877},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.66000000000000003,"5min":0.746,"15min":0.746},"last":0.45500000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.621}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.31,"5min":0.252,"15min":0.252},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58799999999999997}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.753}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.72799999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91100000000000003,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.877}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85699999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.204,"5min":1.502,"15min":1.502},"last":0.91800000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78300000000000003,"15min":0.77600000000000002},"min":{"1min":0.53500000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2270000000000001,"5min":1.456,"15min":1.456},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.746,"15min":0.752},"min":{"1min":0.501,"5min":0.499,"15min":0.499},"max":{"1min":0.96699999999999997,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.83099999999999996}]}]},{"osd":9,"up_from":17,"seq":73014444061,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.436},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54200000000000004}]},{"osd":1,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.626,"15min":0.61299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.35699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":1.601,"15min":1.601},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52500000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":3.0019999999999998,"5min":3.0019999999999998,"15min":3.0019999999999998},"last":0.69699999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.65300000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98599999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.40400000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.93000000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85399999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.39400000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0620000000000001,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.81000000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68500000000000005,"15min":0.69699999999999995},"min":{"1min":0.38900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.876,"5min":1.47,"15min":1.47},"last":0.72999999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81299999999999994,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.35599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95799999999999996,"5min":1.014,"15min":1.014},"last":0.71799999999999997}]},{"osd":10,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84199999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.503},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.623}]},{"osd":11,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.99299999999999999,"5min":1.518,"15min":1.518},"last":0.74299999999999999}]}]},{"osd":8,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.38100000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.47099999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.34200000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.54200000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85299999999999998,"5min":1.006,"15min":1.006},"last":0.85299999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.78700000000000003}]},{"osd":6,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.88300000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0680000000000001,"5min":1.202,"15min":1.202},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.89300000000000002,"5min":1.097,"15min":1.097},"last":0.91200000000000003}]},{"osd":9,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82999999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79700000000000004,"5min":1.29,"15min":1.29},"last":0.71699999999999997}]},{"osd":10,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83199999999999996,"5min":1.22,"15min":1.22},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.48799999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94399999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.89200000000000002}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.42999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.496,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":1.254,"15min":1.254},"last":0.63900000000000001}]}]},{"osd":7,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51240,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320600,"statfs":{"total":96636764160,"available":96584294400,"internally_reserved":0,"allocated":2125824,"data_stored":1524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":50330060},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5620000000000003,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77100000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68300000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56299999999999994}]},{"osd":3,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83799999999999997,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.33300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.871,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.67300000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.76500000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:49:52 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.73799999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.44,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67300000000000004,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96499999999999997,"5min":1.163,"15min":1.163},"last":0.68600000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.93200000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.90800000000000003,"5min":1.026,"15min":1.026},"last":0.75600000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90700000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.70399999999999996},"min":{"1min":0.48899999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98399999999999999,"5min":1.079,"15min":1.079},"last":0.80000000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.52600000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0009999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0129999999999999,"5min":1.04,"15min":1.04},"last":0.77100000000000002}]}]},{"osd":4,"up_from":17,"seq":73014444060,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.68500000000000005}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.82699999999999996}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.501}]},{"osd":6,"last update":"Mon Sep 16 14:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.42699999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.373}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.57699999999999996}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64500000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.749}]},{"osd":11,"last update":"Mon Sep 16 14:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.72199999999999998}]}]},{"osd":3,"up_from":17,"seq":73014444062,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50724,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321116,"statfs":{"total":96636764160,"available":96584822784,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.28699999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.29899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.56000000000000005}]},{"osd":4,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.65900000000000003}]},{"osd":5,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94399999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.504}]},{"osd":6,"last update":"Mon Sep 16 14:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.625},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.72899999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.57999999999999996}]},{"osd":8,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71499999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.67900000000000005,"15min":0.66700000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0469999999999999,"5min":1.079,"15min":1.079},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75800000000000001,"15min":0.754},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.79200000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.306,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.81499999999999995}]}]},{"osd":2,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":1.6990000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":1.6739999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.36599999999999999,"15min":0.35399999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.34599999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.95699999999999996}]},{"osd":3,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.44900000000000001,"15min":0.432},"min":{"1min":0.30099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.498},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.6479999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.54200000000000004,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.83799999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.58499999999999996,"15min":0.56299999999999994},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.534},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.47799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.41499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.876}]},{"osd":8,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69699999999999995,"15min":0.68600000000000005},"min":{"1min":0.53100000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.5660000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.66700000000000004},"min":{"1min":0.51700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.79600000000000004}]},{"osd":9,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.5940000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.454},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.93300000000000005}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.74299999999999999,"15min":0.72999999999999998},"min":{"1min":0.56799999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.6220000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.77200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:50:28.615 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T14:50:28.615 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-16T14:50:28.916 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T14:50:29.639 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T14:50:29.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:50:29.937 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:50:29.937 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:50:29.949 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-16T14:50:28.054349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":183,"num_write_kb":2940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":599724,"kb_used_data":19716,"kb_used_omap":19,"kb_used_meta":579820,"kb_avail":1131862356,"statfs":{"total":1159641169920,"available":1159027052544,"internally_reserved":0,"allocated":20189184,"data_stored":12998546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19502,"internal_metadata":593736658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.797108"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":87,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.398990+0000","last_change":"2024-09-16T14:50:07.667627+0000","last_active":"2024-09-16T14:50:24.398990+0000","last_peered":"2024-09-16T14:50:24.398990+0000","last_clean":"2024-09-16T14:50:24.398990+0000","last_became_active":"2024-09-16T14:49:12.961697+0000","last_became_peered":"2024-09-16T14:49:12.961697+0000","last_unstale":"2024-09-16T14:50:24.398990+0000","last_undegraded":"2024-09-16T14:50:24.398990+0000","last_fullsized":"2024-09-16T14:50:24.398990+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:50:07.667489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:50:07.667489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:24.377449+0000","scrub_duration":0.002219033,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.332431+0000","last_change":"2024-09-16T14:49:54.393780+0000","last_active":"2024-09-16T14:50:24.332431+0000","last_peered":"2024-09-16T14:50:24.332431+0000","last_clean":"2024-09-16T14:50:24.332431+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:50:24.332431+0000","last_undegraded":"2024-09-16T14:50:24.332431+0000","last_fullsized":"2024-09-16T14:50:24.332431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:54.393679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:54.393679+0000","last_clean_scrub_stamp":"2024-09-16T14:49:54.393679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:09.244075+0000","scrub_duration":0.0015851929999999999,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":135,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260080+0000","last_change":"2024-09-16T14:49:43.127050+0000","last_active":"2024-09-16T14:50:23.260080+0000","last_peered":"2024-09-16T14:50:23.260080+0000","last_clean":"2024-09-16T14:50:23.260080+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:50:23.260080+0000","last_undegraded":"2024-09-16T14:50:23.260080+0000","last_fullsized":"2024-09-16T14:50:23.260080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_clean_scrub_stamp":"2024-09-16T14:49:43.126944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:52.930143+0000","scrub_duration":0.0012658890000000001,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.332481+0000","last_change":"2024-09-16T14:49:55.441559+0000","last_active":"2024-09-16T14:50:24.332481+0000","last_peered":"2024-09-16T14:50:24.332481+0000","last_clean":"2024-09-16T14:50:24.332481+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:50:24.332481+0000","last_undegraded":"2024-09-16T14:50:24.332481+0000","last_fullsized":"2024-09-16T14:50:24.332481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:55.441436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:55.441436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:00.621776+0000","scrub_duration":0.0011147150000000001,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":138,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260124+0000","last_change":"2024-09-16T14:49:28.351058+0000","last_active":"2024-09-16T14:50:23.260124+0000","last_peered":"2024-09-16T14:50:23.260124+0000","last_clean":"2024-09-16T14:50:23.260124+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:50:23.260124+0000","last_undegraded":"2024-09-16T14:50:23.260124+0000","last_fullsized":"2024-09-16T14:50:23.260124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-16T14:49:28.350938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:28.350938+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:42.292353+0000","scrub_duration":0.0018950309999999999,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.245674+0000","last_change":"2024-09-16T14:49:27.155463+0000","last_active":"2024-09-16T14:50:24.245674+0000","last_peered":"2024-09-16T14:50:24.245674+0000","last_clean":"2024-09-16T14:50:24.245674+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:50:24.245674+0000","last_undegraded":"2024-09-16T14:50:24.245674+0000","last_fullsized":"2024-09-16T14:50:24.245674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:27.155307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:27.155307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:32.986356+0000","scrub_duration":0.001601157,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":137,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.833368+0000","last_change":"2024-09-16T14:49:53.408437+0000","last_active":"2024-09-16T14:50:24.833368+0000","last_peered":"2024-09-16T14:50:24.833368+0000","last_clean":"2024-09-16T14:50:24.833368+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:50:24.833368+0000","last_undegraded":"2024-09-16T14:50:24.833368+0000","last_fullsized":"2024-09-16T14:50:24.833368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:53.408348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:53.408348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:16.057283+0000","scrub_duration":0.0016182830000000001,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":137,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260124+0000","last_change":"2024-09-16T14:49:40.527258+0000","last_active":"2024-09-16T14:50:23.260124+0000","last_peered":"2024-09-16T14:50:23.260124+0000","last_clean":"2024-09-16T14:50:23.260124+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:50:23.260124+0000","last_undegraded":"2024-09-16T14:50:23.260124+0000","last_fullsized":"2024-09-16T14:50:23.260124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2024-09-16T14:49:40.527164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.527164+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:46.029615+0000","scrub_duration":0.0012857120000000001,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":306,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.833372+0000","last_change":"2024-09-16T14:49:59.398994+0000","last_active":"2024-09-16T14:50:24.833372+0000","last_peered":"2024-09-16T14:50:24.833372+0000","last_clean":"2024-09-16T14:50:24.833372+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:50:24.833372+0000","last_undegraded":"2024-09-16T14:50:24.833372+0000","last_fullsized":"2024-09-16T14:50:24.833372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'137","last_scrub_stamp":"2024-09-16T14:49:59.398890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:49:59.398890+0000","objects_scrubbed":2,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:10.229835+0000","scrub_duration":0.001723186,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411356,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55336,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316504,"statfs":{"total":96636764160,"available":96580100096,"internally_reserved":0,"allocated":2125824,"data_stored":1524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.80900000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.83699999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79500000000000004,"5min":1.044,"15min":1.044},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.75800000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94199999999999995,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81999999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.81999999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.85199999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.66100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.72399999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.622},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68500000000000005,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.86699999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.377,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.629,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.56799999999999995}]},{"osd":9,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.77900000000000003}]},{"osd":10,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.5,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.67200000000000004}]}]},{"osd":10,"up_from":18,"seq":77309411358,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1589248,"data_stored":994178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.73199999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.73899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.753},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.872}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.79200000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71299999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72299999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.66400000000000003}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71899999999999997}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.64600000000000002}]}]},{"osd":6,"up_from":41,"seq":176093659159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":30974,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1965,"internal_metadata":31717459},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.66500000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67500000000000004}]},{"osd":3,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.54600000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48999999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.314},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.39600000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65300000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.624}]},{"osd":10,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.50600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":1,"up_from":18,"seq":77309411358,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72872,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298968,"statfs":{"total":96636764160,"available":96562143232,"internally_reserved":0,"allocated":1597440,"data_stored":995153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.54600000000000004,"15min":0.57199999999999995},"min":{"1min":0.26400000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70499999999999996,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.439,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.81999999999999995,"5min":1.806,"15min":1.806},"last":0.61899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.26000000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.872},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.58999999999999997,"15min":0.61299999999999999},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92700000000000005,"5min":4.0590000000000002,"15min":4.0590000000000002},"last":0.92700000000000005}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.435}]},{"osd":6,"last update":"Mon Sep 16 14:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46600000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91000000000000003,"5min":1.831,"15min":1.831},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.77500000000000002,"15min":0.79600000000000004},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.73199999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.55500000000000005,"5min":0.371,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.85799999999999998,"15min":0.875},"min":{"1min":0.54500000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.069,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.81000000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73599999999999999,"15min":0.73499999999999999},"min":{"1min":0.54100000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1299999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.63400000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.71999999999999997}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.89800000000000002,"15min":0.92300000000000004},"min":{"1min":0.61099999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1000000000000001,"5min":4.6589999999999998,"15min":4.6589999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.091,"5min":1.855,"15min":1.855},"last":0.749}]}]},{"osd":0,"up_from":18,"seq":77309411356,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55600000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.32200000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.39500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39900000000000002}]},{"osd":6,"last update":"Mon Sep 16 14:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.499},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53000000000000003}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66500000000000004,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.51100000000000001}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.55100000000000005}]},{"osd":10,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.60899999999999999}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.623},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56599999999999995}]}]},{"osd":5,"up_from":18,"seq":77309411357,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50724,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321116,"statfs":{"total":96636764160,"available":96584822784,"internally_reserved":0,"allocated":1597440,"data_stored":995153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.433}]},{"osd":1,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.505},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.57899999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68400000000000005,"5min":0.877,"15min":0.877},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.66000000000000003,"5min":0.746,"15min":0.746},"last":0.45500000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.621}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.31,"5min":0.252,"15min":0.252},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58799999999999997}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.753}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.72799999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91100000000000003,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.877}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85699999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.204,"5min":1.502,"15min":1.502},"last":0.91800000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78300000000000003,"15min":0.77600000000000002},"min":{"1min":0.53500000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2270000000000001,"5min":1.456,"15min":1.456},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.746,"15min":0.752},"min":{"1min":0.501,"5min":0.499,"15min":0.499},"max":{"1min":0.96699999999999997,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.83099999999999996}]}]},{"osd":9,"up_from":17,"seq":73014444061,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.436},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54200000000000004}]},{"osd":1,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.626,"15min":0.61299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.35699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":1.601,"15min":1.601},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52500000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":3.0019999999999998,"5min":3.0019999999999998,"15min":3.0019999999999998},"last":0.69699999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.65300000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98599999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.40400000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.93000000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85399999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.39400000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0620000000000001,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.81000000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68500000000000005,"15min":0.69699999999999995},"min":{"1min":0.38900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.876,"5min":1.47,"15min":1.47},"last":0.72999999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81299999999999994,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.35599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95799999999999996,"5min":1.014,"15min":1.014},"last":0.71799999999999997}]},{"osd":10,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84199999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.503},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.623}]},{"osd":11,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.99299999999999999,"5min":1.518,"15min":1.518},"last":0.74299999999999999}]}]},{"osd":8,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.38100000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.47099999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.34200000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.54200000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85299999999999998,"5min":1.006,"15min":1.006},"last":0.85299999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.78700000000000003}]},{"osd":6,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.88300000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0680000000000001,"5min":1.202,"15min":1.202},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.89300000000000002,"5min":1.097,"15min":1.097},"last":0.91200000000000003}]},{"osd":9,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82999999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79700000000000004,"5min":1.29,"15min":1.29},"last":0.71699999999999997}]},{"osd":10,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83199999999999996,"5min":1.22,"15min":1.22},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.48799999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94399999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.89200000000000002}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.42999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.496,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":1.254,"15min":1.254},"last":0.63900000000000001}]}]},{"osd":7,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51240,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320600,"statfs":{"total":96636764160,"available":96584294400,"internally_reserved":0,"allocated":2125824,"data_stored":1524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":50330060},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5620000000000003,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77100000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68300000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56299999999999994}]},{"osd":3,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83799999999999997,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.33300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.871,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.67300000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.76500000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:49:52 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.73799999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.44,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67300000000000004,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96499999999999997,"5min":1.163,"15min":1.163},"last":0.68600000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.93200000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.90800000000000003,"5min":1.026,"15min":1.026},"last":0.75600000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90700000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.70399999999999996},"min":{"1min":0.48899999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98399999999999999,"5min":1.079,"15min":1.079},"last":0.80000000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.52600000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0009999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0129999999999999,"5min":1.04,"15min":1.04},"last":0.77100000000000002}]}]},{"osd":4,"up_from":17,"seq":73014444060,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.68500000000000005}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.82699999999999996}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.501}]},{"osd":6,"last update":"Mon Sep 16 14:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.42699999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.373}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.57699999999999996}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64500000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.749}]},{"osd":11,"last update":"Mon Sep 16 14:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.72199999999999998}]}]},{"osd":3,"up_from":17,"seq":73014444062,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50724,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321116,"statfs":{"total":96636764160,"available":96584822784,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.28699999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.29899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.56000000000000005}]},{"osd":4,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.65900000000000003}]},{"osd":5,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94399999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.504}]},{"osd":6,"last update":"Mon Sep 16 14:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.625},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.72899999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.57999999999999996}]},{"osd":8,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71499999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.67900000000000005,"15min":0.66700000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0469999999999999,"5min":1.079,"15min":1.079},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75800000000000001,"15min":0.754},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.79200000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.306,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.81499999999999995}]}]},{"osd":2,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":1.6990000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":1.6739999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.36599999999999999,"15min":0.35399999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.34599999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.95699999999999996}]},{"osd":3,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.44900000000000001,"15min":0.432},"min":{"1min":0.30099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.498},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.6479999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.54200000000000004,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.83799999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.58499999999999996,"15min":0.56299999999999994},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.534},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.47799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.41499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.876}]},{"osd":8,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69699999999999995,"15min":0.68600000000000005},"min":{"1min":0.53100000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.5660000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.66700000000000004},"min":{"1min":0.51700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.79600000000000004}]},{"osd":9,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.5940000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.454},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.93300000000000005}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.74299999999999999,"15min":0.72999999999999998},"min":{"1min":0.56799999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.6220000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.77200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:50:29.952 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:50:30.238 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:50:30.238 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:50:30.249 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-16T14:50:30.054750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":183,"num_write_kb":2940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":599724,"kb_used_data":19716,"kb_used_omap":19,"kb_used_meta":579820,"kb_avail":1131862356,"statfs":{"total":1159641169920,"available":1159027052544,"internally_reserved":0,"allocated":20189184,"data_stored":12998546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19502,"internal_metadata":593736658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.795874"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":87,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.398990+0000","last_change":"2024-09-16T14:50:07.667627+0000","last_active":"2024-09-16T14:50:24.398990+0000","last_peered":"2024-09-16T14:50:24.398990+0000","last_clean":"2024-09-16T14:50:24.398990+0000","last_became_active":"2024-09-16T14:49:12.961697+0000","last_became_peered":"2024-09-16T14:49:12.961697+0000","last_unstale":"2024-09-16T14:50:24.398990+0000","last_undegraded":"2024-09-16T14:50:24.398990+0000","last_fullsized":"2024-09-16T14:50:24.398990+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:50:07.667489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:50:07.667489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:24.377449+0000","scrub_duration":0.002219033,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.332431+0000","last_change":"2024-09-16T14:49:54.393780+0000","last_active":"2024-09-16T14:50:24.332431+0000","last_peered":"2024-09-16T14:50:24.332431+0000","last_clean":"2024-09-16T14:50:24.332431+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:50:24.332431+0000","last_undegraded":"2024-09-16T14:50:24.332431+0000","last_fullsized":"2024-09-16T14:50:24.332431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:54.393679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:54.393679+0000","last_clean_scrub_stamp":"2024-09-16T14:49:54.393679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:09.244075+0000","scrub_duration":0.0015851929999999999,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":135,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260080+0000","last_change":"2024-09-16T14:49:43.127050+0000","last_active":"2024-09-16T14:50:23.260080+0000","last_peered":"2024-09-16T14:50:23.260080+0000","last_clean":"2024-09-16T14:50:23.260080+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:50:23.260080+0000","last_undegraded":"2024-09-16T14:50:23.260080+0000","last_fullsized":"2024-09-16T14:50:23.260080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_clean_scrub_stamp":"2024-09-16T14:49:43.126944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:52.930143+0000","scrub_duration":0.0012658890000000001,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.332481+0000","last_change":"2024-09-16T14:49:55.441559+0000","last_active":"2024-09-16T14:50:24.332481+0000","last_peered":"2024-09-16T14:50:24.332481+0000","last_clean":"2024-09-16T14:50:24.332481+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:50:24.332481+0000","last_undegraded":"2024-09-16T14:50:24.332481+0000","last_fullsized":"2024-09-16T14:50:24.332481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:55.441436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:55.441436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:00.621776+0000","scrub_duration":0.0011147150000000001,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":138,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260124+0000","last_change":"2024-09-16T14:49:28.351058+0000","last_active":"2024-09-16T14:50:23.260124+0000","last_peered":"2024-09-16T14:50:23.260124+0000","last_clean":"2024-09-16T14:50:23.260124+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:50:23.260124+0000","last_undegraded":"2024-09-16T14:50:23.260124+0000","last_fullsized":"2024-09-16T14:50:23.260124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-16T14:49:28.350938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:28.350938+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:42.292353+0000","scrub_duration":0.0018950309999999999,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.245674+0000","last_change":"2024-09-16T14:49:27.155463+0000","last_active":"2024-09-16T14:50:24.245674+0000","last_peered":"2024-09-16T14:50:24.245674+0000","last_clean":"2024-09-16T14:50:24.245674+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:50:24.245674+0000","last_undegraded":"2024-09-16T14:50:24.245674+0000","last_fullsized":"2024-09-16T14:50:24.245674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:27.155307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:27.155307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:32.986356+0000","scrub_duration":0.001601157,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":137,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.833368+0000","last_change":"2024-09-16T14:49:53.408437+0000","last_active":"2024-09-16T14:50:24.833368+0000","last_peered":"2024-09-16T14:50:24.833368+0000","last_clean":"2024-09-16T14:50:24.833368+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:50:24.833368+0000","last_undegraded":"2024-09-16T14:50:24.833368+0000","last_fullsized":"2024-09-16T14:50:24.833368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:53.408348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:53.408348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:16.057283+0000","scrub_duration":0.0016182830000000001,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":137,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260124+0000","last_change":"2024-09-16T14:49:40.527258+0000","last_active":"2024-09-16T14:50:23.260124+0000","last_peered":"2024-09-16T14:50:23.260124+0000","last_clean":"2024-09-16T14:50:23.260124+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:50:23.260124+0000","last_undegraded":"2024-09-16T14:50:23.260124+0000","last_fullsized":"2024-09-16T14:50:23.260124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2024-09-16T14:49:40.527164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.527164+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:46.029615+0000","scrub_duration":0.0012857120000000001,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":306,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.833372+0000","last_change":"2024-09-16T14:49:59.398994+0000","last_active":"2024-09-16T14:50:24.833372+0000","last_peered":"2024-09-16T14:50:24.833372+0000","last_clean":"2024-09-16T14:50:24.833372+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:50:24.833372+0000","last_undegraded":"2024-09-16T14:50:24.833372+0000","last_fullsized":"2024-09-16T14:50:24.833372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'137","last_scrub_stamp":"2024-09-16T14:49:59.398890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:49:59.398890+0000","objects_scrubbed":2,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:10.229835+0000","scrub_duration":0.001723186,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411356,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55336,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316504,"statfs":{"total":96636764160,"available":96580100096,"internally_reserved":0,"allocated":2125824,"data_stored":1524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.80900000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.83699999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79500000000000004,"5min":1.044,"15min":1.044},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.75800000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94199999999999995,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81999999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.81999999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.85199999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.66100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.72399999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.622},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68500000000000005,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.86699999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.377,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.629,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.56799999999999995}]},{"osd":9,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.77900000000000003}]},{"osd":10,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.5,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.67200000000000004}]}]},{"osd":10,"up_from":18,"seq":77309411358,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1589248,"data_stored":994178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.73199999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.73899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.753},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.872}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.79200000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71299999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72299999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.66400000000000003}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71899999999999997}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.64600000000000002}]}]},{"osd":6,"up_from":41,"seq":176093659159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":30974,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1965,"internal_metadata":31717459},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.66500000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67500000000000004}]},{"osd":3,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.54600000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48999999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.314},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.39600000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65300000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.624}]},{"osd":10,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.50600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":1,"up_from":18,"seq":77309411358,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72872,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298968,"statfs":{"total":96636764160,"available":96562143232,"internally_reserved":0,"allocated":1597440,"data_stored":995153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.54600000000000004,"15min":0.57199999999999995},"min":{"1min":0.26400000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70499999999999996,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.439,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.81999999999999995,"5min":1.806,"15min":1.806},"last":0.61899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.26000000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.872},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.58999999999999997,"15min":0.61299999999999999},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92700000000000005,"5min":4.0590000000000002,"15min":4.0590000000000002},"last":0.92700000000000005}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.435}]},{"osd":6,"last update":"Mon Sep 16 14:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46600000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91000000000000003,"5min":1.831,"15min":1.831},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.77500000000000002,"15min":0.79600000000000004},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.73199999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.55500000000000005,"5min":0.371,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.85799999999999998,"15min":0.875},"min":{"1min":0.54500000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.069,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.81000000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73599999999999999,"15min":0.73499999999999999},"min":{"1min":0.54100000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1299999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.63400000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.71999999999999997}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.89800000000000002,"15min":0.92300000000000004},"min":{"1min":0.61099999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1000000000000001,"5min":4.6589999999999998,"15min":4.6589999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.091,"5min":1.855,"15min":1.855},"last":0.749}]}]},{"osd":0,"up_from":18,"seq":77309411356,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55600000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.32200000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.39500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39900000000000002}]},{"osd":6,"last update":"Mon Sep 16 14:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.499},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53000000000000003}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66500000000000004,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.51100000000000001}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.55100000000000005}]},{"osd":10,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.60899999999999999}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.623},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56599999999999995}]}]},{"osd":5,"up_from":18,"seq":77309411357,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50724,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321116,"statfs":{"total":96636764160,"available":96584822784,"internally_reserved":0,"allocated":1597440,"data_stored":995153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.433}]},{"osd":1,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.505},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.57899999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68400000000000005,"5min":0.877,"15min":0.877},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.66000000000000003,"5min":0.746,"15min":0.746},"last":0.45500000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.621}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.31,"5min":0.252,"15min":0.252},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58799999999999997}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.753}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.72799999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91100000000000003,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.877}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85699999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.204,"5min":1.502,"15min":1.502},"last":0.91800000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78300000000000003,"15min":0.77600000000000002},"min":{"1min":0.53500000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2270000000000001,"5min":1.456,"15min":1.456},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.746,"15min":0.752},"min":{"1min":0.501,"5min":0.499,"15min":0.499},"max":{"1min":0.96699999999999997,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.83099999999999996}]}]},{"osd":9,"up_from":17,"seq":73014444061,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.436},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54200000000000004}]},{"osd":1,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.626,"15min":0.61299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.35699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":1.601,"15min":1.601},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52500000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":3.0019999999999998,"5min":3.0019999999999998,"15min":3.0019999999999998},"last":0.69699999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.65300000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98599999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.40400000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.93000000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85399999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.39400000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0620000000000001,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.81000000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68500000000000005,"15min":0.69699999999999995},"min":{"1min":0.38900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.876,"5min":1.47,"15min":1.47},"last":0.72999999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81299999999999994,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.35599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95799999999999996,"5min":1.014,"15min":1.014},"last":0.71799999999999997}]},{"osd":10,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84199999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.503},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.623}]},{"osd":11,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.99299999999999999,"5min":1.518,"15min":1.518},"last":0.74299999999999999}]}]},{"osd":8,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.38100000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.47099999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.34200000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.54200000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85299999999999998,"5min":1.006,"15min":1.006},"last":0.85299999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.78700000000000003}]},{"osd":6,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.88300000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0680000000000001,"5min":1.202,"15min":1.202},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.89300000000000002,"5min":1.097,"15min":1.097},"last":0.91200000000000003}]},{"osd":9,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82999999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79700000000000004,"5min":1.29,"15min":1.29},"last":0.71699999999999997}]},{"osd":10,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83199999999999996,"5min":1.22,"15min":1.22},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.48799999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94399999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.89200000000000002}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.42999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.496,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":1.254,"15min":1.254},"last":0.63900000000000001}]}]},{"osd":7,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51240,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320600,"statfs":{"total":96636764160,"available":96584294400,"internally_reserved":0,"allocated":2125824,"data_stored":1524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":50330060},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5620000000000003,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77100000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68300000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56299999999999994}]},{"osd":3,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83799999999999997,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.33300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.871,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.67300000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.76500000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:49:52 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.73799999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.44,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67300000000000004,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96499999999999997,"5min":1.163,"15min":1.163},"last":0.68600000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.93200000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.90800000000000003,"5min":1.026,"15min":1.026},"last":0.75600000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90700000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.70399999999999996},"min":{"1min":0.48899999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98399999999999999,"5min":1.079,"15min":1.079},"last":0.80000000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.52600000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0009999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0129999999999999,"5min":1.04,"15min":1.04},"last":0.77100000000000002}]}]},{"osd":4,"up_from":17,"seq":73014444060,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.68500000000000005}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.82699999999999996}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.501}]},{"osd":6,"last update":"Mon Sep 16 14:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.42699999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.373}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.57699999999999996}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64500000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.749}]},{"osd":11,"last update":"Mon Sep 16 14:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.72199999999999998}]}]},{"osd":3,"up_from":17,"seq":73014444062,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50724,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321116,"statfs":{"total":96636764160,"available":96584822784,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.28699999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.29899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.56000000000000005}]},{"osd":4,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.65900000000000003}]},{"osd":5,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94399999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.504}]},{"osd":6,"last update":"Mon Sep 16 14:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.625},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.72899999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.57999999999999996}]},{"osd":8,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71499999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.67900000000000005,"15min":0.66700000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0469999999999999,"5min":1.079,"15min":1.079},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75800000000000001,"15min":0.754},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.79200000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.306,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.81499999999999995}]}]},{"osd":2,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":1.6990000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":1.6739999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.36599999999999999,"15min":0.35399999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.34599999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.95699999999999996}]},{"osd":3,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.44900000000000001,"15min":0.432},"min":{"1min":0.30099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.498},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.6479999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.54200000000000004,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.83799999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.58499999999999996,"15min":0.56299999999999994},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.534},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.47799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.41499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.876}]},{"osd":8,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69699999999999995,"15min":0.68600000000000005},"min":{"1min":0.53100000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.5660000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.66700000000000004},"min":{"1min":0.51700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.79600000000000004}]},{"osd":9,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.5940000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.454},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.93300000000000005}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.74299999999999999,"15min":0.72999999999999998},"min":{"1min":0.56799999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.6220000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.77200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:50:30.309 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T14:50:30.309 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:50:30.593 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:50:30.594 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:50:30.605 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-16T14:50:30.054750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":183,"num_write_kb":2940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":599724,"kb_used_data":19716,"kb_used_omap":19,"kb_used_meta":579820,"kb_avail":1131862356,"statfs":{"total":1159641169920,"available":1159027052544,"internally_reserved":0,"allocated":20189184,"data_stored":12998546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19502,"internal_metadata":593736658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.795874"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":87,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.398990+0000","last_change":"2024-09-16T14:50:07.667627+0000","last_active":"2024-09-16T14:50:24.398990+0000","last_peered":"2024-09-16T14:50:24.398990+0000","last_clean":"2024-09-16T14:50:24.398990+0000","last_became_active":"2024-09-16T14:49:12.961697+0000","last_became_peered":"2024-09-16T14:49:12.961697+0000","last_unstale":"2024-09-16T14:50:24.398990+0000","last_undegraded":"2024-09-16T14:50:24.398990+0000","last_fullsized":"2024-09-16T14:50:24.398990+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:50:07.667489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:50:07.667489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:24.377449+0000","scrub_duration":0.002219033,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.332431+0000","last_change":"2024-09-16T14:49:54.393780+0000","last_active":"2024-09-16T14:50:24.332431+0000","last_peered":"2024-09-16T14:50:24.332431+0000","last_clean":"2024-09-16T14:50:24.332431+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:50:24.332431+0000","last_undegraded":"2024-09-16T14:50:24.332431+0000","last_fullsized":"2024-09-16T14:50:24.332431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:54.393679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:54.393679+0000","last_clean_scrub_stamp":"2024-09-16T14:49:54.393679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:09.244075+0000","scrub_duration":0.0015851929999999999,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":135,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260080+0000","last_change":"2024-09-16T14:49:43.127050+0000","last_active":"2024-09-16T14:50:23.260080+0000","last_peered":"2024-09-16T14:50:23.260080+0000","last_clean":"2024-09-16T14:50:23.260080+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:50:23.260080+0000","last_undegraded":"2024-09-16T14:50:23.260080+0000","last_fullsized":"2024-09-16T14:50:23.260080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_clean_scrub_stamp":"2024-09-16T14:49:43.126944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:52.930143+0000","scrub_duration":0.0012658890000000001,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.332481+0000","last_change":"2024-09-16T14:49:55.441559+0000","last_active":"2024-09-16T14:50:24.332481+0000","last_peered":"2024-09-16T14:50:24.332481+0000","last_clean":"2024-09-16T14:50:24.332481+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:50:24.332481+0000","last_undegraded":"2024-09-16T14:50:24.332481+0000","last_fullsized":"2024-09-16T14:50:24.332481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:55.441436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:55.441436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:00.621776+0000","scrub_duration":0.0011147150000000001,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":138,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260124+0000","last_change":"2024-09-16T14:49:28.351058+0000","last_active":"2024-09-16T14:50:23.260124+0000","last_peered":"2024-09-16T14:50:23.260124+0000","last_clean":"2024-09-16T14:50:23.260124+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:50:23.260124+0000","last_undegraded":"2024-09-16T14:50:23.260124+0000","last_fullsized":"2024-09-16T14:50:23.260124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-16T14:49:28.350938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:28.350938+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:42.292353+0000","scrub_duration":0.0018950309999999999,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":133,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.245674+0000","last_change":"2024-09-16T14:49:27.155463+0000","last_active":"2024-09-16T14:50:24.245674+0000","last_peered":"2024-09-16T14:50:24.245674+0000","last_clean":"2024-09-16T14:50:24.245674+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:50:24.245674+0000","last_undegraded":"2024-09-16T14:50:24.245674+0000","last_fullsized":"2024-09-16T14:50:24.245674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:27.155307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:27.155307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:32.986356+0000","scrub_duration":0.001601157,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":137,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.833368+0000","last_change":"2024-09-16T14:49:53.408437+0000","last_active":"2024-09-16T14:50:24.833368+0000","last_peered":"2024-09-16T14:50:24.833368+0000","last_clean":"2024-09-16T14:50:24.833368+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:50:24.833368+0000","last_undegraded":"2024-09-16T14:50:24.833368+0000","last_fullsized":"2024-09-16T14:50:24.833368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:53.408348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:53.408348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:16.057283+0000","scrub_duration":0.0016182830000000001,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":137,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T14:50:23.260124+0000","last_change":"2024-09-16T14:49:40.527258+0000","last_active":"2024-09-16T14:50:23.260124+0000","last_peered":"2024-09-16T14:50:23.260124+0000","last_clean":"2024-09-16T14:50:23.260124+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:50:23.260124+0000","last_undegraded":"2024-09-16T14:50:23.260124+0000","last_fullsized":"2024-09-16T14:50:23.260124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2024-09-16T14:49:40.527164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.527164+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:46.029615+0000","scrub_duration":0.0012857120000000001,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":306,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T14:50:24.833372+0000","last_change":"2024-09-16T14:49:59.398994+0000","last_active":"2024-09-16T14:50:24.833372+0000","last_peered":"2024-09-16T14:50:24.833372+0000","last_clean":"2024-09-16T14:50:24.833372+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:50:24.833372+0000","last_undegraded":"2024-09-16T14:50:24.833372+0000","last_fullsized":"2024-09-16T14:50:24.833372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'137","last_scrub_stamp":"2024-09-16T14:49:59.398890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:49:59.398890+0000","objects_scrubbed":2,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:10.229835+0000","scrub_duration":0.001723186,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411356,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55336,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316504,"statfs":{"total":96636764160,"available":96580100096,"internally_reserved":0,"allocated":2125824,"data_stored":1524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.80900000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.83699999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79500000000000004,"5min":1.044,"15min":1.044},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.75800000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94199999999999995,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81999999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.81999999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.85199999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.66100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.72399999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.622},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68500000000000005,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.86699999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.377,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.629,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.56799999999999995}]},{"osd":9,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.77900000000000003}]},{"osd":10,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.5,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.67200000000000004}]}]},{"osd":10,"up_from":18,"seq":77309411358,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1589248,"data_stored":994178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.73199999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.73899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.753},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.872}]},{"osd":3,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.79200000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.71299999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72299999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.66400000000000003}]},{"osd":9,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71899999999999997}]},{"osd":11,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.64600000000000002}]}]},{"osd":6,"up_from":41,"seq":176093659159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":30974,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1965,"internal_metadata":31717459},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.66500000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67500000000000004}]},{"osd":3,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.54600000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.48999999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.314},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.39600000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65300000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.624}]},{"osd":10,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.50600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":1,"up_from":18,"seq":77309411358,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72872,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298968,"statfs":{"total":96636764160,"available":96562143232,"internally_reserved":0,"allocated":1597440,"data_stored":995153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.54600000000000004,"15min":0.57199999999999995},"min":{"1min":0.26400000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70499999999999996,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.439,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.81999999999999995,"5min":1.806,"15min":1.806},"last":0.61899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.26000000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.872},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.58999999999999997,"15min":0.61299999999999999},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92700000000000005,"5min":4.0590000000000002,"15min":4.0590000000000002},"last":0.92700000000000005}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.435}]},{"osd":6,"last update":"Mon Sep 16 14:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46600000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91000000000000003,"5min":1.831,"15min":1.831},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.77500000000000002,"15min":0.79600000000000004},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.73199999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.55500000000000005,"5min":0.371,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.85799999999999998,"15min":0.875},"min":{"1min":0.54500000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.069,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.81000000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73599999999999999,"15min":0.73499999999999999},"min":{"1min":0.54100000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1299999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.63400000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.71999999999999997}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.89800000000000002,"15min":0.92300000000000004},"min":{"1min":0.61099999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1000000000000001,"5min":4.6589999999999998,"15min":4.6589999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.091,"5min":1.855,"15min":1.855},"last":0.749}]}]},{"osd":0,"up_from":18,"seq":77309411356,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55600000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.32200000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.39500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39900000000000002}]},{"osd":6,"last update":"Mon Sep 16 14:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.499},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53000000000000003}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66500000000000004,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.51100000000000001}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.55100000000000005}]},{"osd":10,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.60899999999999999}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.623},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56599999999999995}]}]},{"osd":5,"up_from":18,"seq":77309411357,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50724,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321116,"statfs":{"total":96636764160,"available":96584822784,"internally_reserved":0,"allocated":1597440,"data_stored":995153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.433}]},{"osd":1,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.505},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.57899999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68400000000000005,"5min":0.877,"15min":0.877},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.66000000000000003,"5min":0.746,"15min":0.746},"last":0.45500000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.621}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.31,"5min":0.252,"15min":0.252},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58799999999999997}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.753}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.72799999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91100000000000003,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.877}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85699999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.204,"5min":1.502,"15min":1.502},"last":0.91800000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78300000000000003,"15min":0.77600000000000002},"min":{"1min":0.53500000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2270000000000001,"5min":1.456,"15min":1.456},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.746,"15min":0.752},"min":{"1min":0.501,"5min":0.499,"15min":0.499},"max":{"1min":0.96699999999999997,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.83099999999999996}]}]},{"osd":9,"up_from":17,"seq":73014444061,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.436},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54200000000000004}]},{"osd":1,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.626,"15min":0.61299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.67500000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.35699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":1.601,"15min":1.601},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52500000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":3.0019999999999998,"5min":3.0019999999999998,"15min":3.0019999999999998},"last":0.69699999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.65300000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98599999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.40400000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.93000000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85399999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.39400000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0620000000000001,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.81000000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68500000000000005,"15min":0.69699999999999995},"min":{"1min":0.38900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.876,"5min":1.47,"15min":1.47},"last":0.72999999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81299999999999994,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.35599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95799999999999996,"5min":1.014,"15min":1.014},"last":0.71799999999999997}]},{"osd":10,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84199999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.503},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.623}]},{"osd":11,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.99299999999999999,"5min":1.518,"15min":1.518},"last":0.74299999999999999}]}]},{"osd":8,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.38100000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.47099999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.34200000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.54200000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85299999999999998,"5min":1.006,"15min":1.006},"last":0.85299999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.78700000000000003}]},{"osd":6,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.88300000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0680000000000001,"5min":1.202,"15min":1.202},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.89300000000000002,"5min":1.097,"15min":1.097},"last":0.91200000000000003}]},{"osd":9,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82999999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79700000000000004,"5min":1.29,"15min":1.29},"last":0.71699999999999997}]},{"osd":10,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83199999999999996,"5min":1.22,"15min":1.22},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.48799999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94399999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.89200000000000002}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.42999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.496,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":1.254,"15min":1.254},"last":0.63900000000000001}]}]},{"osd":7,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51240,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320600,"statfs":{"total":96636764160,"available":96584294400,"internally_reserved":0,"allocated":2125824,"data_stored":1524062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":50330060},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5620000000000003,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77100000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68300000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56299999999999994}]},{"osd":3,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83799999999999997,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.33300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.871,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.67300000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.76500000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:49:52 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.73799999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.44,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67300000000000004,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96499999999999997,"5min":1.163,"15min":1.163},"last":0.68600000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.93200000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.90800000000000003,"5min":1.026,"15min":1.026},"last":0.75600000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90700000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.70399999999999996},"min":{"1min":0.48899999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98399999999999999,"5min":1.079,"15min":1.079},"last":0.80000000000000004}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.52600000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0009999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0129999999999999,"5min":1.04,"15min":1.04},"last":0.77100000000000002}]}]},{"osd":4,"up_from":17,"seq":73014444060,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50720,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321120,"statfs":{"total":96636764160,"available":96584826880,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.68500000000000005}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.82699999999999996}]},{"osd":5,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.501}]},{"osd":6,"last update":"Mon Sep 16 14:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.42699999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.373}]},{"osd":8,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.57699999999999996}]},{"osd":9,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64500000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.749}]},{"osd":11,"last update":"Mon Sep 16 14:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.72199999999999998}]}]},{"osd":3,"up_from":17,"seq":73014444062,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50724,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321116,"statfs":{"total":96636764160,"available":96584822784,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.28699999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.29899999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":4.3440000000000003,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.56000000000000005}]},{"osd":4,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.65900000000000003}]},{"osd":5,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94399999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.504}]},{"osd":6,"last update":"Mon Sep 16 14:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.625},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.72899999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.57999999999999996}]},{"osd":8,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71499999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.67900000000000005,"15min":0.66700000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0469999999999999,"5min":1.079,"15min":1.079},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75800000000000001,"15min":0.754},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.79200000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.306,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.81499999999999995}]}]},{"osd":2,"up_from":17,"seq":73014444061,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1593344,"data_stored":995134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":1.6990000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":1.6739999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.36599999999999999,"15min":0.35399999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.34599999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.95699999999999996}]},{"osd":3,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.44900000000000001,"15min":0.432},"min":{"1min":0.30099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.498},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.6479999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.54200000000000004,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.83799999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.58499999999999996,"15min":0.56299999999999994},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.534},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.47799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.41499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.876}]},{"osd":8,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69699999999999995,"15min":0.68600000000000005},"min":{"1min":0.53100000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.5660000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.66700000000000004},"min":{"1min":0.51700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.79600000000000004}]},{"osd":9,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.5940000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.454},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.93300000000000005}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.74299999999999999,"15min":0.72999999999999998},"min":{"1min":0.56799999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.6220000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.77200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:50:30.609 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T14:50:30.892 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:50:30.893 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":135,"fsid":"aaaa1c0f-de8b-48c8-9412-815c8b70b7b8","created":"2024-09-16T14:48:10.063465+0000","modified":"2024-09-16T14:50:27.519473+0000","last_up_change":"2024-09-16T14:48:47.933230+0000","last_in_change":"2024-09-16T14:49:11.939833+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T14:48:20.279244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":11.760000228881836,"score_stable":11.760000228881836,"optimal_score":0.17000000178813934,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T14:48:22.580854+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":23,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":0.75,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ffcfdeb7-eb75-4b3a-bc2a-f1834271654c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":725262},{"type":"v1","addr":"172.21.15.40:6801","nonce":725262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":725262},{"type":"v1","addr":"172.21.15.40:6803","nonce":725262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":725262},{"type":"v1","addr":"172.21.15.40:6807","nonce":725262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":725262},{"type":"v1","addr":"172.21.15.40:6805","nonce":725262}]},"public_addr":"172.21.15.40:6801/725262","cluster_addr":"172.21.15.40:6803/725262","heartbeat_back_addr":"172.21.15.40:6807/725262","heartbeat_front_addr":"172.21.15.40:6805/725262","state":["exists","up"]},{"osd":1,"uuid":"6fb8c93f-5e3c-492f-bd0c-6d97764467f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6819","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6821","nonce":501335559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6823","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6825","nonce":501335559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6831","nonce":501335559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6827","nonce":501335559},{"type":"v1","addr":"172.21.15.40:6829","nonce":501335559}]},"public_addr":"172.21.15.40:6821/501335559","cluster_addr":"172.21.15.40:6825/501335559","heartbeat_back_addr":"172.21.15.40:6831/501335559","heartbeat_front_addr":"172.21.15.40:6829/501335559","state":["exists","up"]},{"osd":2,"uuid":"500457a9-0d59-451c-b2ff-28aa363056bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6809","nonce":1490162929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6811","nonce":1490162929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6815","nonce":1490162929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1490162929},{"type":"v1","addr":"172.21.15.40:6813","nonce":1490162929}]},"public_addr":"172.21.15.40:6809/1490162929","cluster_addr":"172.21.15.40:6811/1490162929","heartbeat_back_addr":"172.21.15.40:6815/1490162929","heartbeat_front_addr":"172.21.15.40:6813/1490162929","state":["exists","up"]},{"osd":3,"uuid":"f6e48d98-5d87-4cd5-8bd8-6e4c9dc56a39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6817","nonce":682463486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6820","nonce":682463486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6828","nonce":682463486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":682463486},{"type":"v1","addr":"172.21.15.40:6824","nonce":682463486}]},"public_addr":"172.21.15.40:6817/682463486","cluster_addr":"172.21.15.40:6820/682463486","heartbeat_back_addr":"172.21.15.40:6828/682463486","heartbeat_front_addr":"172.21.15.40:6824/682463486","state":["exists","up"]},{"osd":4,"uuid":"474dd40c-2b73-44ea-b6a9-48605b78125c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6818","nonce":1550824417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6819","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6821","nonce":1550824417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6828","nonce":1550824417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6823","nonce":1550824417},{"type":"v1","addr":"172.21.15.111:6824","nonce":1550824417}]},"public_addr":"172.21.15.111:6818/1550824417","cluster_addr":"172.21.15.111:6821/1550824417","heartbeat_back_addr":"172.21.15.111:6828/1550824417","heartbeat_front_addr":"172.21.15.111:6824/1550824417","state":["exists","up"]},{"osd":5,"uuid":"479bfe9b-00f6-4763-9740-9c9fd11e2587","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6801","nonce":3785556374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6803","nonce":3785556374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6807","nonce":3785556374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":3785556374},{"type":"v1","addr":"172.21.15.111:6805","nonce":3785556374}]},"public_addr":"172.21.15.111:6801/3785556374","cluster_addr":"172.21.15.111:6803/3785556374","heartbeat_back_addr":"172.21.15.111:6807/3785556374","heartbeat_front_addr":"172.21.15.111:6805/3785556374","state":["exists","up"]},{"osd":6,"uuid":"6ac36041-045e-4813-ad35-ca4e224811fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":23,"up_from":41,"up_thru":92,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6817","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6820","nonce":722937064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6825","nonce":722937064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6831","nonce":722937064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6827","nonce":722937064},{"type":"v1","addr":"172.21.15.111:6829","nonce":722937064}]},"public_addr":"172.21.15.111:6820/722937064","cluster_addr":"172.21.15.111:6825/722937064","heartbeat_back_addr":"172.21.15.111:6831/722937064","heartbeat_front_addr":"172.21.15.111:6829/722937064","state":["exists","up"]},{"osd":7,"uuid":"129006cf-b82b-4e11-a007-63deb355d007","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6809","nonce":3591930331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6811","nonce":3591930331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6815","nonce":3591930331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3591930331},{"type":"v1","addr":"172.21.15.111:6813","nonce":3591930331}]},"public_addr":"172.21.15.111:6809/3591930331","cluster_addr":"172.21.15.111:6811/3591930331","heartbeat_back_addr":"172.21.15.111:6815/3591930331","heartbeat_front_addr":"172.21.15.111:6813/3591930331","state":["exists","up"]},{"osd":8,"uuid":"27e746ee-f67a-4a73-8976-86d012469657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6817","nonce":2473986485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6819","nonce":2473986485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6824","nonce":2473986485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":2473986485},{"type":"v1","addr":"172.21.15.114:6821","nonce":2473986485}]},"public_addr":"172.21.15.114:6817/2473986485","cluster_addr":"172.21.15.114:6819/2473986485","heartbeat_back_addr":"172.21.15.114:6824/2473986485","heartbeat_front_addr":"172.21.15.114:6821/2473986485","state":["exists","up"]},{"osd":9,"uuid":"90975d4f-24b3-4c84-916e-3138bc52a076","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6801","nonce":2290892207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6803","nonce":2290892207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6807","nonce":2290892207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":2290892207},{"type":"v1","addr":"172.21.15.114:6805","nonce":2290892207}]},"public_addr":"172.21.15.114:6801/2290892207","cluster_addr":"172.21.15.114:6803/2290892207","heartbeat_back_addr":"172.21.15.114:6807/2290892207","heartbeat_front_addr":"172.21.15.114:6805/2290892207","state":["exists","up"]},{"osd":10,"uuid":"89c8157b-52ce-4c1c-9df7-e214df30a827","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6809","nonce":1795470140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6811","nonce":1795470140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6815","nonce":1795470140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":1795470140},{"type":"v1","addr":"172.21.15.114:6813","nonce":1795470140}]},"public_addr":"172.21.15.114:6809/1795470140","cluster_addr":"172.21.15.114:6811/1795470140","heartbeat_back_addr":"172.21.15.114:6815/1795470140","heartbeat_front_addr":"172.21.15.114:6813/1795470140","state":["exists","up"]},{"osd":11,"uuid":"0dfbd657-c9f0-42f9-9b43-323c45d89c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6823","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6825","nonce":1823608150}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6827","nonce":1823608150}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6831","nonce":1823608150}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":1823608150},{"type":"v1","addr":"172.21.15.114:6829","nonce":1823608150}]},"public_addr":"172.21.15.114:6825/1823608150","cluster_addr":"172.21.15.114:6827/1823608150","heartbeat_back_addr":"172.21.15.114:6831/1823608150","heartbeat_front_addr":"172.21.15.114:6829/1823608150","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:21.058050+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:29.713159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:46.059619+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:45.409424+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:19.879455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:39.305292+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:32.379251+0000","dead_epoch":25},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:50:27.463913+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:32.215102+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:19.285409+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:36.675606+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T14:49:32.320290+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"lrcprofile":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","k":"4","l":"3","m":"2","name":"lrcprofile","plugin":"lrc"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":70,"length":1},{"begin":76,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T14:50:31.905 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T14:50:31.905 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:50:32.046 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:50:32.046 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:50:32.046 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:50:32.057 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T14:50:32.342 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T14:50:32.355 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T14:50:32.355 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:50:32.479 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:50:32.480 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:50:32.480 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:50:32.490 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T14:50:32.774 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T14:50:32.786 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T14:50:32.787 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:50:32.911 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:50:32.911 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:50:32.912 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:50:32.922 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T14:50:33.205 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T14:50:33.218 INFO:tasks.ceph:Scrubbing osd.3 2024-09-16T14:50:33.218 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:50:33.344 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:50:33.344 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:50:33.344 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:50:33.354 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-16T14:50:33.640 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 3 to deep-scrub 2024-09-16T14:50:33.653 INFO:tasks.ceph:Scrubbing osd.4 2024-09-16T14:50:33.654 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:50:33.779 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:50:33.779 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:50:33.779 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:50:33.789 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-16T14:50:34.071 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 4 to deep-scrub 2024-09-16T14:50:34.084 INFO:tasks.ceph:Scrubbing osd.5 2024-09-16T14:50:34.084 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:50:34.210 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:50:34.210 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:50:34.211 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:50:34.221 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-16T14:50:34.510 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 5 to deep-scrub 2024-09-16T14:50:34.523 INFO:tasks.ceph:Scrubbing osd.6 2024-09-16T14:50:34.523 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:50:34.649 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:50:34.650 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:50:34.650 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:50:34.660 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-16T14:50:34.944 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 6 to deep-scrub 2024-09-16T14:50:34.957 INFO:tasks.ceph:Scrubbing osd.7 2024-09-16T14:50:34.957 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:50:35.085 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:50:35.085 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:50:35.085 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:50:35.095 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-16T14:50:35.383 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 7 to deep-scrub 2024-09-16T14:50:35.396 INFO:tasks.ceph:Scrubbing osd.8 2024-09-16T14:50:35.396 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:50:35.521 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:50:35.522 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:50:35.522 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:50:35.532 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2024-09-16T14:50:35.818 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 8 to deep-scrub 2024-09-16T14:50:35.830 INFO:tasks.ceph:Scrubbing osd.9 2024-09-16T14:50:35.831 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:50:35.956 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:50:35.956 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:50:35.956 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:50:35.966 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2024-09-16T14:50:36.250 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 9 to deep-scrub 2024-09-16T14:50:36.263 INFO:tasks.ceph:Scrubbing osd.10 2024-09-16T14:50:36.263 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:50:36.388 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:50:36.388 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:50:36.388 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:50:36.398 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2024-09-16T14:50:36.686 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 10 to deep-scrub 2024-09-16T14:50:36.699 INFO:tasks.ceph:Scrubbing osd.11 2024-09-16T14:50:36.699 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2024-09-16T14:50:36.825 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-16T14:50:36.825 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T14:50:36.825 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-16T14:50:36.836 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2024-09-16T14:50:37.121 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 11 to deep-scrub 2024-09-16T14:50:37.134 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:50:37.417 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:50:37.417 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:50:37.428 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-16T14:50:36.056034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":183,"num_write_kb":2940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":599864,"kb_used_data":19872,"kb_used_omap":19,"kb_used_meta":579820,"kb_avail":1131862216,"statfs":{"total":1159641169920,"available":1159026909184,"internally_reserved":0,"allocated":20348928,"data_stored":13098048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19502,"internal_metadata":593736658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001684"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":89,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T14:50:28.879829+0000","last_change":"2024-09-16T14:50:07.667627+0000","last_active":"2024-09-16T14:50:28.879829+0000","last_peered":"2024-09-16T14:50:28.879829+0000","last_clean":"2024-09-16T14:50:28.879829+0000","last_became_active":"2024-09-16T14:49:12.961697+0000","last_became_peered":"2024-09-16T14:49:12.961697+0000","last_unstale":"2024-09-16T14:50:28.879829+0000","last_undegraded":"2024-09-16T14:50:28.879829+0000","last_fullsized":"2024-09-16T14:50:28.879829+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:50:07.667489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:50:07.667489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:24.377449+0000","scrub_duration":0.002219033,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":135,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T14:50:27.975004+0000","last_change":"2024-09-16T14:49:54.393780+0000","last_active":"2024-09-16T14:50:27.975004+0000","last_peered":"2024-09-16T14:50:27.975004+0000","last_clean":"2024-09-16T14:50:27.975004+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:50:27.975004+0000","last_undegraded":"2024-09-16T14:50:27.975004+0000","last_fullsized":"2024-09-16T14:50:27.975004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:54.393679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:54.393679+0000","last_clean_scrub_stamp":"2024-09-16T14:49:54.393679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:09.244075+0000","scrub_duration":0.0015851929999999999,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":137,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T14:50:27.523407+0000","last_change":"2024-09-16T14:49:43.127050+0000","last_active":"2024-09-16T14:50:27.523407+0000","last_peered":"2024-09-16T14:50:27.523407+0000","last_clean":"2024-09-16T14:50:27.523407+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:50:27.523407+0000","last_undegraded":"2024-09-16T14:50:27.523407+0000","last_fullsized":"2024-09-16T14:50:27.523407+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:49:43.126944+0000","last_clean_scrub_stamp":"2024-09-16T14:49:43.126944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:52.930143+0000","scrub_duration":0.0012658890000000001,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":136,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-16T14:50:32.006502+0000","last_change":"2024-09-16T14:49:55.441559+0000","last_active":"2024-09-16T14:50:32.006502+0000","last_peered":"2024-09-16T14:50:32.006502+0000","last_clean":"2024-09-16T14:50:32.006502+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:50:32.006502+0000","last_undegraded":"2024-09-16T14:50:32.006502+0000","last_fullsized":"2024-09-16T14:50:32.006502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:55.441436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:55.441436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:00.621776+0000","scrub_duration":0.0011147150000000001,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'2","reported_seq":140,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T14:50:28.879727+0000","last_change":"2024-09-16T14:49:28.351058+0000","last_active":"2024-09-16T14:50:28.879727+0000","last_peered":"2024-09-16T14:50:28.879727+0000","last_clean":"2024-09-16T14:50:28.879727+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:50:28.879727+0000","last_undegraded":"2024-09-16T14:50:28.879727+0000","last_fullsized":"2024-09-16T14:50:28.879727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2","last_scrub_stamp":"2024-09-16T14:49:28.350938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:28.350938+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:42.292353+0000","scrub_duration":0.0018950309999999999,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":135,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T14:50:28.134432+0000","last_change":"2024-09-16T14:49:27.155463+0000","last_active":"2024-09-16T14:50:28.134432+0000","last_peered":"2024-09-16T14:50:28.134432+0000","last_clean":"2024-09-16T14:50:28.134432+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:50:28.134432+0000","last_undegraded":"2024-09-16T14:50:28.134432+0000","last_fullsized":"2024-09-16T14:50:28.134432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:27.155307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:27.155307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:32.986356+0000","scrub_duration":0.001601157,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":138,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T14:50:26.121957+0000","last_change":"2024-09-16T14:49:53.408437+0000","last_active":"2024-09-16T14:50:26.121957+0000","last_peered":"2024-09-16T14:50:26.121957+0000","last_clean":"2024-09-16T14:50:26.121957+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:50:26.121957+0000","last_undegraded":"2024-09-16T14:50:26.121957+0000","last_fullsized":"2024-09-16T14:50:26.121957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:49:53.408348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:53.408348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:16.057283+0000","scrub_duration":0.0016182830000000001,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'1","reported_seq":139,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-16T14:50:28.879727+0000","last_change":"2024-09-16T14:49:40.527258+0000","last_active":"2024-09-16T14:50:28.879727+0000","last_peered":"2024-09-16T14:50:28.879727+0000","last_clean":"2024-09-16T14:50:28.879727+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:50:28.879727+0000","last_undegraded":"2024-09-16T14:50:28.879727+0000","last_fullsized":"2024-09-16T14:50:28.879727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'1","last_scrub_stamp":"2024-09-16T14:49:40.527164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:23.132742+0000","last_clean_scrub_stamp":"2024-09-16T14:49:40.527164+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:50:46.029615+0000","scrub_duration":0.0012857120000000001,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":307,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-16T14:50:26.121988+0000","last_change":"2024-09-16T14:49:59.398994+0000","last_active":"2024-09-16T14:50:26.121988+0000","last_peered":"2024-09-16T14:50:26.121988+0000","last_clean":"2024-09-16T14:50:26.121988+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:50:26.121988+0000","last_undegraded":"2024-09-16T14:50:26.121988+0000","last_fullsized":"2024-09-16T14:50:26.121988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'137","last_scrub_stamp":"2024-09-16T14:49:59.398890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:48:21.118926+0000","last_clean_scrub_stamp":"2024-09-16T14:49:59.398890+0000","objects_scrubbed":2,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:10.229835+0000","scrub_duration":0.001723186,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411357,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55352,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316488,"statfs":{"total":96636764160,"available":96580083712,"internally_reserved":0,"allocated":2138112,"data_stored":1531716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.80900000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.48299999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79500000000000004,"5min":1.044,"15min":1.044},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.60199999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94199999999999995,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81999999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.70799999999999996}]},{"osd":4,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.68300000000000005}]},{"osd":5,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.66100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.57099999999999995}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.65900000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.62},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68500000000000005,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.82899999999999996}]},{"osd":8,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.377,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.629,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.505}]},{"osd":9,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.747},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.77500000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.5,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.73099999999999998}]}]},{"osd":10,"up_from":18,"seq":77309411359,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50728,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321112,"statfs":{"total":96636764160,"available":96584818688,"internally_reserved":0,"allocated":1601536,"data_stored":1001832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.32300000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.97199999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.38100000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73899999999999999,"5min":1.149,"15min":1.149},"last":0.53100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.33400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.48399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.872,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.63800000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.501},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79200000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46999999999999997}]},{"osd":4,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.628,"15min":0.624},"min":{"1min":0.40500000000000003,"5min":0.31,"15min":0.31},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.71799999999999997}]},{"osd":5,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60999999999999999,"15min":0.60299999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64000000000000001,"15min":0.63300000000000001},"min":{"1min":0.42599999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.58999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72299999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.82299999999999995}]},{"osd":9,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.37,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.38,"5min":0.314,"15min":0.314},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.73999999999999999}]},{"osd":11,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66800000000000004,"15min":0.67400000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96799999999999997,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.70499999999999996}]}]},{"osd":6,"up_from":41,"seq":176093659160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":30974,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1617920,"data_stored":1010442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1965,"internal_metadata":31717459},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.499},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.79400000000000004}]},{"osd":2,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.57099999999999995}]},{"osd":3,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.54600000000000004}]},{"osd":4,"last update":"Mon Sep 16 14:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.69899999999999995}]},{"osd":5,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.60699999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.71399999999999997}]},{"osd":8,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.76900000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.87}]},{"osd":10,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.91400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":1,"up_from":18,"seq":77309411359,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72880,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298960,"statfs":{"total":96636764160,"available":96562135040,"internally_reserved":0,"allocated":1609728,"data_stored":1002807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.54600000000000004,"15min":0.57199999999999995},"min":{"1min":0.26400000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70499999999999996,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.439,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.81999999999999995,"5min":1.806,"15min":1.806},"last":0.29999999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.26000000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.376},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.58999999999999997,"15min":0.61299999999999999},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92700000000000005,"5min":4.0590000000000002,"15min":4.0590000000000002},"last":0.312}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.433}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":5,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66600000000000004}]},{"osd":6,"last update":"Mon Sep 16 14:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.50900000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91000000000000003,"5min":1.831,"15min":1.831},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.77500000000000002,"15min":0.79600000000000004},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.49199999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.55500000000000005,"5min":0.371,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.85799999999999998,"15min":0.875},"min":{"1min":0.54500000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.069,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.59999999999999998}]},{"osd":9,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73599999999999999,"15min":0.73499999999999999},"min":{"1min":0.54100000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1299999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.56999999999999995}]},{"osd":10,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.56299999999999994}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.89800000000000002,"15min":0.92300000000000004},"min":{"1min":0.61099999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1000000000000001,"5min":4.6589999999999998,"15min":4.6589999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.091,"5min":1.855,"15min":1.855},"last":0.60999999999999999}]}]},{"osd":0,"up_from":18,"seq":77309411357,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50732,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321108,"statfs":{"total":96636764160,"available":96584814592,"internally_reserved":0,"allocated":1605632,"data_stored":1002788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55600000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.32200000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.32300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.39500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39900000000000002}]},{"osd":6,"last update":"Mon Sep 16 14:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.499},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.40999999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.53000000000000003}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66500000000000004,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.51100000000000001}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.55100000000000005}]},{"osd":10,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.60899999999999999}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.623},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.56599999999999995}]}]},{"osd":5,"up_from":18,"seq":77309411358,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50736,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321104,"statfs":{"total":96636764160,"available":96584810496,"internally_reserved":0,"allocated":1609728,"data_stored":1002807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.433}]},{"osd":1,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.57899999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60999999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68400000000000005,"5min":0.877,"15min":0.877},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.66000000000000003,"5min":0.746,"15min":0.746},"last":0.51100000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:49:34 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58999999999999997}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.31,"5min":0.252,"15min":0.252},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.437},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.41299999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:49:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.497},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.626}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.72799999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.55300000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91100000000000003,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.79400000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85699999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.204,"5min":1.502,"15min":1.502},"last":0.75800000000000001}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78300000000000003,"15min":0.77600000000000002},"min":{"1min":0.53500000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2270000000000001,"5min":1.456,"15min":1.456},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.746,"15min":0.752},"min":{"1min":0.501,"5min":0.499,"15min":0.499},"max":{"1min":0.96699999999999997,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.78500000000000003}]}]},{"osd":9,"up_from":17,"seq":73014444062,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50732,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321108,"statfs":{"total":96636764160,"available":96584814592,"internally_reserved":0,"allocated":1605632,"data_stored":1002788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53100000000000003}]},{"osd":1,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.626,"15min":0.61299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.55300000000000005}]},{"osd":2,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.35699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":1.601,"15min":1.601},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.498}]},{"osd":3,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":3.0019999999999998,"5min":3.0019999999999998,"15min":3.0019999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.65300000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98599999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.40400000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.93000000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.72699999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.39400000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0620000000000001,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.754}]},{"osd":7,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68500000000000005,"15min":0.69699999999999995},"min":{"1min":0.38900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.876,"5min":1.47,"15min":1.47},"last":0.64400000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81299999999999994,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.35599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95799999999999996,"5min":1.014,"15min":1.014},"last":0.79400000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84199999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.73899999999999999}]},{"osd":11,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.99299999999999999,"5min":1.518,"15min":1.518},"last":0.83299999999999996}]}]},{"osd":8,"up_from":17,"seq":73014444062,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50732,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321108,"statfs":{"total":96636764160,"available":96584814592,"internally_reserved":0,"allocated":1605632,"data_stored":1002788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.38100000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.61299999999999999}]},{"osd":1,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.624},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.34200000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.54200000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85299999999999998,"5min":1.006,"15min":1.006},"last":0.59899999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:49:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.58199999999999996}]},{"osd":6,"last update":"Mon Sep 16 14:49:49 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.63800000000000001}]},{"osd":7,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0680000000000001,"5min":1.202,"15min":1.202},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.89300000000000002,"5min":1.097,"15min":1.097},"last":0.73899999999999999}]},{"osd":9,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82999999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79700000000000004,"5min":1.29,"15min":1.29},"last":0.72299999999999998}]},{"osd":10,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83199999999999996,"5min":1.22,"15min":1.22},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.48799999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94399999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.77800000000000002}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.42999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.496,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":1.254,"15min":1.254},"last":0.70199999999999996}]}]},{"osd":7,"up_from":17,"seq":73014444062,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51252,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320588,"statfs":{"total":96636764160,"available":96584282112,"internally_reserved":0,"allocated":2138112,"data_stored":1531716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":50330060},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5620000000000003,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77100000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68300000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.72799999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83799999999999997,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.33300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.871,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.69499999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.76500000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.61799999999999999}]},{"osd":5,"last update":"Mon Sep 16 14:49:33 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:49:52 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.57899999999999996}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.44,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67300000000000004,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96499999999999997,"5min":1.163,"15min":1.163},"last":0.64400000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.93200000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.90800000000000003,"5min":1.026,"15min":1.026},"last":0.76700000000000002}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90700000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.70399999999999996},"min":{"1min":0.48899999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98399999999999999,"5min":1.079,"15min":1.079},"last":0.81100000000000005}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.52600000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0009999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0129999999999999,"5min":1.04,"15min":1.04},"last":0.74399999999999999}]}]},{"osd":4,"up_from":17,"seq":73014444061,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50732,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321108,"statfs":{"total":96636764160,"available":96584814592,"internally_reserved":0,"allocated":1605632,"data_stored":1002788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47699999999999998,"15min":0.47599999999999998},"min":{"1min":0.318,"5min":0.248,"15min":0.248},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51100000000000001,"15min":0.50700000000000001},"min":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65800000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51100000000000001,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.35599999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.749,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.69699999999999995}]},{"osd":3,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81599999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.72899999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.39900000000000002},"min":{"1min":0.248,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.625},{"interface":"front","average":{"1min":0.443,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.68799999999999994,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.60499999999999998}]},{"osd":6,"last update":"Mon Sep 16 14:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.58699999999999997}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47499999999999998,"15min":0.47299999999999998},"min":{"1min":0.34999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.67600000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.44,"5min":0.45000000000000001,"15min":0.45200000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.71099999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.71099999999999997}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85699999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63600000000000001,"15min":0.63300000000000001},"min":{"1min":0.504,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96699999999999997,"5min":1.018,"15min":1.018},"last":0.88200000000000001}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66700000000000004,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.01,"5min":1.05,"15min":1.05},"last":1.01},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66600000000000004,"15min":0.65800000000000003},"min":{"1min":0.53700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":1.0429999999999999}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.48099999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66500000000000004,"15min":0.65600000000000003},"min":{"1min":0.56599999999999995,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0409999999999999,"5min":1.143,"15min":1.143},"last":0.94899999999999995}]},{"osd":11,"last update":"Mon Sep 16 14:49:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.97399999999999998}]}]},{"osd":3,"up_from":17,"seq":73014444063,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50732,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321108,"statfs":{"total":96636764160,"available":96584814592,"internally_reserved":0,"allocated":1605632,"data_stored":1002788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.28699999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:49:38 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.25},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.29999999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.628,"15min":0.65400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.99199999999999999,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.31900000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.44},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.52600000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94399999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.42899999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.48999999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.03,"5min":1.044,"15min":1.044},"last":0.501}]},{"osd":8,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.61899999999999999}]},{"osd":9,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.67900000000000005,"15min":0.66700000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0469999999999999,"5min":1.079,"15min":1.079},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75800000000000001,"15min":0.754},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.60799999999999998}]},{"osd":10,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.306,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.58999999999999997}]},{"osd":11,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.66200000000000003}]}]},{"osd":2,"up_from":17,"seq":73014444062,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1605632,"data_stored":1002788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:41 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.34699999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.36599999999999999,"15min":0.35399999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.34599999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.28100000000000003}]},{"osd":3,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.44900000000000001,"15min":0.432},"min":{"1min":0.30099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.30199999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.504},{"interface":"front","average":{"1min":0.629,"5min":0.54200000000000004,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.435}]},{"osd":5,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.58499999999999996,"15min":0.56299999999999994},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.47799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.52400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.41499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.442},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.47199999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69699999999999995,"15min":0.68600000000000005},"min":{"1min":0.53100000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.66700000000000004},"min":{"1min":0.51700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.47799999999999998}]},{"osd":9,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.57899999999999996}]},{"osd":10,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.497}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.74299999999999999,"15min":0.72999999999999998},"min":{"1min":0.56799999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.55000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:50:37.434 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T14:50:07.667489+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=50, tm_sec=7, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=50, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:50:37.434 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-16T14:49:54.393679+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=49, tm_sec=54, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=50, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:50:37.434 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T14:49:43.126944+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=49, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=50, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:50:37.434 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-16T14:49:55.441436+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=49, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=50, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:50:37.434 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-16T14:49:28.350938+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=49, tm_sec=28, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=50, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:50:37.434 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-16T14:49:27.155307+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=49, tm_sec=27, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=50, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:50:37.434 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T14:49:53.408348+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=49, tm_sec=53, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=50, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:50:37.435 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-16T14:49:40.527164+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=49, tm_sec=40, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=50, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:50:37.435 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T14:49:59.398890+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=49, tm_sec=59, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=14, tm_min=50, tm_sec=30, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T14:50:37.435 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T14:50:57.435 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T14:50:57.737 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-16T14:50:57.738 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-16T14:50:57.749 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-16T14:50:57.164531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":183,"num_write_kb":2940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":10,"kb":1132462080,"kb_used":600708,"kb_used_data":20724,"kb_used_omap":19,"kb_used_meta":579820,"kb_avail":1131861372,"statfs":{"total":1159641169920,"available":1159026044928,"internally_reserved":0,"allocated":21221376,"data_stored":13641482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19502,"internal_metadata":593736658},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.105697"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":106,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T14:50:47.847413+0000","last_change":"2024-09-16T14:50:38.629001+0000","last_active":"2024-09-16T14:50:47.847413+0000","last_peered":"2024-09-16T14:50:47.847413+0000","last_clean":"2024-09-16T14:50:47.847413+0000","last_became_active":"2024-09-16T14:49:12.961697+0000","last_became_peered":"2024-09-16T14:49:12.961697+0000","last_unstale":"2024-09-16T14:50:47.847413+0000","last_undegraded":"2024-09-16T14:50:47.847413+0000","last_fullsized":"2024-09-16T14:50:47.847413+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:50:38.628884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:50:38.628884+0000","last_clean_scrub_stamp":"2024-09-16T14:50:38.628884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:43.657195+0000","scrub_duration":0.0019174909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10],"acting":[6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":153,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T14:50:48.202834+0000","last_change":"2024-09-16T14:50:33.292947+0000","last_active":"2024-09-16T14:50:48.202834+0000","last_peered":"2024-09-16T14:50:48.202834+0000","last_clean":"2024-09-16T14:50:48.202834+0000","last_became_active":"2024-09-16T14:48:24.155902+0000","last_became_peered":"2024-09-16T14:48:24.155902+0000","last_unstale":"2024-09-16T14:50:48.202834+0000","last_undegraded":"2024-09-16T14:50:48.202834+0000","last_fullsized":"2024-09-16T14:50:48.202834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:50:33.292833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:50:33.292833+0000","last_clean_scrub_stamp":"2024-09-16T14:50:33.292833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:52:00.435701+0000","scrub_duration":0.0018343980000000001,"objects_trimmed":0,"snaptrim_duration":0.00040417200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8],"acting":[1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":155,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T14:50:47.848141+0000","last_change":"2024-09-16T14:50:38.410959+0000","last_active":"2024-09-16T14:50:47.848141+0000","last_peered":"2024-09-16T14:50:47.848141+0000","last_clean":"2024-09-16T14:50:47.848141+0000","last_became_active":"2024-09-16T14:48:24.163044+0000","last_became_peered":"2024-09-16T14:48:24.163044+0000","last_unstale":"2024-09-16T14:50:47.848141+0000","last_undegraded":"2024-09-16T14:50:47.848141+0000","last_fullsized":"2024-09-16T14:50:47.848141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:50:38.410814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:50:38.410814+0000","last_clean_scrub_stamp":"2024-09-16T14:50:38.410814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:55.864241+0000","scrub_duration":0.001833791,"objects_trimmed":0,"snaptrim_duration":0.00031000599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":153,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T14:50:48.202836+0000","last_change":"2024-09-16T14:50:38.225091+0000","last_active":"2024-09-16T14:50:48.202836+0000","last_peered":"2024-09-16T14:50:48.202836+0000","last_clean":"2024-09-16T14:50:48.202836+0000","last_became_active":"2024-09-16T14:48:24.163074+0000","last_became_peered":"2024-09-16T14:48:24.163074+0000","last_unstale":"2024-09-16T14:50:48.202836+0000","last_undegraded":"2024-09-16T14:50:48.202836+0000","last_fullsized":"2024-09-16T14:50:48.202836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:50:38.224966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:50:38.224966+0000","last_clean_scrub_stamp":"2024-09-16T14:50:38.224966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:52:01.406302+0000","scrub_duration":0.0013687949999999999,"objects_trimmed":0,"snaptrim_duration":0.00043194099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"137'3","reported_seq":159,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T14:50:47.351589+0000","last_change":"2024-09-16T14:50:39.211312+0000","last_active":"2024-09-16T14:50:47.351589+0000","last_peered":"2024-09-16T14:50:47.351589+0000","last_clean":"2024-09-16T14:50:47.351589+0000","last_became_active":"2024-09-16T14:48:24.147958+0000","last_became_peered":"2024-09-16T14:48:24.147958+0000","last_unstale":"2024-09-16T14:50:47.351589+0000","last_undegraded":"2024-09-16T14:50:47.351589+0000","last_fullsized":"2024-09-16T14:50:47.351589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'3","last_scrub_stamp":"2024-09-16T14:50:39.211187+0000","last_deep_scrub":"137'3","last_deep_scrub_stamp":"2024-09-16T14:50:39.211187+0000","last_clean_scrub_stamp":"2024-09-16T14:50:39.211187+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:52:03.460491+0000","scrub_duration":0.003587406,"objects_trimmed":0,"snaptrim_duration":0.00075345000000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":164,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T14:50:47.837976+0000","last_change":"2024-09-16T14:50:35.179004+0000","last_active":"2024-09-16T14:50:47.837976+0000","last_peered":"2024-09-16T14:50:47.837976+0000","last_clean":"2024-09-16T14:50:47.837976+0000","last_became_active":"2024-09-16T14:48:24.163034+0000","last_became_peered":"2024-09-16T14:48:24.163034+0000","last_unstale":"2024-09-16T14:50:47.837976+0000","last_undegraded":"2024-09-16T14:50:47.837976+0000","last_fullsized":"2024-09-16T14:50:47.837976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:50:35.178919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:50:35.178919+0000","last_clean_scrub_stamp":"2024-09-16T14:50:35.178919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:55.031972+0000","scrub_duration":0.0010967100000000001,"objects_trimmed":0,"snaptrim_duration":0.00031534899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":156,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T14:50:47.847414+0000","last_change":"2024-09-16T14:50:42.607998+0000","last_active":"2024-09-16T14:50:47.847414+0000","last_peered":"2024-09-16T14:50:47.847414+0000","last_clean":"2024-09-16T14:50:47.847414+0000","last_became_active":"2024-09-16T14:48:24.165792+0000","last_became_peered":"2024-09-16T14:48:24.165792+0000","last_unstale":"2024-09-16T14:50:47.847414+0000","last_undegraded":"2024-09-16T14:50:47.847414+0000","last_fullsized":"2024-09-16T14:50:47.847414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T14:50:42.607872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T14:50:42.607872+0000","last_clean_scrub_stamp":"2024-09-16T14:50:42.607872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:43.844195+0000","scrub_duration":0.0018025579999999999,"objects_trimmed":0,"snaptrim_duration":0.00028622599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,10],"acting":[7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"137'2","reported_seq":158,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T14:50:47.351595+0000","last_change":"2024-09-16T14:50:36.208761+0000","last_active":"2024-09-16T14:50:47.351595+0000","last_peered":"2024-09-16T14:50:47.351595+0000","last_clean":"2024-09-16T14:50:47.351595+0000","last_became_active":"2024-09-16T14:48:24.147884+0000","last_became_peered":"2024-09-16T14:48:24.147884+0000","last_unstale":"2024-09-16T14:50:47.351595+0000","last_undegraded":"2024-09-16T14:50:47.351595+0000","last_fullsized":"2024-09-16T14:50:47.351595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'2","last_scrub_stamp":"2024-09-16T14:50:36.208646+0000","last_deep_scrub":"137'2","last_deep_scrub_stamp":"2024-09-16T14:50:36.208646+0000","last_clean_scrub_stamp":"2024-09-16T14:50:36.208646+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:52:00.050516+0000","scrub_duration":0.002543188,"objects_trimmed":0,"snaptrim_duration":0.000311608,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"137'139","reported_seq":327,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-16T14:50:47.847427+0000","last_change":"2024-09-16T14:50:39.581029+0000","last_active":"2024-09-16T14:50:47.847427+0000","last_peered":"2024-09-16T14:50:47.847427+0000","last_clean":"2024-09-16T14:50:47.847427+0000","last_became_active":"2024-09-16T14:48:22.359007+0000","last_became_peered":"2024-09-16T14:48:22.359007+0000","last_unstale":"2024-09-16T14:50:47.847427+0000","last_undegraded":"2024-09-16T14:50:47.847427+0000","last_fullsized":"2024-09-16T14:50:47.847427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'139","last_scrub_stamp":"2024-09-16T14:50:39.580887+0000","last_deep_scrub":"137'139","last_deep_scrub_stamp":"2024-09-16T14:50:39.580887+0000","last_clean_scrub_stamp":"2024-09-16T14:50:39.580887+0000","objects_scrubbed":2,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T14:51:58.296475+0000","scrub_duration":0.0048622259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11],"acting":[7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":11,"up_from":18,"seq":77309411362,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55420,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316420,"statfs":{"total":96636764160,"available":96580014080,"internally_reserved":0,"allocated":2211840,"data_stored":1577640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54524357},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.499,"15min":0.495},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.80900000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.71699999999999997}]},{"osd":1,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.56899999999999995}]},{"osd":2,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79500000000000004,"5min":1.044,"15min":1.044},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.77400000000000002}]},{"osd":3,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94199999999999995,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.81999999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.74099999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.42299999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.628}]},{"osd":5,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66200000000000003,"15min":0.66100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66800000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.59099999999999997}]},{"osd":6,"last update":"Mon Sep 16 14:50:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.66900000000000004,"15min":0.66300000000000003},"min":{"1min":0.58999999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.57799999999999996,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.51800000000000002}]},{"osd":7,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.68500000000000005,"15min":0.68000000000000005},"min":{"1min":0.51800000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.61199999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.377,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.629,"15min":0.627},"min":{"1min":0.45400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.46300000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:50:36 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65900000000000003,"15min":0.66000000000000003},"min":{"1min":0.42999999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85399999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.55900000000000005}]},{"osd":10,"last update":"Mon Sep 16 14:50:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.5,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.755}]}]},{"osd":10,"up_from":18,"seq":77309411364,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50800,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321040,"statfs":{"total":96636764160,"available":96584744960,"internally_reserved":0,"allocated":1675264,"data_stored":1047756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.495,"15min":0.48699999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.32300000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.97199999999999998,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73899999999999999,"5min":1.149,"15min":1.149},"last":0.57799999999999996}]},{"osd":2,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52600000000000002,"15min":0.51600000000000001},"min":{"1min":0.33400000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.48399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.872,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.624}]},{"osd":3,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79200000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.49299999999999999}]},{"osd":4,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.628,"15min":0.624},"min":{"1min":0.40500000000000003,"5min":0.31,"15min":0.31},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.751}]},{"osd":5,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60999999999999999,"15min":0.60299999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64000000000000001,"15min":0.63300000000000001},"min":{"1min":0.42599999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.79200000000000004}]},{"osd":6,"last update":"Mon Sep 16 14:50:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.83299999999999996}]},{"osd":7,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72299999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.75600000000000001,"5min":1,"15min":1},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.66800000000000004}]},{"osd":9,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.37,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.38,"5min":0.314,"15min":0.314},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.81499999999999995}]},{"osd":11,"last update":"Mon Sep 16 14:50:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66800000000000004,"15min":0.67400000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96799999999999997,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.77800000000000002}]}]},{"osd":6,"up_from":41,"seq":176093659164,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":30974,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1679360,"data_stored":1048712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1965,"internal_metadata":31717459},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:39 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.54100000000000004}]},{"osd":1,"last update":"Mon Sep 16 14:50:50 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51200000000000001,"15min":0.503},"min":{"1min":0.438,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80800000000000005,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.625,"5min":0.50600000000000001,"15min":0.48599999999999999},"min":{"1min":0.41399999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:50:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"min":{"1min":0.35899999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.51900000000000002,"15min":0.505},"min":{"1min":0.432,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83499999999999996,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56899999999999995}]},{"osd":3,"last update":"Mon Sep 16 14:50:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.79700000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.56299999999999994,"15min":0.55100000000000005},"min":{"1min":0.442,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85399999999999998,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.754}]},{"osd":4,"last update":"Mon Sep 16 14:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.65100000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:50:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.47799999999999998,"15min":0.46800000000000003},"min":{"1min":0.314,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.36399999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83999999999999997,"5min":0.872,"15min":0.872},"last":0.73799999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:50:50 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.34799999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.39600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.88100000000000001,"5min":1.288,"15min":1.288},"last":0.66800000000000004}]},{"osd":8,"last update":"Mon Sep 16 14:50:50 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68600000000000005,"15min":0.67400000000000004},"min":{"1min":0.57899999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.96399999999999997,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.67600000000000005,"15min":0.65700000000000003},"min":{"1min":0.47499999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.96899999999999997}]},{"osd":9,"last update":"Mon Sep 16 14:50:50 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.44600000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.98199999999999998,"5min":1.262,"15min":1.262},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.70099999999999996,"15min":0.68700000000000006},"min":{"1min":0.60599999999999998,"5min":0.437,"15min":0.437},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.93999999999999995}]},{"osd":10,"last update":"Mon Sep 16 14:50:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.63800000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.046,"5min":1.194,"15min":1.194},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.50600000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.034,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.91200000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]}]},{"osd":1,"up_from":18,"seq":77309411364,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72952,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298888,"statfs":{"total":96636764160,"available":96562061312,"internally_reserved":0,"allocated":1683456,"data_stored":1048731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.54600000000000004,"15min":0.57199999999999995},"min":{"1min":0.26400000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70499999999999996,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.439,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.81999999999999995,"5min":1.806,"15min":1.806},"last":0.376}]},{"osd":2,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.26000000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.315},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.58999999999999997,"15min":0.61299999999999999},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92700000000000005,"5min":4.0590000000000002,"15min":4.0590000000000002},"last":0.41599999999999998}]},{"osd":3,"last update":"Mon Sep 16 14:49:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":5.7359999999999998,"5min":5.7359999999999998,"15min":5.7359999999999998},"last":0.433}]},{"osd":4,"last update":"Mon Sep 16 14:50:41 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.72399999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61899999999999999,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.80900000000000005,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.63100000000000001}]},{"osd":6,"last update":"Mon Sep 16 14:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58299999999999996}]},{"osd":7,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.41099999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91000000000000003,"5min":1.831,"15min":1.831},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.77500000000000002,"15min":0.79600000000000004},"min":{"1min":0.46800000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0389999999999999,"5min":3.9620000000000002,"15min":3.9620000000000002},"last":0.69499999999999995}]},{"osd":8,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.55500000000000005,"5min":0.371,"15min":0.371},"max":{"1min":0.97899999999999998,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.85799999999999998,"15min":0.875},"min":{"1min":0.54500000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.069,"5min":4.0030000000000001,"15min":4.0030000000000001},"last":0.56499999999999995}]},{"osd":9,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73599999999999999,"15min":0.73499999999999999},"min":{"1min":0.54100000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.1299999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.46999999999999997}]},{"osd":10,"last update":"Mon Sep 16 14:50:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.51900000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.93700000000000006,"5min":1.179,"15min":1.179},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.77500000000000002,"15min":0.78200000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0920000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.499}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.89800000000000002,"15min":0.92300000000000004},"min":{"1min":0.61099999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1000000000000001,"5min":4.6589999999999998,"15min":4.6589999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78900000000000003,"15min":0.79400000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.091,"5min":1.855,"15min":1.855},"last":0.52000000000000002}]}]},{"osd":0,"up_from":18,"seq":77309411362,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50804,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321036,"statfs":{"total":96636764160,"available":96584740864,"internally_reserved":0,"allocated":1679360,"data_stored":1048712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.41199999999999998,"15min":0.41799999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.436,"15min":0.443},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55600000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.63600000000000001}]},{"osd":2,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.32300000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:50:39 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.56499999999999995}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.39500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.78700000000000003}]},{"osd":5,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58499999999999996,"15min":0.58399999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.39900000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.83799999999999997}]},{"osd":6,"last update":"Mon Sep 16 14:50:15 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.82099999999999995}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.624,"15min":0.624},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.745},{"interface":"front","average":{"1min":0.627,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.70499999999999996}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.66500000000000004,"15min":0.66000000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67900000000000005,"15min":0.68000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.88800000000000001}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.68200000000000005,"15min":0.68400000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90400000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70699999999999996,"15min":0.70899999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.95499999999999996}]},{"osd":10,"last update":"Mon Sep 16 14:50:39 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.51900000000000002,"5min":0.503,"15min":0.503},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.82199999999999995}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.93700000000000006,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":1.109}]}]},{"osd":5,"up_from":18,"seq":77309411363,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50808,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321032,"statfs":{"total":96636764160,"available":96584736768,"internally_reserved":0,"allocated":1683456,"data_stored":1048731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:25 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.433}]},{"osd":1,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.77000000000000002,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.33400000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.57899999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67700000000000005}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54000000000000004,"15min":0.54300000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68400000000000005,"5min":0.877,"15min":0.877},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.66000000000000003,"5min":0.746,"15min":0.746},"last":0.629}]},{"osd":3,"last update":"Mon Sep 16 14:50:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.65700000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.746}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.31,"5min":0.252,"15min":0.252},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84799999999999998,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Mon Sep 16 14:50:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.45000000000000001,"15min":0.432},"min":{"1min":0.29199999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.52400000000000002,"15min":0.498},"min":{"1min":0.44500000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.71999999999999997}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.376,"5min":0.311,"15min":0.311},"max":{"1min":0.72799999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.39900000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.61799999999999999}]},{"osd":8,"last update":"Mon Sep 16 14:50:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.93600000000000005}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71399999999999997,"15min":0.71799999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.91100000000000003,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.98899999999999999}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85699999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.204,"5min":1.502,"15min":1.502},"last":0.82199999999999995}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78300000000000003,"15min":0.77600000000000002},"min":{"1min":0.53500000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2270000000000001,"5min":1.456,"15min":1.456},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.746,"15min":0.752},"min":{"1min":0.501,"5min":0.499,"15min":0.499},"max":{"1min":0.96699999999999997,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.97499999999999998}]}]},{"osd":9,"up_from":17,"seq":73014444067,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50804,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321036,"statfs":{"total":96636764160,"available":96584740864,"internally_reserved":0,"allocated":1679360,"data_stored":1048712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":50330049},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55500000000000005,"15min":0.55600000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.44700000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.626,"15min":0.61299999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.8460000000000001,"5min":2.8460000000000001,"15min":2.8460000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.46200000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.35699999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.80900000000000005,"5min":1.601,"15min":1.601},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.65700000000000003}]},{"osd":3,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":3.0019999999999998,"5min":3.0019999999999998,"15min":3.0019999999999998},"last":0.40799999999999997}]},{"osd":4,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.65300000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.98599999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64300000000000002,"15min":0.64400000000000002},"min":{"1min":0.40400000000000003,"5min":0.308,"15min":0.308},"max":{"1min":0.93000000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.60599999999999998}]},{"osd":5,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65600000000000003,"15min":0.65300000000000002},"min":{"1min":0.39400000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0620000000000001,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.69099999999999995}]},{"osd":7,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.69899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68500000000000005,"15min":0.69699999999999995},"min":{"1min":0.38900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.876,"5min":1.47,"15min":1.47},"last":0.55500000000000005}]},{"osd":8,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81299999999999994,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.627},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.35599999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95799999999999996,"5min":1.014,"15min":1.014},"last":0.61499999999999999}]},{"osd":10,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84199999999999997,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65600000000000003,"15min":0.66300000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.56599999999999995}]},{"osd":11,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.67200000000000004,"15min":0.68500000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.99299999999999999,"5min":1.518,"15min":1.518},"last":0.64200000000000002}]}]},{"osd":8,"up_from":17,"seq":73014444067,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50804,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321036,"statfs":{"total":96636764160,"available":96584740864,"internally_reserved":0,"allocated":1679360,"data_stored":1048712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.499,"15min":0.48999999999999999},"min":{"1min":0.38100000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.66500000000000004}]},{"osd":1,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.34200000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.64800000000000002}]},{"osd":2,"last update":"Mon Sep 16 14:50:40 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.76100000000000001}]},{"osd":3,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58799999999999997,"15min":0.57999999999999996},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65300000000000002,"15min":0.64700000000000002},"min":{"1min":0.54200000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85299999999999998,"5min":1.006,"15min":1.006},"last":0.69599999999999995}]},{"osd":5,"last update":"Mon Sep 16 14:50:40 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.51000000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86299999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.70999999999999996}]},{"osd":6,"last update":"Mon Sep 16 14:50:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.64800000000000002,"15min":0.63400000000000001},"min":{"1min":0.48599999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.024,"5min":1.151,"15min":1.151},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64500000000000002,"15min":0.63300000000000001},"min":{"1min":0.48499999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.98899999999999999,"5min":1.044,"15min":1.044},"last":0.73999999999999999}]},{"osd":7,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.45000000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0680000000000001,"5min":1.202,"15min":1.202},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.89300000000000002,"5min":1.097,"15min":1.097},"last":0.77700000000000002}]},{"osd":9,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82999999999999996,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79700000000000004,"5min":1.29,"15min":1.29},"last":0.42899999999999999}]},{"osd":10,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83199999999999996,"5min":1.22,"15min":1.22},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.68899999999999995,"15min":0.67800000000000005},"min":{"1min":0.48799999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.94399999999999995,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.57599999999999996}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.42999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86099999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68200000000000005,"15min":0.68500000000000005},"min":{"1min":0.496,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":1.254,"15min":1.254},"last":0.46000000000000002}]}]},{"osd":7,"up_from":17,"seq":73014444067,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51324,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320516,"statfs":{"total":96636764160,"available":96584208384,"internally_reserved":0,"allocated":2211840,"data_stored":1577640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":50330060},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:24 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.5620000000000003,"5min":4.5620000000000003,"15min":4.5620000000000003},"last":0.46000000000000002}]},{"osd":1,"last update":"Mon Sep 16 14:50:40 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.63},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.48599999999999999}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.77100000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68300000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68999999999999995}]},{"osd":3,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34999999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83799999999999997,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.499},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.33300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.871,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.59899999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.76500000000000001,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.505,"15min":0.5},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82199999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.44}]},{"osd":5,"last update":"Mon Sep 16 14:50:37 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46300000000000002,"15min":0.46000000000000002},"min":{"1min":0.313,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.81100000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89500000000000002,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.42599999999999999}]},{"osd":6,"last update":"Mon Sep 16 14:50:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.505,"15min":0.50700000000000001},"min":{"1min":0.29999999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.71399999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.38900000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86199999999999999,"5min":0.874,"15min":0.874},"last":0.39200000000000002}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.65600000000000003},"min":{"1min":0.44,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.67300000000000004,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96499999999999997,"5min":1.163,"15min":1.163},"last":0.751}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.93200000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.90800000000000003,"5min":1.026,"15min":1.026},"last":0.67400000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90700000000000003,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.70399999999999996},"min":{"1min":0.48899999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98399999999999999,"5min":1.079,"15min":1.079},"last":0.73799999999999999}]},{"osd":11,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70499999999999996,"15min":0.69699999999999995},"min":{"1min":0.52600000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0009999999999999,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0129999999999999,"5min":1.04,"15min":1.04},"last":0.70599999999999996}]}]},{"osd":4,"up_from":17,"seq":73014444066,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50804,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321036,"statfs":{"total":96636764160,"available":96584740864,"internally_reserved":0,"allocated":1679360,"data_stored":1048712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47699999999999998,"15min":0.47599999999999998},"min":{"1min":0.318,"5min":0.248,"15min":0.248},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51100000000000001,"15min":0.50700000000000001},"min":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.65800000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.51100000000000001,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.35599999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.749,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.56799999999999995}]},{"osd":3,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.81599999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.55000000000000004}]},{"osd":5,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.40100000000000002,"15min":0.39900000000000002},"min":{"1min":0.248,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.45500000000000002,"15min":0.45700000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.68799999999999994,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.378}]},{"osd":6,"last update":"Mon Sep 16 14:50:13 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.59699999999999998}]},{"osd":7,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47499999999999998,"15min":0.47299999999999998},"min":{"1min":0.34999999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.67600000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.45000000000000001,"15min":0.45200000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.71099999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.41199999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85699999999999998,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63600000000000001,"15min":0.63300000000000001},"min":{"1min":0.504,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96699999999999997,"5min":1.018,"15min":1.018},"last":0.65400000000000003}]},{"osd":9,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66700000000000004,"15min":0.66300000000000003},"min":{"1min":0.45000000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.01,"5min":1.05,"15min":1.05},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.66600000000000004,"15min":0.65800000000000003},"min":{"1min":0.53700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.79400000000000004}]},{"osd":10,"last update":"Mon Sep 16 14:50:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66600000000000004,"15min":0.66500000000000004},"min":{"1min":0.48099999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66500000000000004,"15min":0.65600000000000003},"min":{"1min":0.56599999999999995,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0409999999999999,"5min":1.143,"15min":1.143},"last":0.71899999999999997}]},{"osd":11,"last update":"Mon Sep 16 14:50:32 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98199999999999998,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0649999999999999,"5min":1.119,"15min":1.119},"last":0.746}]}]},{"osd":3,"up_from":17,"seq":73014444068,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50804,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321036,"statfs":{"total":96636764160,"available":96584740864,"internally_reserved":0,"allocated":1679360,"data_stored":1048712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:49:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.28699999999999998}]},{"osd":1,"last update":"Mon Sep 16 14:50:40 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.435,"15min":0.438},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.498},{"interface":"front","average":{"1min":0.377,"5min":0.40500000000000003,"15min":0.40999999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.40400000000000003}]},{"osd":2,"last update":"Mon Sep 16 14:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38200000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.628,"15min":0.65400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.99199999999999999,"5min":4.3440000000000003,"15min":4.3440000000000003},"last":0.44400000000000001}]},{"osd":4,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.52800000000000002}]},{"osd":5,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.94399999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53800000000000003}]},{"osd":6,"last update":"Mon Sep 16 14:50:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.622},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55300000000000005}]},{"osd":7,"last update":"Mon Sep 16 14:50:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.03,"5min":1.044,"15min":1.044},"last":0.60499999999999998}]},{"osd":8,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.66600000000000004}]},{"osd":9,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.67900000000000005,"15min":0.66700000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0469999999999999,"5min":1.079,"15min":1.079},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75800000000000001,"15min":0.754},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.64100000000000001}]},{"osd":10,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.306,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.75}]},{"osd":11,"last update":"Mon Sep 16 14:50:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.78900000000000003}]}]},{"osd":2,"up_from":17,"seq":73014444067,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1679360,"data_stored":1048712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 14:50:41 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.38600000000000001,"15min":0.372},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.436,"15min":0.41999999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.45000000000000001}]},{"osd":1,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.36599999999999999,"15min":0.35399999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.42199999999999999,"15min":0.40500000000000003},"min":{"1min":0.34599999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.36099999999999999}]},{"osd":3,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.44900000000000001,"15min":0.432},"min":{"1min":0.30099999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.434},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.32300000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.41699999999999998}]},{"osd":4,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.54200000000000004,"15min":0.52800000000000002},"min":{"1min":0.40799999999999997,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51600000000000001}]},{"osd":5,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.58499999999999996,"15min":0.56299999999999994},"min":{"1min":0.45400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.57799999999999996},"min":{"1min":0.47799999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":7,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.41499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77000000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64100000000000001}]},{"osd":8,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.69699999999999995,"15min":0.68600000000000005},"min":{"1min":0.53100000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.67500000000000004,"15min":0.66700000000000004},"min":{"1min":0.51700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.70799999999999996}]},{"osd":9,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.72899999999999998}]},{"osd":10,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.71699999999999997,"15min":0.70299999999999996},"min":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.78700000000000003}]},{"osd":11,"last update":"Mon Sep 16 14:50:25 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.74299999999999999,"15min":0.72999999999999998},"min":{"1min":0.56799999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.66800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T14:50:57.754 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T14:50:58.053 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T14:50:58.054 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T14:50:58.054 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T14:50:58.054 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.092 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T14:50:58.092 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T14:50:58.093 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.163 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T14:50:58.164 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T14:50:58.164 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.233 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T14:50:58.233 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T14:50:58.233 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.310 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T14:50:58.310 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-16T14:50:58.310 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.353 INFO:tasks.ceph.osd.4:Stopped 2024-09-16T14:50:58.354 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-16T14:50:58.354 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.393 INFO:tasks.ceph.osd.5:Stopped 2024-09-16T14:50:58.393 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-16T14:50:58.393 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.469 INFO:tasks.ceph.osd.6:Stopped 2024-09-16T14:50:58.469 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-16T14:50:58.469 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.547 INFO:tasks.ceph.osd.7:Stopped 2024-09-16T14:50:58.547 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2024-09-16T14:50:58.547 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.590 INFO:tasks.ceph.osd.8:Stopped 2024-09-16T14:50:58.590 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2024-09-16T14:50:58.591 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.623 INFO:tasks.ceph.osd.9:Stopped 2024-09-16T14:50:58.624 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2024-09-16T14:50:58.624 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.704 INFO:tasks.ceph.osd.10:Stopped 2024-09-16T14:50:58.704 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2024-09-16T14:50:58.704 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.777 INFO:tasks.ceph.osd.11:Stopped 2024-09-16T14:50:58.777 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T14:50:58.778 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T14:50:58.778 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.808 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T14:50:58.808 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T14:50:58.808 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T14:50:58.808 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.821 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T14:50:58.821 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T14:50:58.821 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.861 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T14:50:58.861 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T14:50:58.861 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T14:50:58.874 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T14:50:58.874 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T14:50:58.875 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-16T14:50:58.911 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-16T14:50:58.911 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T14:50:59.113 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-16T14:50:59.113 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T14:50:59.188 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-16T14:50:59.188 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T14:50:59.243 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-16T14:50:59.243 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-16T14:50:59.294 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-16T14:50:59.295 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-16T14:50:59.445 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-16T14:50:59.445 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-16T14:50:59.534 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-16T14:50:59.534 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-16T14:50:59.581 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi111.front.sepia.ceph.com 2024-09-16T14:50:59.581 DEBUG:teuthology.orchestra.run.smithi111:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-16T14:50:59.624 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-16T14:50:59.624 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2024-09-16T14:50:59.758 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-16T14:50:59.758 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2024-09-16T14:50:59.870 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-16T14:50:59.871 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2024-09-16T14:50:59.956 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@smithi114.front.sepia.ceph.com 2024-09-16T14:50:59.957 DEBUG:teuthology.orchestra.run.smithi114:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2024-09-16T14:51:00.070 INFO:tasks.ceph:Archiving mon data... 2024-09-16T14:51:00.125 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764/data/mon.a.tgz 2024-09-16T14:51:00.126 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-09-16T14:51:00.150 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.YMaZOeICQ5 2024-09-16T14:51:00.150 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.YMaZOeICQ5 2024-09-16T14:51:00.308 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.YMaZOeICQ5 2024-09-16T14:51:00.395 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.YMaZOeICQ5 is 336KB 2024-09-16T14:51:00.490 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.YMaZOeICQ5 2024-09-16T14:51:00.507 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764/data/mon.b.tgz 2024-09-16T14:51:00.507 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2024-09-16T14:51:00.525 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.xcuT8PgfwD 2024-09-16T14:51:00.526 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.xcuT8PgfwD 2024-09-16T14:51:00.720 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.xcuT8PgfwD 2024-09-16T14:51:00.805 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.xcuT8PgfwD is 464KB 2024-09-16T14:51:00.905 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.xcuT8PgfwD 2024-09-16T14:51:00.920 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764/data/mon.c.tgz 2024-09-16T14:51:00.920 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-09-16T14:51:00.939 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.cazzDdlvDQ 2024-09-16T14:51:00.939 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.cazzDdlvDQ 2024-09-16T14:51:01.128 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.cazzDdlvDQ 2024-09-16T14:51:01.212 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.cazzDdlvDQ is 463KB 2024-09-16T14:51:01.309 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.cazzDdlvDQ 2024-09-16T14:51:01.324 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T14:51:01.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T14:51:01.327 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T14:51:01.329 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T14:51:01.388 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-16T14:51:01.388 DEBUG:teuthology.orchestra.run.smithi040:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T14:51:01.397 DEBUG:teuthology.orchestra.run.smithi111:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T14:51:01.399 DEBUG:teuthology.orchestra.run.smithi114:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T14:51:01.425 INFO:teuthology.orchestra.run.smithi111.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T14:51:01.427 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T14:51:01.455 INFO:teuthology.orchestra.run.smithi114.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-16T14:51:01.459 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T14:51:01.460 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764/remote/smithi040/crash 2024-09-16T14:51:01.461 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T14:51:01.501 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764/remote/smithi111/crash 2024-09-16T14:51:01.502 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T14:51:01.527 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764/remote/smithi114/crash 2024-09-16T14:51:01.528 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T14:51:01.552 INFO:tasks.ceph:Compressing logs... 2024-09-16T14:51:01.552 DEBUG:teuthology.orchestra.run.smithi040:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T14:51:01.555 DEBUG:teuthology.orchestra.run.smithi111:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T14:51:01.572 DEBUG:teuthology.orchestra.run.smithi114:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T14:51:06.324 INFO:tasks.ceph:Archiving logs... 2024-09-16T14:51:06.324 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764/remote/smithi040/log 2024-09-16T14:51:06.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T14:51:07.095 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764/remote/smithi111/log 2024-09-16T14:51:07.096 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T14:51:07.646 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764/remote/smithi114/log 2024-09-16T14:51:07.647 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T14:51:10.851 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T14:51:10.858 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T14:51:10.858 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T14:51:10.861 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T14:51:10.864 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T14:51:10.969 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T14:51:10.979 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T14:51:10.979 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:51:10.982 DEBUG:teuthology.orchestra.run.smithi111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:51:10.984 DEBUG:teuthology.orchestra.run.smithi114:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T14:51:10.998 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-09-16T14:51:10.999 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-09-16T14:51:11.002 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T14:51:11.002 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-09-16T14:51:11.002 INFO:teuthology.orchestra.run.smithi040.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 +154us[ +156us] +/- 51ms 2024-09-16T14:51:11.002 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 -350us[ -350us] +/- 62ms 2024-09-16T14:51:11.003 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +255us[ +255us] +/- 84ms 2024-09-16T14:51:11.003 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:51:11.004 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T14:51:11.004 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-09-16T14:51:11.005 INFO:teuthology.orchestra.run.smithi111.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 +152us[ +156us] +/- 51ms 2024-09-16T14:51:11.005 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 38 -344us[ -341us] +/- 62ms 2024-09-16T14:51:11.005 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +253us[ +256us] +/- 84ms 2024-09-16T14:51:11.005 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:51:11.005 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-09-16T14:51:11.009 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T14:51:11.009 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-09-16T14:51:11.009 INFO:teuthology.orchestra.run.smithi114.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 +158us[ +168us] +/- 51ms 2024-09-16T14:51:11.009 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 -352us[ -343us] +/- 62ms 2024-09-16T14:51:11.009 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 +237us[ +246us] +/- 84ms 2024-09-16T14:51:11.009 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T14:51:11.010 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T14:51:11.019 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T14:51:11.019 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T14:51:11.028 DEBUG:teuthology.orchestra.run.smithi040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T14:51:11.125 DEBUG:teuthology.orchestra.run.smithi111:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T14:51:11.196 DEBUG:teuthology.orchestra.run.smithi114:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T14:51:11.268 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T14:51:11.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:51:11.301 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T14:51:11.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:51:11.330 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T14:51:11.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:51:11.360 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T14:51:11.371 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T14:51:11.380 INFO:teuthology.task.internal:Duration was 595.655990 seconds 2024-09-16T14:51:11.380 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T14:51:11.391 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T14:51:11.391 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T14:51:11.394 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T14:51:11.397 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T14:51:11.434 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T14:51:11.439 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T14:51:11.442 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T14:51:11.783 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T14:51:11.783 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2024-09-16T14:51:11.783 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T14:51:11.811 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi111.front.sepia.ceph.com 2024-09-16T14:51:11.812 DEBUG:teuthology.orchestra.run.smithi111:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T14:51:11.845 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi114.front.sepia.ceph.com 2024-09-16T14:51:11.846 DEBUG:teuthology.orchestra.run.smithi114:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T14:51:11.880 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T14:51:11.880 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T14:51:11.883 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T14:51:11.888 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T14:51:11.947 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T14:51:11.948 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T14:51:11.956 DEBUG:teuthology.orchestra.run.smithi111:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T14:51:11.958 DEBUG:teuthology.orchestra.run.smithi114:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T14:51:12.094 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T14:51:12.104 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T14:51:12.105 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T14:51:12.132 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T14:51:12.157 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T14:51:12.183 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T14:51:12.194 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T14:51:12.197 DEBUG:teuthology.orchestra.run.smithi111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T14:51:12.201 DEBUG:teuthology.orchestra.run.smithi114:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T14:51:12.220 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2024-09-16T14:51:12.221 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = core 2024-09-16T14:51:12.246 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = core 2024-09-16T14:51:12.266 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T14:51:12.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:51:12.300 DEBUG:teuthology.orchestra.run.smithi111:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T14:51:12.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:51:12.315 DEBUG:teuthology.orchestra.run.smithi114:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T14:51:12.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T14:51:12.329 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T14:51:12.340 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T14:51:12.341 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764/remote/smithi040 2024-09-16T14:51:12.341 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T14:51:12.377 DEBUG:teuthology.misc:Transferring archived files from smithi111:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764/remote/smithi111 2024-09-16T14:51:12.378 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T14:51:12.411 DEBUG:teuthology.misc:Transferring archived files from smithi114:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_22:00:02-rados-reef-distro-default-smithi/7906764/remote/smithi114 2024-09-16T14:51:12.412 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T14:51:12.445 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T14:51:12.445 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T14:51:12.448 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T14:51:12.450 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T14:51:12.497 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T14:51:12.508 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T14:51:12.508 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T14:51:12.630 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T14:51:12.630 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T14:51:12.633 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T14:51:12.635 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T14:51:12.646 INFO:teuthology.orchestra.run.smithi040.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 14:51 /home/ubuntu/cephtest 2024-09-16T14:51:12.649 INFO:teuthology.orchestra.run.smithi111.stdout: 266054 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 14:51 /home/ubuntu/cephtest 2024-09-16T14:51:12.650 INFO:teuthology.orchestra.run.smithi114.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 14:51 /home/ubuntu/cephtest 2024-09-16T14:51:12.650 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T14:51:12.661 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T14:51:12.686 INFO:teuthology.run:Summary data: description: rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} duration: 595.6559903621674 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-16T14:51:12.686 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T14:51:12.767 INFO:teuthology.run:pass